无法将某些字段从一种方法传递到另一种方法

Question

我试图用python scrapy创建一个解析器，它应该从网页中提取一些字段。我在这里面临的问题是，我无法以正确的方式将参数传递给蜘蛛类中的“target_page”方法。我知道我将“标题”和“值”字段从“parse”方法传递给“target_page”的方式并不是pythonic。但是，我做到了，因为我真的不知道如何以正确的方式在“target_page”方法中传递上述字段。我该怎么做？提前致谢。

import scrapy 

class BrokerSpider(scrapy.Spider): 

    name = "brokersp" 
    start_urls = ["https://www.mql5.com/en/signals/mt4/page{0}".format(page_num) for page_num in range(1,10)] 

    def parse(self, response): 
     for links in response.css("div#signals-table div.signal"): 
      caption = links.css('span.name::text').extract_first() 
      value = links.css('div.col-price::text').extract_first() 
      link = links.css('a.signal-avatar::attr(href)').extract_first() 
      yield scrapy.Request(caption, value, url = link, callback = self.target_page) 

    def target_page(self, title, price, response): 
     jobber = response.css('div.header span a::text').extract_first() 
     profit = response.css('div.cell.total a.blue::text').extract_first() 
     yield {'title':title,'price':price,'jobber':jobber,'profit':profit} 

Answer 1

可以使用要求（documentation）的meta属性：

def parse(self, response): 
    for links in response.css("div#signals-table div.signal"): 
     caption = links.css('span.name::text').extract_first() 
     value = links.css('div.col-price::text').extract_first() 
     link = links.css('a.signal-avatar::attr(href)').extract_first() 
     request = scrapy.Request(url = link, callback = self.target_page) 
     request.meta['caption'] = caption 
     request.meta['value'] = value 
     yield request 

def target_page(self, response): 
    caption = response.meta['caption'] 
    value = response.meta['value'] 
    jobber = response.css('div.header span a::text').extract_first() 
    profit = response.css('div.cell.total a.blue::text').extract_first() 
    yield {'title':caption,'price':value,'jobber':jobber,'profit':profit}