我试图写一个网页被称为具有下列参数的通用履带:传递使用XPath作为参数传递给Scrapy
- 允许域
- URL被抓取
- 的XPath提取网页内的价格
URL和允许的域参数似乎工作正常,但我无法让xPath参数工作。
我猜我需要声明一个变量来保持它正确,因为其他两个参数被分配给现有的类元素。
这里是我的蜘蛛:
import scrapy
from Spotlite.items import SpotliteItem
class GenericSpider(scrapy.Spider):
name = "generic"
def __init__(self, start_url=None, allowed_domains=None, xpath_string=None, *args, **kwargs):
super(GenericSpider, self).__init__(*args, **kwargs)
self.start_urls = ['%s' % start_url]
self.allowed_domains = ['%s' % allowed_domains]
xpath_string = ['%s' % xpath_string]
def parse(self, response):
self.logger.info('Hi, this is an item page! %s', response.url)
item = SpotliteItem()
item['url'] = response.url
item['price'] = response.xpath(xpath_string).extract()
return item
我得到以下错误:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/ubuntu/spotlite/spotlite/spiders/generic.py", line 23, in parse
item['price'] = response.xpath(xpath_string).extract()
NameError:全局名称 'xpath_string' 没有定义
任何援助将不胜感激!
感谢,
迈克尔