7
我想通过scrapy下载图像。这里是我的不同的文件:在scrapy中下载图像
items.py
class DmozItem(Item):
title = Field()
image_urls = Field()
images = Field()
settings.py
BOT_NAME = 'tutorial'
SPIDER_MODULES = ['tutorial.spiders']
NEWSPIDER_MODULE = 'tutorial.spiders'
ITEM_PIPELINES = ['scrapy.contrib.pipeline.images.ImagesPipeline']
IMAGES= '/home/mayank/Desktop/sc/tutorial/tutorial'
蜘蛛
class DmozSpider(BaseSpider):
name = "wikipedia"
allowed_domains = ["wikipedia.org"]
start_urls = [
"http://en.wikipedia.org/wiki/Pune"
]
def parse(self, response):
hxs = HtmlXPathSelector(response)
items = []
images=hxs.select('//a[@class="image"]')
for image in images:
item = DmozItem()
link=image.select('@href').extract()[0]
link = 'http://en.wikipedia.com'+link
item['image_urls']=link
items.append(item)
尽管所有这些设置我在我的流水线没有得到激活。请帮助。我是这个框架的新手。
您是否安装了PIL(Python Imaging Library)?这是图像下载的先决条件:http://doc.scrapy.org/en/latest/topics/images.html – Talvalin
你怎么知道管道没有被激活?你可以包含一些日志输出,如:'2013-04-16 16:40:31-0500 [scrapy] DEBUG:启用项目管道:ImagesPipeline'。 –