无法看清scrapy

Question

甩统计当我运行在scrapy教程提供的例子，我可以看到在标准输出打印日志：

2014-07-10 16:08:21+0100 [pubs] INFO: Spider opened 
2014-07-10 16:08:21+0100 [pubs] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 
2014-07-10 16:08:21+0100 [pubs] INFO: Closing spider (finished) 
2014-07-10 16:08:21+0100 [pubs] INFO: Dumping Scrapy stats: 
{'downloader/request_bytes': 471, 
'downloader/request_count': 2, 
'downloader/request_method_count/GET': 2, 
'downloader/response_bytes': 3897, 
'downloader/response_count': 2, 
'downloader/response_status_count/200': 1, 
'downloader/response_status_count/302': 1, 
'finish_reason': 'finished', 
'finish_time': datetime.datetime(2014, 7, 10, 15, 8, 21, 970741), 
'item_scraped_count': 1, 
'response_received_count': 1, 
'scheduler/dequeued': 2, 
'scheduler/dequeued/memory': 2, 
'scheduler/enqueued': 2, 
'scheduler/enqueued/memory': 2, 
'start_time': datetime.datetime(2014, 7, 10, 15, 8, 21, 584373)} 
2014-07-10 16:08:21+0100 [pubs] INFO: Spider closed (finished) 

然而，当我更改设置“FEED_URI”导出结果文件到S3，我没有看到任何地方的统计数据。我试过打印crawler.stats.spider_stats，但它仍然是空的。有任何想法吗？

Answer 1

即使'LOG_ENABLED'和'DUMP_STATS'设置为true，我也无法让scrapy转储统计信息。然而，我发现一种解决方法，通过在我的反应器模拟的末尾添加这行代码手动倾倒统计：

log.msg("Dumping Scrapy stats:\n" + pprint.pformat(crawler.stats.get_stats()))