python中concurrent.futures的async和requests-futures异步有什么区别？

Question

我用requests-futures来异步抓取网页，而且我的机器有多核，所以我也想同时抓多个网站，然后我试着用concurrent.futures，看来concurrent.futures也提供异步方法，所以什么是concurrent.futures'async和request-futures'异步之间的区别？如果它们相同，意味着我可以反对请求 - 期货？

Answer 1

requests-futures只是在concurrent.futures之上的一个非常小的包装。您可以通过查看source code（为了简洁，删除文档字符串）看到这一点：

from concurrent.futures import ThreadPoolExecutor 
from requests import Session 
from requests.adapters import DEFAULT_POOLSIZE, HTTPAdapter 

class FuturesSession(Session): 

    def __init__(self, executor=None, max_workers=2, *args, **kwargs): 
     super(FuturesSession, self).__init__(*args, **kwargs) 
     if executor is None: 
      executor = ThreadPoolExecutor(max_workers=max_workers) 
      # set connection pool size equal to max_workers if needed 
      if max_workers > DEFAULT_POOLSIZE: 
       adapter_kwargs = dict(pool_connections=max_workers, 
             pool_maxsize=max_workers) 
       self.mount('https://', HTTPAdapter(**adapter_kwargs)) 
       self.mount('http://', HTTPAdapter(**adapter_kwargs)) 

     self.executor = executor 

    def request(self, *args, **kwargs): 
     func = sup = super(FuturesSession, self).request 

     background_callback = kwargs.pop('background_callback', None) 
     if background_callback: 
      def wrap(*args_, **kwargs_): 
       resp = sup(*args_, **kwargs_) 
       background_callback(self, resp) 
       return resp 

      func = wrap 

     return self.executor.submit(func, *args, **kwargs) # This returns a concurrent.futures.Future 

当您使用requests-futures，你真的使用concurrent.futures.ThreadPoolExecutor，它返回一个concurrent.futures.Future当你submit给它的任务。如果您更方便地使用requests-futures提供的API来处理HTTP请求，可以坚持使用它，甚至可以使用由concurrent.futures模块提供的其他方法返回的对象。