2016-01-05 21 views
-1

我有一个主要的,看起来像下面这样:使用futures.concurrent带班

import gather_filings_per_filer 
import os 
from concurrent import futures 

def put_last_i(arg): 
    with open(os.path.dirname(os.path.abspath(__file__)) + '/last_i.txt','w') as f: 
      f.write(arg) 

def get_last_i(): 
    with open(os.path.dirname(os.path.abspath(__file__)) + '/last_i.txt','r') as f: 
      data = f.readlines() 
     return data[0] 

if __name__=='__main__': 
    i = int(get_last_i()) 
    g_f_p_f = gather_filings_per_filer.gather_filings_per() 
    jobs=[] 


    with futures.ThreadPoolExecutor(max_workers = 3) as executor: 
     my_d = dict((executor.submit(g_f_p_f.mt_main),j) for j in range(i,297085)) 

     for future in futures.as_completed(my_d): 
      print future 
      print future.exception() 

而且g_f_p_f.mt_main如下所示:

class gather_filings_per: 
     def mt_main(self, i): 
      print "Started:",i 
      self.w_m = wrap_mysql.wrap_mysql() 
      flag = True 
      #do stuff... 

给了我下面的结果:

<Future at 0x5dc2710 state=finished raised TypeError> 
mt_main() takes exactly 2 arguments (1 given) 
<Future at 0x5dc25b0 state=finished raised TypeError> 
mt_main() takes exactly 2 arguments (1 given) 

从我的角度来看mt_main只需要1个参数(自我不应该重新要求典型的self行为)。

在这里我错过了什么似乎出错?

回答

2

你是对的,你只需要提供一个额外的参数,超出了隐含的self。但你没有给。所以你是一个短。分裂的提交,使之在视觉上清晰:

my_d = dict( # Making a dict 
      # With key as result object from invoking `mt_main` with no additional arguments 
      (executor.submit(g_f_p_f.mt_main), 
      # And value j 
      j) 
      for j in range(i,297085)) 

也许你的意思是通过j作为参数?假设也应该是值,这将是:

# Note second argument to submit, which becomes only non-self argument to mt_main 
my_d = dict((executor.submit(g_f_p_f.mt_main, j),j) for j in range(i,297085)) 

或者简化titch,因为有concurrent.futures应该意味着你可以使用dict comprehensions(也分开submit呼叫从它与:价值更好的视觉解析):

my_d = {executor.submit(g_f_p_f.mt_main, j): j for j in range(i, 297085)}