我怎么能得到多个进程中的共享目录时，按ctrl + c

Question

我正在使用多处理模块来做我的程序中的并行处理，我想获得多进程之间的共享字典对象，我可以做到当多进程是正常关闭的，但按CTRL + C时无法得到它，我怎么能实现我的目标？ 我的代码如下

#!/usr/bin/python 
from multiprocessing import Process, Manager, Pool 
import os 
import signal 
import time 

def init_worker(): 
    signal.signal(signal.SIGINT, signal.SIG_IGN) 

def run_worker(i,my_d): 
    print 'Work Started: %d %d' % (os.getpid(), i) 
    for j in range(5): 
     print j 
     tmp1 = str(i) + str(j) 
     my_d[tmp1] = j 
     time.sleep(1) 

def main(): 
    print "Initializng 3 workers" 
    pool = Pool(3, init_worker) 

    manager = Manager() 
    my_d = manager.dict() 
    try: 
     for i in range(3): 
      pool.apply_async(run_worker,args=(i,my_d)) 
     pool.close() 
     pool.join() 
     print my_d 
# When process is closed normally, I could get the my_d successfully 

    except KeyboardInterrupt: 
     print "Caught KeyboardInterrupt, terminating workers" 
     pool.terminate() 
     pool.join() 
     print my_d 
#When process is closed by Ctrl+C, couldn't I get the my_d ? 

if __name__ == "__main__": 
    main() 

Answer 1

你需要一个共享的字典，从Manager -object的多。

在这里看到这个类似的问题（第一个答案）： Python multiprocessing: How do I share a dict among multiple processes?

Answer 2

看，当你中断父进程所产生的错误：

Caught KeyboardInterrupt, terminating workers 
<DictProxy object, typeid 'dict' at 0x801abe150; '__str__()' failed> 

尝试改变print "Caught KeyboardInterrupt, terminating workers"到print len(my_d)，你可以看到详细地发生了什么。请注意，这是你尝试之前和终止/参加工人的池：

Traceback (most recent call last): 
    File "manager-test.py", line 39, in <module> 
    main() 
    File "manager-test.py", line 33, in main 
    print len(my_d) 
    File "<string>", line 2, in __len__ 
    File "/usr/local/lib/python2.7/multiprocessing/managers.py", line 755, in _callmethod 
    self._connect() 
    File "/usr/local/lib/python2.7/multiprocessing/managers.py", line 742, in _connect 
    conn = self._Client(self._token.address, authkey=self._authkey) 
    File "/usr/local/lib/python2.7/multiprocessing/connection.py", line 169, in Client 
    c = SocketClient(address) 
    File "/usr/local/lib/python2.7/multiprocessing/connection.py", line 293, in SocketClient 
    s.connect(address) 
    File "/usr/local/lib/python2.7/socket.py", line 224, in meth 
    return getattr(self._sock,name)(*args) 
socket.error: [Errno 2] No such file or directory 

当中断主程序，从子进程管理器连接将被打破。这使经理（及其管理的对象）处于不可用状态。从管理器到子进程的套接字连接不再工作，所以代理无法获取数据。

如果你想在不丢失数据的情况下中断长时间运行的进程，我认为你应该更温和一些。就像这样：

import select 
import sys 

print 'Type q<enter> it you want to quit...' 
while True: 
    r, foo, bla = select.select([sys.stdin], [], [], 1) 
    if len(r): 
     what = sys.stdin.readline() 
     if 'q' in what: 
      print 'bye!' 
      break; 
    # E.g. check on the progress of your calculation here 
# Close and join the pool here, and do other clean-up.