在Python中的多个进程之间的同步

我有一个python应用程序，它产生了一个单独的进程来完成一些工作（由于GIL（全局解释器锁），我遇到了使用线程的性能问题）。现在我在Python中使用什么方法来跨进程同步共享资源？在Python中的多个进程之间的同步

我将数据移入队列，并且产生进程在接收到来自该队列的数据时完成工作。但我需要能够保证数据以有序的方式出来，与复制的顺序相同，所以我需要保证任何时候只有一个进程可以从队列中读/写。我该如何做到最好？

谢谢，罗恩

来源

2013-05-20 cerr

http://en.wikipedia.org/wiki/Semaphore_(programming）？ – Noelkd

Noelkd，谢谢 - 我发现这个：http://docs.python.org/release/2.4.2/lib/semaphore-examples.html会这样做吗？ – cerr

是的，这是正确的应该是一些更新的文档。可能写一个快速的例子，如果没有人也击败了我！ – Noelkd

我想你需要一个信号量，检查此示例代码：

import threading 
import datetime 


class ThreadClass(threading.Thread): 
    def run(self): 
     now = datetime.datetime.now() 
     pool.acquire() 
     print "%s says hello, World! at time: %s" % (self.getName(),now) 
     pool.release() 


pool = threading.BoundedSemaphore(value=1) 


for i in range(10): 
     t = ThreadClass() 
     t.start()

有这样的输出：

Thread-1 says hello, World! at time: 2013-05-20 18:57:47.609000 
Thread-2 says hello, World! at time: 2013-05-20 18:57:47.609000 
Thread-3 says hello, World! at time: 2013-05-20 18:57:47.609000 
Thread-4 says hello, World! at time: 2013-05-20 18:57:47.609000 
Thread-5 says hello, World! at time: 2013-05-20 18:57:47.609000 
Thread-6 says hello, World! at time: 2013-05-20 18:57:47.609000 
Thread-7 says hello, World! at time: 2013-05-20 18:57:47.609000 
Thread-8 says hello, World! at time: 2013-05-20 18:57:47.609000 
Thread-9 says hello, World! at time: 2013-05-20 18:57:47.609000 
Thread-10 says hello, World! at time: 2013-05-20 18:57:47.609000

凡为：

import threading 
import datetime 


class ThreadClass(threading.Thread): 
    def run(self): 
     now = datetime.datetime.now() 
     print "%s says hello, World! at time: %s" % (self.getName(),now) 




for i in range(10): 
     t = ThreadClass() 
     t.start()

有这样的输出：

Thread-1 says hello, World! at time: 2013-05-20 18:58:05.531000Thread-2 says hello, World! at time: 2013-05-20 18:58:05. 
531000 

Thread-4 says hello, World! at time: 2013-05-20 18:58:05.531000Thread-3 says hello, World! at time: 2013-05-20 18:58:05 
.531000 

Thread-6 says hello, World! at time: 2013-05-20 18:58:05.531000Thread-5 says hello, World! at time: 2013-05-20 18:58:05 
.531000 

Thread-8 says hello, World! at time: 2013-05-20 18:58:05.531000Thread-7 says hello, World! at time: 2013-05-20 18:58:05 
.531000 

Thread-10 says hello, World! at time: 2013-05-20 18:58:05.531000Thread-9 says hello, World! at time: 2013-05-20 18:58:0 
5.531000

因此，与Python中的BoundedSemaphore可以确保之前有人写你的队列，他们必须有信号量。这并不能确保您的结果以正确的顺序添加到队列中。

编辑：

如果你要做到这一点，并维持秩序你会需要这样的事：

import multiprocessing 
import datetime 
import random 
import time 

def funfun(number): 
    time.sleep(random.randint(0,10)) 
    now = datetime.datetime.now() 
    return "%s says hello, World! at time: %s" % (number,now) 

if __name__ == "__main__": 
    pool = multiprocessing.Pool(10) 
    for item in pool.imap(funfun,[i for i in range(10)]): 
     print item

它打印：

0 says hello, World! at time: 2013-05-21 00:38:48.546000 
1 says hello, World! at time: 2013-05-21 00:38:55.562000 
2 says hello, World! at time: 2013-05-21 00:38:47.562000 
3 says hello, World! at time: 2013-05-21 00:38:51.578000 
4 says hello, World! at time: 2013-05-21 00:38:50.578000 
5 says hello, World! at time: 2013-05-21 00:38:48.593000 
6 says hello, World! at time: 2013-05-21 00:38:52.593000 
7 says hello, World! at time: 2013-05-21 00:38:48.593000 
8 says hello, World! at time: 2013-05-21 00:38:50.593000 
9 says hello, World! at time: 2013-05-21 00:38:51.609000

所以用这个你可以只以正确的顺序附加到队列中，并且作业将等待轮到他们加入队列。

来源

2013-05-20 18:01:34 Noelkd

在Python中的多个进程之间的同步

回答

相关问题