Python numpy.fft在子进程中运行时非常慢（10倍慢）

Question

我发现numpy.fft.fft（及其变体）在后台运行时非常慢。下面是一个例子就是我在谈论

import numpy as np 
import multiprocessing as mproc 
import time 
import sys 

# the producer function, which will run in the background and produce data 
def Producer(dataQ): 
    numFrames = 5 
    n = 0 
    while n < numFrames: 
     data = np.random.rand(3000, 200) 
     dataQ.put(data) # send the datta to the consumer 
     time.sleep(0.1) # sleep for 0.5 second, so we dont' overload CPU 
     n += 1    

# the consumer function, which will run in the backgrounnd and consume data from the producer 
def Consumer(dataQ): 
    while True: 
     data = dataQ.get() 
     t1 = time.time() 
     fftdata = np.fft.rfft(data, n=3000*5) 
     tDiff = time.time() - t1 
     print("Elapsed time is %0.3f" % tDiff) 
     time.sleep(0.01) 
     sys.stdout.flush() 

# the main program if __name__ == '__main__': is necessary to prevent this code from being run 
# only when this program is started by user 
if __name__ == '__main__':  
    data = np.random.rand(3000, 200) 
    t1 = time.time() 
    fftdata = np.fft.rfft(data, n=3000*5, axis=0) 
    tDiff = time.time() - t1 
    print("Elapsed time is %0.3f" % tDiff) 

    # generate a queue for transferring data between the producedr and the consumer 
    dataQ = mproc.Queue(4) 

    # start up the processoso 
    producerProcess = mproc.Process(target=Producer, args=[dataQ], daemon=False) 
    consumerProcess = mproc.Process(target=Consumer, args=[dataQ], daemon=False) 
    print("starting up processes") 

    producerProcess.start() 
    consumerProcess.start() 
    time.sleep(10) # let program run for 5 seconds 

    producerProcess.terminate() 
    consumerProcess.terminate() 

我的机器上它produes输出：在后台运行时

Elapsed time is 0.079 
starting up processes 
Elapsed time is 0.859 
Elapsed time is 0.861 
Elapsed time is 0.878 
Elapsed time is 0.863 
Elapsed time is 0.758 

正如你所看到的，它是大约慢10倍，和我无法弄清楚为什么会出现这种情况。 time.sleep（）调用应确保其他进程（主进程和生产者进程）在计算FFT时不做任何事情，因此它应该使用所有内核。我已经通过Windows任务管理器检查了CPU使用率，并且在单进程和多进程情况下调用numpy.fft.fft时似乎占用了大约25％。

任何人都有一个想法是怎么回事？

Answer 1

的主要问题是，在后台线程的FFT电话是：

fftdata = np.fft.rfft(data, n=3000*5) 

而不是：

fftdata = np.fft.rfft(data, n=3000*5, axis=0) 

这对我来说一切都不同。

还有其他一些值得注意的事情。为什么不让处理器自己处理这个问题，而不是到处都有time.sleep()？更进一步，而不是暂停主线程，你可以使用

consumerProcess.join() 

，然后有生产者进程运行dataQ.put(None)一旦完成加载数据，并打破了在消费过程中的循环，即：

def Consumer(dataQ): 
    while True: 
     data = dataQ.get() 
     if(data is None): 
      break 
     ...