Python - ImportError：用于[module]的加载程序无法处理__mp_main__

Question

我在网上搜索过，无法找到遇到同样问题的任何人。

我正在创建一个使用烧瓶作为后端的文件复制器。复制器接收上传的文件并在导出文件之前屏蔽文件中的一些数据。它为用户指定的数量执行此操作。为了加快复制速度，我使用了python的多处理库来加速这个过程，并使它在macOS中工作。但是，当我尝试使用Windows 2008 Server在虚拟机上运行它时，我遇到了一个ImportError。

Traceback (most recent call last): 
File "<string>", line 1, in <module> 
File "C:\Program Files (x86)\Python36-32\lib\multiprocessing\spawn.py", line 105, in spawn_main 
    exitcode = _main(fd) 
File "C:\Program Files (x86)\Python36-32\lib\multiprocessing\spawn.py", line 114, in _main 
    prepare(preparation_data) 
File "C:\Program Files (x86)\Python36-32\lib\multiprocessing\spawn.py", line 223, in prepare 
    _fixup_main_from_name(data['init_main_from_name']) 
File "C:\Program Files (x86)\Python36-32\lib\multiprocessing\spawn.py", line 249, in _fixup_main_from_name 
alter_sys=True) 
File "C:\Program Files (x86)\Python36-32\lib\runpy.py", line 205, in run_module 
    return _run_module_code(code, init_globals, run_name, mod_spec) 
File "C:\Program Files (x86)\Python36-32\lib\runpy.py", line 96, in _run_module_code 
    mod_name, mod_spec, pkg_name, script_name) 
File "C:\Program Files (x86)\Python36-32\lib\runpy.py", line 85, in _run_code 
    exec(code, run_globals) 
File "C:\Users\esglabuser\Desktop\asib-de-identifier\server\server.py", line 22, in <module> 
    app = Flask(__name__) 
File "C:\Program Files (x86)\Python36-32\lib\site-packages\flask\app.py", line 357, in __init__ 
    instance_path = self.auto_find_instance_path() 
File "C:\Program Files (x86)\Python36-32\lib\site-packages\flask\app.py", line 662, in auto_find_instance_path 
    prefix, package_path = find_package(self.import_name) 
File "C:\Program Files (x86)\Python36-32\lib\site-packages\flask\helpers.py", line 766, in find_package 
    filename = loader.get_filename(root_mod_name) 
File "<frozen importlib._bootstrap_external>", line 398, in _check_name_wrapper 

ImportError: loader for server.server cannot handle __mp_main__ 

尝试调试后，该函数调用进程启动，然后引发错误。

# function starts processes to replicate file 'amount' times 
proc_queue = Queue() 
proc_lock = Lock() 

proc_queue.put((file_path, file_type, 0, amount)) 

# ----- Create Worker Processes ----- # 
try: 
    num_cores = cpu_count() 
except NotImplementedError: 
    num_cores = 8 

workers = [Process(target=replicate, args=(proc_queue, proc_lock)) for _ in range(num_cores)] 
for p in workers: 
    p.daemon = True 
    p.start() # <--- When the ImportError is raised 

# ----- Join all workers ----- # 
for proc in workers: 
    proc.join() 

复制功能的工作原理如下：

def replicate(queue, lock): 
    while True: 
     # ----- CRITICAL SECTION START ----- # 
     with lock: 
      file_path, file_type, num_times, amount = queue.get() 

      if num_times >= amount: 
       queue.put((file_path, file_type, num_times, amount)) 
       break 
      else: 
       num_times += 1 

      queue.put((file_path, file_type, num_times, amount)) 
     # ----- CRITICAL SECTION END ----- # 

    # Do some data transformation with file 
    # Export new file 

我发现主题上与Windows和Mac系统如何处理孩子的过程和传递参数的不同，但我传递的所有参数，通过给孩子实例化过程。是否还有其他东西，我错过了两个操作系统之间的差异，或者是我的实现导致问题。

任何人有什么想法/想法可能会出错？

Answer 1

这可能是由于在Windows上，fork processus与你在Posix系统（如Linux或OSX）上可以找到的有点不同。

正如你可以看到here，你应该保护你的方法if __name__=="__main__":以避免副作用。

Safe importing of main module

Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process).

For example, using the spawn or forkserver start method running the following module would fail with a RuntimeError:

from multiprocessing import Process 

def foo(): 
    print('hello') 

p = Process(target=foo) 
p.start() 

Instead one should protect the “entry point” of the program by using if name == 'main': as follows:

from multiprocessing import Process, freeze_support, set_start_method 

def foo(): 
    print('hello') 

if __name__ == '__main__': 
    freeze_support() 
    set_start_method('spawn') 
    p = Process(target=foo) 
    p.start()