2017-02-04 66 views
1

这是一条错误消息,我得到当我执行import theano:在Windows 10导入错误:动态模块没有定义模块输出功能(PyInit_m3d1cf20adb1014f04986e6a344a55bde)

ImportError: dynamic module does not define module export function (PyInit_m3d1cf20adb1014f04986e6a344a55bde)

我使用Python 3.5.2

我也对GitHub上的Theano的作者提出过这个问题。我在这里问,希望有人能够帮助我,即使他/她不知道Theano的内部。我怀疑这更多的是与Python 2与3

以下是完整的错误:

ERROR (theano.gpuarray): Could not initialize pygpu, support disabled 
Traceback (most recent call last): 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gpuarray\__init__.py", line 164, in <module> 
    use(config.device) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gpuarray\__init__.py", line 151, in use 
    init_dev(device) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gpuarray\__init__.py", line 66, in init_dev 
    avail = dnn.dnn_available(name) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gpuarray\dnn.py", line 174, in dnn_available 
    if not dnn_present(): 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gpuarray\dnn.py", line 157, in dnn_present 
    dnn_present.avail, dnn_present.msg = _dnn_check_version() 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gpuarray\dnn.py", line 130, in _dnn_check_version 
    v = version() 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gpuarray\dnn.py", line 316, in version 
    profile=False) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\compile\function.py", line 326, in function 
    output_keys=output_keys) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\compile\pfunc.py", line 486, in pfunc 
    output_keys=output_keys) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\compile\function_module.py", line 1795, in orig_function 
    defaults) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\compile\function_module.py", line 1661, in create 
    input_storage=input_storage_lists, storage_map=storage_map) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof\link.py", line 699, in make_thunk 
    storage_map=storage_map)[:3] 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof\vm.py", line 1063, in make_all 
    impl=impl)) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof\op.py", line 924, in make_thunk 
    no_recycling) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof\op.py", line 828, in make_c_thunk 
    output_storage=node_output_storage) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof\cc.py", line 1190, in make_thunk 
    keep_lock=keep_lock) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof\cc.py", line 1131, in __compile__ 
    keep_lock=keep_lock) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof\cc.py", line 1589, in cthunk_factory 
    key=key, lnk=self, keep_lock=keep_lock) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof\cmodule.py", line 1155, in module_from_key 
    module = lnk.compile_cmodule(location) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof\cc.py", line 1492, in compile_cmodule 
    preargs=preargs) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof\cmodule.py", line 2318, in compile_str 
    return dlimport(lib_filename) 
File "C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof\cmodule.py", line 302, in dlimport 
    rval = __import__(module_name, {}, {}, [module_name]) 
ImportError: dynamic module does not define module export function (PyInit_m3d1cf20adb1014f04986e6a344a55bde) 

据我所知,Theano编译一个CPP文件。以下是文件的编译后:

Directory of [..]\tmp04_uyjkc 

04/02/2017 16:41 <DIR>   . 
04/02/2017 16:41 <DIR>   .. 
04/02/2017 16:08     0 delete.me 
04/02/2017 16:08   108.834 m3d1cf20adb1014f04986e6a344a55bde.pyd 
04/02/2017 16:08    5.502 mod.cpp 
04/02/2017 16:41     0 ok.txt 
04/02/2017 16:08     0 __init__.py 
04/02/2017 16:08 <DIR>   __pycache__ 
      5 File(s)  114.336 bytes 

Directory of [..]\tmp04_uyjkc\__pycache__ 

04/02/2017 16:08 <DIR>   . 
04/02/2017 16:08 <DIR>   .. 
04/02/2017 16:08    239 __init__.cpython-35.pyc 
      1 File(s)   239 bytes 

这里的mod.cpp的内容:

#include <Python.h> 
#include <iostream> 
#include "theano_mod_helper.h" 
#include "cudnn.h" 
////////////////////// 
//// Support Code 
////////////////////// 

#if PY_MAJOR_VERSION >= 3 
#define PyInt_FromLong PyLong_FromLong 
#endif 


    namespace { 
    struct __struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde { 
     PyObject* __ERROR; 

     PyObject* storage_V1; 


     __struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde() { 
      // This is only somewhat safe because we: 
      // 1) Are not a virtual class 
      // 2) Do not use any virtual classes in the members 
      // 3) Deal with mostly POD and pointers 

      // If this changes, we would have to revise this, but for 
      // now I am tired of chasing segfaults because 
      // initialization code had an error and some pointer has 
      // a junk value. 
      memset(this, 0, sizeof(*this)); 
     } 
     ~__struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde(void) { 
      cleanup(); 
     } 

     int init(PyObject* __ERROR, PyObject* storage_V1) { 
      Py_XINCREF(storage_V1); 
      this->storage_V1 = storage_V1; 



      this->__ERROR = __ERROR; 
      return 0; 
     } 
     void cleanup(void) { 
      __label_1: 

double __DUMMY_1; 
__label_4: 

double __DUMMY_4; 

      Py_XDECREF(this->storage_V1); 
     } 
     int run(void) { 
      int __failure = 0; 

    PyObject* py_V1; 

     PyObject* V1; 

{ 

    py_V1 = Py_None; 
    {Py_XINCREF(py_V1);} 

     V1 = NULL; 

{ 
// Op class DnnVersion 

     V1 = PyTuple_Pack(2, PyInt_FromLong(CUDNN_VERSION), PyInt_FromLong(cudnnGetVersion())); 
     __label_3: 

double __DUMMY_3; 

} 
__label_2: 

    if (!__failure) { 

     assert(py_V1->ob_refcnt > 1); 
     Py_DECREF(py_V1); 
     py_V1 = V1 ? V1 : Py_None; 
     Py_INCREF(py_V1); 

     PyObject* old = PyList_GET_ITEM(storage_V1, 0); 
     {Py_XINCREF(py_V1);} 
     PyList_SET_ITEM(storage_V1, 0, py_V1); 
     {Py_XDECREF(old);} 
    } 

     Py_XDECREF(V1); 

    {Py_XDECREF(py_V1);} 

double __DUMMY_2; 

} 


     if (__failure) { 
      // When there is a failure, this code puts the exception 
      // in __ERROR. 
      PyObject* err_type = NULL; 
      PyObject* err_msg = NULL; 
      PyObject* err_traceback = NULL; 
      PyErr_Fetch(&err_type, &err_msg, &err_traceback); 
      if (!err_type) {err_type = Py_None;Py_INCREF(Py_None);} 
      if (!err_msg) {err_msg = Py_None; Py_INCREF(Py_None);} 
      if (!err_traceback) {err_traceback = Py_None; Py_INCREF(Py_None);} 
      PyObject* old_err_type = PyList_GET_ITEM(__ERROR, 0); 
      PyObject* old_err_msg = PyList_GET_ITEM(__ERROR, 1); 
      PyObject* old_err_traceback = PyList_GET_ITEM(__ERROR, 2); 
      PyList_SET_ITEM(__ERROR, 0, err_type); 
      PyList_SET_ITEM(__ERROR, 1, err_msg); 
      PyList_SET_ITEM(__ERROR, 2, err_traceback); 
      {Py_XDECREF(old_err_type);} 
      {Py_XDECREF(old_err_msg);} 
      {Py_XDECREF(old_err_traceback);} 
     } 
     // The failure code is returned to index what code block failed. 
     return __failure; 

     } 
    }; 
    } 


     static int __struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde_executor(__struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde *self) { 
      return self->run(); 
     } 

     static void __struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde_destructor(PyObject *capsule) { 
      __struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde *self = (__struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde *)PyCapsule_GetContext(capsule); 
      delete self; 
     } 

////////////////////// 
//// Functions 
////////////////////// 
static PyObject * instantiate(PyObject * self, PyObject *argtuple) { 
    assert(PyTuple_Check(argtuple)); 
    if (2 != PyTuple_Size(argtuple)){ 
    PyErr_Format(PyExc_TypeError, "Wrong number of arguments, expected 2, got %i", (int)PyTuple_Size(argtuple)); 
    return NULL; 
    } 
    __struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde* struct_ptr = new __struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde(); 
    if (struct_ptr->init(PyTuple_GET_ITEM(argtuple, 0),PyTuple_GET_ITEM(argtuple, 1)) != 0) { 
    delete struct_ptr; 
    return NULL; 
    } 
    PyObject* thunk = PyCapsule_New((void*)(&__struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde_executor), NULL, __struct_compiled_op_m3d1cf20adb1014f04986e6a344a55bde_destructor); 
    if (thunk != NULL && PyCapsule_SetContext(thunk, struct_ptr) != 0) { 
     PyErr_Clear(); 
     Py_DECREF(thunk); 
     thunk = NULL; 
    } 

    return thunk; } 

////////////////////// 
//// Module init 
////////////////////// 
static PyMethodDef MyMethods[] = { 
    {"instantiate", instantiate, METH_VARARGS, "undocumented"} , 
    {NULL, NULL, 0, NULL} 
}; 
static struct PyModuleDef moduledef = { 
     PyModuleDef_HEAD_INIT, 
     "m3d1cf20adb1014f04986e6a344a55bde", 
     NULL, 
     -1, 
     MyMethods, 
}; 

PyMODINIT_FUNC PyInit_m3d1cf20adb1014f04986e6a344a55bde(void) { 
    PyObject *m = PyModule_Create(&moduledef); 
    return m; 
} 

功能PyInit_m3d1cf20adb1014f04986e6a344a55bde似乎是在场,所以有什么问题呢?

编辑1

我复制的文件到d:\测试和管理,以重新编译使用由Theano使用的相同的确切命令.pyd文件(由于将procmon):

import os 

add_dir = r'C:\Users\Kiuhnm\Anaconda3\Library\mingw-w64\bin' 
os.environ['PATH'] += os.pathsep + add_dir 

cmd = r'""C:\Users\Kiuhnm\Anaconda3\Library\mingw-w64\bin\g++.exe" -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -Wl,-rpath, -march=haswell -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-pcommit -mno-mwaitx --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=haswell -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64 -I"C:\Users\Kiuhnm\Anaconda3\lib\site-packages\numpy\core\include" -I"C:\Users\Kiuhnm\Anaconda3\include" -I"C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof" -L"C:\Users\Kiuhnm\Anaconda3\libs" -L"C:\Users\Kiuhnm\Anaconda3" -o d:\test\m3d1cf20adb1014f04986e6a344a55bde.pyd d:\test\mod.cpp -lcudnn -lpython35"' 
os.system(cmd) 

我读该pyd文件只是DLL(至少在Windows上)。奇怪的是编译后的pyd文件似乎没有导出任何函数。也许我不明白Python如何找到导出函数:

D:\test>dumpbin /EXPORTS m3d1cf20adb1014f04986e6a344a55bde.pyd 
Microsoft (R) COFF/PE Dumper Version 12.00.40629.0 
Copyright (C) Microsoft Corporation. All rights reserved. 


Dump of file m3d1cf20adb1014f04986e6a344a55bde.pyd 

File Type: DLL 

    Section contains the following exports for m3d1cf20adb1014f04986e6a344a55bde.pyd 

    00000000 characteristics 
    5896198F time date stamp Sat Feb 04 19:12:31 2017 
     0.00 version 
      1 ordinal base 
      0 number of functions 
      0 number of names 

    ordinal hint RVA  name 


    Summary 

     1000 .CRT 
     1000 .bss 
     1000 .data 
     2000 .debug_abbrev 
     1000 .debug_aranges 
     1000 .debug_frame 
     9000 .debug_info 
     2000 .debug_line 
     3000 .debug_loc 
     1000 .debug_ranges 
     1000 .debug_str 
     1000 .edata 
     1000 .idata 
     1000 .pdata 
     1000 .rdata 
     1000 .reloc 
     2000 .text 
     1000 .tls 
     1000 .xdata 

回答

1

我终于解决了这个难题!

让我们来看看++再次命令G:

"C:\Users\Kiuhnm\Anaconda3\Library\mingw-w64\bin\g++.exe" -shared -g -O3 -fno-math-errno 
-Wno-unused-label -Wno-unused-variable -Wno-write-strings -Wl,-rpath, -march=haswell -mmmx 
-mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul 
-mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 
-msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx 
-mfxsr -mxsave -mxsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 
-mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma 
-mno-avx512vbmi -mno-clwb -mno-pcommit -mno-mwaitx --param l1-cache-size=32 --param 
l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=haswell 
-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64 
-I"C:\Users\Kiuhnm\Anaconda3\lib\site-packages\numpy\core\include" 
-I"C:\Users\Kiuhnm\Anaconda3\include" 
-I"C:\Users\Kiuhnm\Anaconda3\lib\site-packages\theano\theano\gof" 
-L"C:\Users\Kiuhnm\Anaconda3\libs" -L"C:\Users\Kiuhnm\Anaconda3" -o 
d:\test\m3d1cf20adb1014f04986e6a344a55bde.pyd d:\test\mod.cpp -lcudnn -lpython35 

你看到-Wl,-rpath,第二排?那么,事实证明,至少在Windows中,吞噬了它后面的一切!

-Wl,a,b,c,d用于将“a b c d”传递给链接程序,但在Windows中后面的逗号后跟空格是有问题的。也许它与Windows允许文件名空格有关。

删除有问题的部分后,输出文件变为40 KB,表明它现在包含mod.cpp的目标代码。最后,导入模块成功了,但现在我必须深入研究Theano代码,以查看错误,因为该命令是动态生成的。

编辑:这并不难。所有人要做的就是添加

[dnn] 
library_path=d:\whatever 

.theanorc。我使用虚拟目录,因为cuDNN的文件已经可以在我的系统上访问过(我必须更新PATH)。

如果您没有安装cuDNN,那么您应该永远不会遇到此问题。

现在-Wl,-rpath,应该是-Wl,-rpath,d:\whatever,但我没有检查。

只是为了保持完整性,相关的代码是在[...]Lib\site-packages\Theano\theano\gpuarray\dnn.py在行267:

def c_compile_args(self): 
    return ['-Wl,-rpath,' + config.dnn.library_path] 

config.dnn.library_path现在是 “d:\无所谓” 和一切都很好。

相关问题