2013-09-21 108 views
2

我正在使用scikit-learn解决大量标签的多标签分类问题。来自该项目的核心开发人员之一(larsmans)的followed the ideas。它给了我一个运行时警告,然后最终出现内存错误。带特征选择的多标签分类(scikit-learn)

>>> clf = Pipeline([('chi2', SelectKBest(chi2, k=1000)),('svm', LinearSVC())]) 
>>> 
>>> multiclf = OneVsRestClassifier(clf, n_jobs=-1) 
>>> 
>>> multiclf.fit(Xtr, ytr) 
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: 
RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: 
RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157:    
RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 

然后多个这样的警告后,它失败:下面

Process PoolWorker-21: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Exception in thread Thread-1: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 808, in __bootstrap_inner 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 761, in run 
    self.__target(*self.__args, **self.__kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 325, in _handle_workers 
    pool._maintain_pool() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 229, in _maintain_pool 
    self._repopulate_pool() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 222, in _repopulate_pool 
    w.start() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 130, in start 
    self._popen = Popen(self) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/forking.py", line 121, in __init__ 
    self.pid = os.fork() 
OSError: [Errno 12] Cannot allocate memory 

/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
Process PoolWorker-22: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-23: 
Process PoolWorker-30: 
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
Process PoolWorker-31: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-28: 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-32: 
Process PoolWorker-29: 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-27: 
Process PoolWorker-26: 
Traceback (most recent call last): 
Process PoolWorker-25: 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self.run() 
    self.run() 
    self.run() 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
    return recv() 
    return recv() 
MemoryError 
MemoryError 
MemoryError 
    return recv() 
MemoryError 
    return recv() 
    return recv() 
MemoryError 
MemoryError 
    return recv() 
MemoryError 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
Process PoolWorker-24: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, 'k'. 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-1: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-2: 
Traceback (most recent call last): 
Process PoolWorker-3: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-4: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-6: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
Process PoolWorker-5: 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-7: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: unpickling stack underflow 
Process PoolWorker-8: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, ''. 
Process PoolWorker-10: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, '�'. 
Process PoolWorker-21: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Exception in thread Thread-1: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 808, in __bootstrap_inner 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 761, in run 
    self.__target(*self.__args, **self.__kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 325, in _handle_workers 
    pool._maintain_pool() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 229, in _maintain_pool 
    self._repopulate_pool() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 222, in _repopulate_pool 
    w.start() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 130, in start 
    self._popen = Popen(self) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/forking.py", line 121, in __init__ 
    self.pid = os.fork() 
OSError: [Errno 12] Cannot allocate memory 

/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
Process PoolWorker-22: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-23: 
Process PoolWorker-30: 
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
Process PoolWorker-31: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-28: 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-32: 
Process PoolWorker-29: 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-27: 
Process PoolWorker-26: 
Traceback (most recent call last): 
Process PoolWorker-25: 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self.run() 
    self.run() 
    self.run() 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
    return recv() 
    return recv() 
MemoryError 
MemoryError 
MemoryError 
    return recv() 
MemoryError 
    return recv() 
    return recv() 
MemoryError 
MemoryError 
    return recv() 
MemoryError 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
Process PoolWorker-24: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, 'k'. 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-1: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-2: 
Traceback (most recent call last): 
Process PoolWorker-3: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-4: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-6: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
Process PoolWorker-5: 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-7: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: unpickling stack underflow 
Process PoolWorker-8: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, ''. 
Process PoolWorker-10: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, '�'. 

回答

0

Seing所有MemoryErrors错误转储,你可能已经耗尽内存。尝试用n_jobs = 1替换n_jobs = -1。这将控制产生的进程数量(从而控制使用的内存量)。

如果可行,只要你没有耗尽内存,即n_jobs = 2,n_jobs = 3等等,你就可以增加n_jobs。