Python的矩阵分解

Question

我有蟒蛇下面的代码

############################################################################### 

""" 
@INPUT: 
    R  : a matrix to be factorized, dimension N x M 
    P  : an initial matrix of dimension N x K 
    Q  : an initial matrix of dimension M x K 
    K  : the number of latent features 
    steps : the maximum number of steps to perform the optimisation 
    alpha : the learning rate 
    beta : the regularization parameter 
@OUTPUT: 
    the final matrices P and Q 
""" 
def matrix_factorization(R, P, Q, K, steps=5000, alpha=0.0002, beta=0.02): 
    Q = Q.T 
    for step in xrange(steps): 
     for i in xrange(len(R)): 
      for j in xrange(len(R[i])): 
       if R[i][j] > 0: 
        eij = R[i][j] - numpy.dot(P[i,:],Q[:,j]) 
        for k in xrange(K): 
         P[i][k] = P[i][k] + alpha * (2 * eij * Q[k][j] - beta * P[i][k]) 
         Q[k][j] = Q[k][j] + alpha * (2 * eij * P[i][k] - beta * Q[k][j]) 
     eR = numpy.dot(P,Q) 
     e = 0 
     for i in xrange(len(R)): 
      for j in xrange(len(R[i])): 
       if R[i][j] > 0: 
        e = e + pow(R[i][j] - numpy.dot(P[i,:],Q[:,j]), 2) 
        for k in xrange(K): 
         e = e + (beta/2) * (pow(P[i][k],2) + pow(Q[k][j],2)) 
     if e < 0.001: 
      break 
    return P, Q.T 

############################################################################### 

这个代码链接如下： http://www.quuxlabs.com/blog/2010/09/matrix-factorization-a-simple-tutorial-and-implementation-in-python/

代码工作正常为小矩阵，但我有两个大型矩阵P（15715 ，203）和Q（203,16384），当我尝试P上执行该代码和Q它给了我下面的错误

K=203 

matrix_factorization(R, P, Q, K) 
--------------------------------------------------------------------------- 
TypeError         Traceback (most recent call last) 
<ipython-input-3-00b8211f2507> in <module>() 
----> 1 matrix_factorization(R, P, Q, K) 

/Users/ajinkyachandrakantbobade/Desktop/random_choicefile/trial.py in matrix_factorization(R, P, Q, K, steps, alpha, beta) 
    52    for j in xrange(len(R[i])): 
    53     if R[i][j] > 0: 
---> 54      eij = R[i][j] - numpy.dot(P[i,:],Q[:,j]) 
    55      for k in xrange(K): 
    56       P[i][k] = P[i][k] + alpha * (2 * eij * Q[k][j] - beta * P[i][k]) 

/Users/ajinkyachandrakantbobade/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/pandas/core/frame.pyc in __getitem__(self, key) 
    1967    return self._getitem_multilevel(key) 
    1968   else: 
-> 1969    return self._getitem_column(key) 
    1970 
    1971  def _getitem_column(self, key): 

/Users/ajinkyachandrakantbobade/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/pandas/core/frame.pyc in _getitem_column(self, key) 
    1974   # get column 
    1975   if self.columns.is_unique: 
-> 1976    return self._get_item_cache(key) 
    1977 
    1978   # duplicate columns & possible reduce dimensionality 

/Users/ajinkyachandrakantbobade/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/pandas/core/generic.pyc in _get_item_cache(self, item) 
    1087   """ return the cached item, item represents a label indexer """ 
    1088   cache = self._item_cache 
-> 1089   res = cache.get(item) 
    1090   if res is None: 
    1091    values = self._data.get(item) 

TypeError: unhashable type 

灿任何人请帮助关于这个错误？

Answer 1

您试图乘法的矩阵大小过大，并且您没有足够的内存来完成计算。有几件事情，可以帮助：

• 得到更多的内存
• 如果你的基体中含有大量的0，你可以尝试使用稀疏矩阵。它们是常规矩阵，只存储表示值不等于零的元素。 The documentation of scipy会给你一些关于这方面的信息。
• 似乎你已经在使用Python的64位，但如果没有这样的工作比Python的32位

基于目录名（即Canopy_64）