Bypass“Array is too big”python error

我有一个csv文件，其中包含超过200 000行的气象数据。当我想将数据与matplotlib建模，出现此错误：Bypass“Array is too big”python error

Traceback (most recent call last): 
    File "try4.py", line 19, in <module> 
    X,Y = meshgrid(data_x,data_y) 
    File "C:\Python27\lib\site-packages\numpy\lib\function_base.py", line 3378, in meshgrid 
    mult_fact = np.ones(shape, dtype=int) 
    File "C:\Python27\lib\site-packages\numpy\core\numeric.py", line 148, in ones 
    a = empty(shape, dtype, order) 
    ValueError: array is too big.

我发现有5000行最大一个文件进行处理。如何绕过错误以处理200000行的所有文件？这里是我的代码：

import numpy as np 
import matplotlib.pyplot as plt 
import matplotlib.mlab as mlab 
from pylab import * 


# read CSV as a numpy array 
data = mlab.csv2rec('datasets/mix.csv') 

# print CSV file headers 
print data.dtype.names 

# load columns as vectors 
data_x = data['longitude'] 
data_y = data['latitude'] 
data_u = data['x'] 
data_v = data['y'] 

X,Y = meshgrid(data_x,data_y) 
U = cos(data_u) 
V = sin(data_v) 


# plot raw data 
Q = quiver(X, Y, U, V, units='width') 
qk = quiverkey(Q, 0.5, 0.92, 2, '.', labelpos='W', 
       fontproperties={'weight': 'bold'}) 
title('Current Surface') 

plt.show()

来源

2013-06-26 So4ne

数组元素看起来像什么？这个答案：http://stackoverflow.com/questions/13652650/matplotlib-plot-array-size-limit似乎认为没有一个明显的数组限制（如果这样至少在1m +元素） – ydaetskcoR

@ydaetskcoR在这里是一个示例 //头X，Y，经纬度，VOGRD_201304100000 //数据292,1,000000，-50.530000,0和200000其他类似的行 – So4ne

您能否提供更多信息？你正在使用哪种'matplotlib'调用等。 – tiago

为什么您使用meshgrid（doc）？它很好地生成了一个200k×200k的数组，它与你的u和v数据的尺寸不匹配。我想你想这样做

import numpy as np 
import matplotlib.pyplot as plt 
import matplotlib.mlab as mlab 
from pylab import * 


# read CSV as a numpy array 
data = mlab.csv2rec('datasets/mix.csv') 

# print CSV file headers 
print data.dtype.names 

# load columns as vectors 
data_x = data['longitude'] 
data_y = data['latitude'] 
data_u = data['x'] 
data_v = data['y'] 

U = cos(data_u) 
V = sin(data_v) 


# plot raw data 
Q = quiver(data_x, data_y, U, V, units='width') 
qk = quiverkey(Q, 0.5, 0.92, 2, '.', labelpos='W', 
       fontproperties={'weight': 'bold'}) 
title('Current Surface')

来源

2013-06-26 09:47:46 tacaswell

你是对的，它完美的作品。谢谢 – So4ne

Bypass“Array is too big”python error

回答

相关问题