蟒蛇符号化的UnicodeDecodeError

python
nlp

2016-05-18 84 views 2 likes

我试图来标记一些文件，但我有这个错误蟒蛇符号化的UnicodeDecodeError

UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 6: ordinal not in range(128)

import nltk 
import pandas as pd 

df = pd.DataFrame(pd.read_csv('status2.csv')) 
documents = df['status'] 

result = [nltk.word_tokenize(sent) for sent in documents]

我认为这是unicode的问题，所以我说

documents = unicode(documents, 'utf-8')

另一个错误

TypeError: coercing to Unicode: need string or buffer, Series found

print documents 

1  Brandon Cachia ,All I know is that,you're so n... 
2  Melissa Zejtunija:HAM AND CHEESE BIEX INI??? *... 
3       .........Where is my mind????? 
4  Having a philosophical discussion with Trudy D...

来源

2016-05-18 Lucia

回答

unicode对字符串或字节进行操作，但documents是一个熊猫系列。

可能：

result = [nltk.word_tokenize(unicode(sent, 'utf-8')) for sent in documents]

来源

2016-05-18 05:11:53 Neapolitan

蟒蛇符号化的UnicodeDecodeError

回答

相关问题