将Sklearn的CountVectorizer的词汇表设置为词组的短语

Question

你好我一直在玩弄使用scikit-learn的文本分析，并且我有使用CountVectorizer来检测文档是否包含一组关键字和短语的想法。

我知道我们可以做到这一点：

words = ['cat', 'dog', 'walking'] 
example = ['I was walking my dog and cat in the park'] 
vect = CountVectorizer(vocabulary=words) 
dtm = vect.fit_transform(example) 
>>> pd.DataFrame(dtm.toarray(), columns=vect.get_feature_names()) 

...

cat dog walking 
    1 1  1 

我不知道是否有可能调整的东西，这样我可以用词组只是个别的，而不是词语

从上面的例子：

phrases = ['cat in the park', 'walking my dog'] 
example = ['I was walking my dog and cat in the park'] 
vect = CountVectorizer(vocabulary=phrases) 
dtm = vect.fit_transform(example) 
>>> pd.DataFrame(dtm.toarray(), columns=vect.get_feature_names()) 
... 

     cat in the park walking my dog 
      1     1 

现在使用的短语代码只是输出

cat in the park walking my dog 
    0     0 

预先感谢您！

Answer 1

试试这个：完美的例子

In [104]: lens = [len(x.split()) for x in phrases] 

In [105]: mn, mx = min(lens), max(lens) 

In [106]: vect = CountVectorizer(vocabulary=phrases, ngram_range=(mn, mx)) 

In [107]: dtm = vect.fit_transform(example) 

In [108]: pd.DataFrame(dtm.toarray(), columns=vect.get_feature_names()) 
Out[108]: 
    cat in the park walking my dog 
0    1    1 

In [109]: print(mn, mx) 
3 4