mergnig名词短语块的POS标签

Question

我的问题类似于这个question。在spacy中，我可以分别进行词性标注和名词短语标识，例如

import spacy 
nlp = spacy.load('en') 
sentence = 'For instance , consider one simple phenomena : 
      a question is typically followed by an answer , 
      or some explicit statement of an inability or refusal to answer .' 
token = nlp(sentence) 
token_tag = [(word.text, word.pos_) for word in token] 

输出的样子：

[('For', 'ADP'), 
('instance', 'NOUN'), 
(',', 'PUNCT'), 
('consider', 'VERB'), 
('one', 'NUM'), 
('simple', 'ADJ'), 
('phenomena', 'NOUN'), 
...] 

对于名词短语或块，我可以得到noun_chunks这是词的一大块如下：

[nc for nc in token.noun_chunks] # [instance, one simple phenomena, an answer, ...] 

我想知道是否有是一种基于noun_chunks对POS标签进行聚类的方式，以便我得到输出为

[('For', 'ADP'), 
('instance', 'NOUN'), # or NOUN_CHUNKS 
(',', 'PUNCT'), 
('one simple phenomena', 'NOUN_CHUNKS'), 
...] 

Answer 1

我想出了如何做到这一点。基本上，我们可以得到启动和名词短语符的结束位置如下：

noun_phrase_position = [(s.start, s.end) for s in token.noun_chunks] 
noun_phrase_text = dict([(s.start, s.text) for s in token.noun_chunks]) 
token_pos = [(i, t.text, t.pos_) for i, t in enumerate(token)] 

然后我用这个solution相结合，以合并基础上start的token_pos名单，stop位置

result = [] 
for start, end in noun_phrase_position: 
    result += token_pos[index:start] 
    result.append(token_pos[start:end]) 
    index = end 

result_merge = [] 
for i, r in enumerate(result): 
    if len(r) > 0 and isinstance(r, list): 
     result_merge.append((r[0][0], noun_phrase_text.get(r[0][0]), 'NOUN_PHRASE')) 
    else: 
     result_merge.append(r) 

输出

[(1, 'instance', 'NOUN_PHRASE'), 
(2, ',', 'PUNCT'), 
(3, 'consider', 'VERB'), 
(4, 'one simple phenomena', 'NOUN_PHRASE'), 
(7, ':', 'PUNCT'), 
(8, 'a', 'DET'), ...