检查输入文本中来自单词集合的单词

Question

我收集了名词短语约10,000个单词。我想检查这些NP收集的每个新的输入文本数据，并提取那些包含这些NP的任何句子。我不想为每个单词运行循环，因为这会使我的代码变得很慢。我正在使用Java和Stanford CoreNLP。

Answer 1

一个快速简便的方法是使用Regexner来识别字典中任何东西的所有示例，然后检查句子中的非“O”NER标签。

package edu.stanford.nlp.examples; 

import edu.stanford.nlp.ling.*; 
import edu.stanford.nlp.pipeline.*; 
import edu.stanford.nlp.util.*; 
import java.util.*; 
import java.util.stream.Collectors; 

public class FindSentencesWithPhrase { 

    public static boolean checkForNamedEntity(CoreMap sentence) { 
    for (CoreLabel token : sentence.get(CoreAnnotations.TokensAnnotation.class)) { 
     if (token.ner() != null && !token.ner().equals("O")) { 
     return true; 
     } 
    } 
    return false; 
    } 

    public static void main(String[] args) { 
    Properties props = new Properties(); 
    props.setProperty("annotators", "tokenize,ssplit,pos,lemma,regexner"); 
    props.setProperty("regexner.mapping", "phrases.rules"); 
    StanfordCoreNLP pipeline = new StanfordCoreNLP(props); 
    String exampleText = "This sentence contains the phrase \"ice cream\"." + 
     "This sentence is not of interest. This sentences contains pizza."; 
    Annotation ann = new Annotation(exampleText); 
    pipeline.annotate(ann); 
    for (CoreMap sentence : ann.get(CoreAnnotations.SentencesAnnotation.class)) { 
     if (checkForNamedEntity(sentence)) { 
     System.out.println("---"); 
     System.out.println(sentence.get(CoreAnnotations.TokensAnnotation.class). 
      stream().map(token -> token.word()).collect(Collectors.joining(" "))); 
     } 
    } 
    } 
} 

文件“phrases.rules”应该是这样的：

ice cream  PHRASE_OF_INTEREST  MISC 1 
pizza PHRASE_OF_INTEREST  MISC 1