2017-10-08 71 views
1

鸣叫提取@mentions我有一个CSV文件是这样的使用的findall蟒蛇(给出不正确的结果)

text 
RT @CritCareMed: New Article: Male-Predominant Plasma Transfusion Strategy for Preventing Transfusion-Related Acute Lung Injury... htp://… 
#CRISPR Inversion of CTCF Sites Alters Genome Topology & Enhancer/Promoter Function in @CellCellPress htp://.co/HrjDwbm7NN 
RT @gvwilson: Where's the theory for software engineering? Behind a paywall, that's where. htp://.co/1t3TymiF3M #semat #fail 
RT @sciencemagazine: What’s killing off the sea stars? htp://.co/J19FnigwM9 #ecology 
RT @MHendr1cks: Eve Marder describes a horror that is familiar to worm connectome gazers. htp://.co/AEqc7NOWoR via @nucAmbiguous htp://… 

我想提取所有提及(以“@”)从鸣叫文本。到目前为止,我已经做到了这一点

import pandas as pd 
import re 

mydata = pd.read_csv("C:/Users/file.csv") 
X = mydata.ix[:,:] 
X=X.iloc[:,:1] #I have multiple columns so I'm selecting the first column only that is 'text' 

for i in range(X.shape[0]): 
result = re.findall("(^|[^@\w])@(\w{1,25})", str(X.iloc[:i,:])) 

print(result); 

有两个问题在这里: 第一:在str(X.iloc[:1,:])它给了我['CritCareMed']这也不行,因为它应该给我['CellCellPress'],在str(X.iloc[:2,:])再次给了我['CritCareMed']这是当然不会再罚款。最后的结果,我得到的是

[( ' ' 'CritCareMed'),('', 'gvwilson'),(””, 'sciencemagazine')]

一点也没有不包括第二排的提及和最后一排的两个提及。 我想应该是这个样子:

enter image description here

我怎样才能取得这些成果?这只是一个示例数据,我的原始数据有很多推文,所以方法好吗?

回答

0

您可以使用str.findall方法来避免for循环,使用落后负的样子,以取代(^|[^@\w])形成你不要在你的正则表达式需要另一个捕获组:

df['mention'] = df.text.str.findall(r'(?<![@\w])@(\w{1,25})').apply(','.join) 
df 
#            text mention 
#0 RT @CritCareMed: New Article: Male-Predominant... CritCareMed 
#1 #CRISPR Inversion of CTCF Sites Alters Genome ... CellCellPress 
#2 RT @gvwilson: Where's the theory for software ... gvwilson 
#3 RT @sciencemagazine: What’s killing off the se... sciencemagazine 
#4 RT @MHendr1cks: Eve Marder describes a horror ... MHendr1cks,nucAmbiguous 

而且X.iloc[:i,:]给出一个数据帧,因此str(X.iloc[:i,:])为您提供了一个数据帧的字符串表示形式,它与单元格中的元素非常不同,从text列中提取实际的字符串,您可以使用X.text.iloc[0]或一个tter的方式通过蒸馏塔循环使用iteritems

import re 
for index, s in df.text.iteritems(): 
    result = re.findall("(?<![@\w])@(\w{1,25})", s) 
    print(','.join(result)) 

#CritCareMed 
#CellCellPress 
#gvwilson 
#sciencemagazine 
#MHendr1cks,nucAmbiguous 
+0

如何从df中选择第一列?如果iloc给出数据帧。在我的文件中有多个列,并且必须仅处理第一列,即'text' – melissa

+0

要选择第一列,您可以使用列名,即'df.text','df ['text'] '或使用'iloc','df.iloc [:,0]'。 – Psidom

1

虽然你已经有了答案,你甚至可以尝试优化整个导入过程,像这样:

import re, pandas as pd 

rx = re.compile(r'@([^:\s]+)') 

with open("test.txt") as fp: 
    dft = ([line, ",".join(rx.findall(line))] for line in fp.readlines()) 

    df = pd.DataFrame(dft, columns = ['text', 'mention']) 
    print(df) 


其中产量:

           text     mention 
0 RT @CritCareMed: New Article: Male-Predominant...    CritCareMed 
1 #CRISPR Inversion of CTCF Sites Alters Genome ...   CellCellPress 
2 RT @gvwilson: Where's the theory for software ...     gvwilson 
3 RT @sciencemagazine: What’s killing off the se...   sciencemagazine 
4 RT @MHendr1cks: Eve Marder describes a horror ... MHendr1cks,nucAmbiguous 

这可能是有点快,你不需要改变df一旦它已经构造ucted。

+0

非常感谢你,我会尽力而为:) – melissa