python列中具有相同值的行数

Question

我试图在python中重现R aggregate()函数，但没有连接。对于每一行，我只想计算给定列中具有相似值的行的出现次数。

我试图从这里迈出了一段代码解决它： http://timotheepoisot.fr/2011/12/01/the-aggregate-function-in-python/

我实现的修改由###表示。我目前遇到的问题是第一列[0]包含字符串，而代码似乎只能与浮动工作。

import numpy as np 
import scipy as sp 

def MSD(vec): 
    return [np.mean(vec),np.std(vec)] 
def aggregate(df,by=0,to=1,func=np.sum): 
    Dat = [] 
# ColBy = df.T[by] 
    ColBy = int(df.T[by][3:]) ### my attempt to read only the numbers in the first column's character strings 
    ColTo = df.T[to] 
    UniqueBy = np.sort(np.unique(ColBy)) 
    for ub in UniqueBy: 
     uTo = ColTo[ColBy==ub] 
     Out = func(uTo) 
#  Dat.append(np.concatenate(([ub],Out))) 
     Dat.append([ub],Out) ### because I do not want to concatenate 
    return Dat 

test_df = np.loadtxt('in_test.txt') 
Agr = aggregate(test_df,0,3,MSD) 
sp.savetxt("out_test.txt", Agr) 

这是错误消息：

Traceback (most recent call last): 
    File "count_same_reads.py", line 30, in <module> 
    test_df = np.loadtxt('in_test.txt') 
    File "/usr/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 796, in loadtxt 
    items = [conv(val) for (conv, val) in zip(converters, vals)] 
ValueError: could not convert string to float: Tag19184 

我的数据是制表符分隔的，主要含有字符串，除了第3列中，我想要写的行出现的次数。

下面是测试数据：

Tag19184 CTAAC hffef 1 a 36 - chr1 10006 0 36M 36 
Tag19184 CTAAC hffef 1 a 36 - chr1 10012 0 36M 36 
Tag19184 CTAAC hffef 1 a 36 - chr1 10018 0 36M 36 
Tag19184 CTAAC hffef 1 a 36 - chr1 10024 0 36M 36 
Tag19184 CTAAC hffef 1 a 36 - chr1 10030 0 36M 36 
Tag19184 CTAAC hffef 1 a 36 - chr1 10036 0 36M 36 
Tag19184 CTAAC hffef 1 a 36 - chr1 10042 0 36M 36 
Tag20198 CTAAC hffef 1 a 36 - chr1 10048 0 36M 36 
Tag20198 CTAAC hffef 1 a 36 - chr1 10054 0 36M 36 
Tag45093 CTAAC hffef 1 a 36 - chr1 10060 0 36M 36 

结果应该是这样的：

Tag19184 CTAAC hffef 7 a 36 - chr1 10006 0 36M 36 
Tag19184 CTAAC hffef 7 a 36 - chr1 10012 0 36M 36 
Tag19184 CTAAC hffef 7 a 36 - chr1 10018 0 36M 36 
Tag19184 CTAAC hffef 7 a 36 - chr1 10024 0 36M 36 
Tag19184 CTAAC hffef 7 a 36 - chr1 10030 0 36M 36 
Tag19184 CTAAC hffef 7 a 36 - chr1 10036 0 36M 36 
Tag19184 CTAAC hffef 7 a 36 - chr1 10042 0 36M 36 
Tag20198 CTAAC hffef 2 a 36 - chr1 10048 0 36M 36 
Tag20198 CTAAC hffef 2 a 36 - chr1 10054 0 36M 36 
Tag45093 CTAAC hffef 1 a 36 - chr1 10060 0 36M 36 

正如你可能会说，我不是在蟒蛇那么好还。任何的建议都受欢迎。

[编辑] PS。数据已按列[0]排序。

Answer 1

我会建议pandas，尤其是在你的基因组数据的情况下，数据的大小可能非常大：

In [44]: 
#you can read you data by pandas.read_csv() 
import pandas as pd 
print df 
     v0  v1  v2 v3 v4 v5 v6 v7  v8 v9 v10 v11 
0 Tag19184 CTAAC hffef 1 a 36 - chr1 10006 0 36M 36 
1 Tag19184 CTAAC hffef 1 a 36 - chr1 10012 0 36M 36 
2 Tag19184 CTAAC hffef 1 a 36 - chr1 10018 0 36M 36 
3 Tag19184 CTAAC hffef 1 a 36 - chr1 10024 0 36M 36 
4 Tag19184 CTAAC hffef 1 a 36 - chr1 10030 0 36M 36 
5 Tag19184 CTAAC hffef 1 a 36 - chr1 10036 0 36M 36 
6 Tag19184 CTAAC hffef 1 a 36 - chr1 10042 0 36M 36 
7 Tag20198 CTAAC hffef 1 a 36 - chr1 10048 0 36M 36 
8 Tag20198 CTAAC hffef 1 a 36 - chr1 10054 0 36M 36 
9 Tag45093 CTAAC hffef 1 a 36 - chr1 10060 0 36M 36 
In [45]: 
#if we want to group by the first 3 fields 
df.groupby(['v0','v1','v2']).transform(sum).v3 
Out[45]: 
0 7 
1 7 
2 7 
3 7 
4 7 
5 7 
6 7 
7 2 
8 2 
9 1 
Name: v3, dtype: int64 
In [46]: 
#all it takes is just one line 
df['v3']=df.groupby(['v0','v1','v2']).transform(sum).v3 
print df 
     v0  v1  v2 v3 v4 v5 v6 v7  v8 v9 v10 v11 
0 Tag19184 CTAAC hffef 7 a 36 - chr1 10006 0 36M 36 
1 Tag19184 CTAAC hffef 7 a 36 - chr1 10012 0 36M 36 
2 Tag19184 CTAAC hffef 7 a 36 - chr1 10018 0 36M 36 
3 Tag19184 CTAAC hffef 7 a 36 - chr1 10024 0 36M 36 
4 Tag19184 CTAAC hffef 7 a 36 - chr1 10030 0 36M 36 
5 Tag19184 CTAAC hffef 7 a 36 - chr1 10036 0 36M 36 
6 Tag19184 CTAAC hffef 7 a 36 - chr1 10042 0 36M 36 
7 Tag20198 CTAAC hffef 2 a 36 - chr1 10048 0 36M 36 
8 Tag20198 CTAAC hffef 2 a 36 - chr1 10054 0 36M 36 
9 Tag45093 CTAAC hffef 1 a 36 - chr1 10060 0 36M 36