合并许多TSV文件第一列

Question

我有一个目录中只有两列的许多（几十个）TSV文件，我想根据第一列值合并所有这些文件（两列都有我需要维护的标题）;如果此值存在，则它必须添加相应第二列的值，依此类推（请参阅示例）。文件可能有不同数量的行，并且没有按第一列排序，但这可以通过排序轻松完成。

我试过加入但是，它只适用于两个文件。可以针对目录中的所有文件进行连接扩展吗？我认为awk可能是一个更好的解决方案，但我在awk中的知识非常有限。有任何想法吗？

这里是只有三个文件的例子：

S01.tsv 

Accesion S01 
AJ863320 1 
AM930424 1 
AY664038 2 

S02.tsv 

Accesion S02 
AJ863320 2 
AM930424 1 
EU236327 1 
EU434346 2 

S03.tsv 

Accesion S03 
AJ863320 5 
EU236327 2 
EU434346 2 

OUTFILE应该是：

Accesion S01 S02 S03 
    AJ863320 1  2  5 
    AM930424 1  1 
    AY664038 2 
    EU236327   1  2 
    EU434346   2  2 

确定，这要归功于詹姆斯布朗，我得到这个代码的工作（我把它命名为compile.awk）有一些小问题：

BEGIN { OFS="\t" }       # tab separated columns 
FNR==1 { f++ }        # counter of files 
{ 
    a[0][$1]=$1        # reset the key for every record 
    for(i=2;i<=NF;i++)      # for each non-key element 
     a[f][$1]=a[f][$1] $i (i==NF?"":OFS) # combine them to array element 
} 
END {           # in the end 
    for(i in a[0])       # go thru every key 
     for(j=0;j<=f;j++)      # and all related array elements 
      printf "%s%s", a[j][i], (j==f?ORS:OFS) 
}            # output them, nonexistent will output empty 

当我实际的文件运行它作为

awk -f compile.awk 01.tsv 02.tsv 03.tsv 

我得到的输出：

LN854586.1.1236   1 
JF128382.1.1303  1 
Accesion S01 S02 S03 
JN233077.1.1420 1  
HQ836180.1.1388  1 
KP718814.1.1338   1 
JQ781640.1.1200   2 

前两行不属于那里作为文件应与所有文件的标题（三号线）开始。 任何想法如何解决这个问题？

Answer 1

我可能会解决它是这样的：

#!/usr/bin/perl 
use strict; 
use warnings; 
use Data::Dumper; 

my @header; 
my %all_rows; 
my %seen_cols; 


#read STDIN or files specified as args. 
while (<>) { 
    #detect a header row by keyword. 
    #can probably do this after 'open' but this way 
    #means we can use <> and an arbitrary file list. 
    if (m/^Accesion/) { 
     @header = split;  
     shift @header; #drop "accession" off the list so it's just S01,02,03 etc. 
     $seen_cols{$_}++ for @header; #keep track of uniques. 
    } 
    else { 
     #not a header row - split the row on whitespace. 
     #can do /\t/ if that's not good enough, but it looks like it should be. 
     my ($ID, @fields) = split; 
     #use has slice to populate row. 

     my %this_row; 
     @this_row{@header} = @fields; 

     #debugging 
     print Dumper \%this_row; 

     #push each field onto the all rows hash. 
     foreach my $column (@header) { 
     #append current to field, in case there's duplicates (no overwriting) 
     $all_rows{$ID}{$column} .= $this_row{$column}; 
     } 
    } 
} 

#print for debugging 
print Dumper \%all_rows; 
print Dumper \%seen_cols; 

#grab list of column headings we've seen, and order them. 
my @cols_to_print = sort keys %seen_cols; 

#print header row. 
print join "\t", "Accesion", @cols_to_print,"\n"; 
#iteate keys, and splice. 
foreach my $key (sort keys %all_rows) { 
    #print one row at a time. 
    #map iterates all the columns, and gives the value or an empty string 
    #if it's undefined. (prevents errors) 
    print join "\t", $key, (map { $all_rows{$key}{$_} // '' } @cols_to_print),"\n" 
} 

鉴于你的输入 - 调试排除 - 打印：

Accesion S01 S02 S03 
AJ863320 1 2 5 
AM930424 1 1  
AY664038 2   
EU236327  1 2 
EU434346  2 2