2014-01-16 41 views
0

我有以下3个命令(每个都包含xargs和并行进程),我希望并行运行它们。Xargs:并行运行3个单独的xargs命令

1. ls filenames_for_bld*_all | xargs -i bash -c 'calculateMd5Sums "[email protected]"' _ {} 
2. cat filenames_for_bld*_all | awk -F'|' '{print $2}' | xargs -n 1 -P 10 -I {} remotecmd -q -n server1 md5sum {} 2>/dev/null| tee -a ${dir}/md5sums_for_server1 
3. cat filenames_for_bld*_all | awk -F'|' '{print $2}' | xargs -n 1 -P 23 -I {} remotecmd -q -n server2 md5sum {} 2>/dev/null | tee -a ${dir}/md5sums_for_server2 

每个xargs的命令运行在单独的服务器几个进程。所以1运行在服务器1-5上,2运行在服务器6上,3运行在服务器7上。

是否有可能并行运行上述3个?

在上面使用xargs会很好。我想上述3设置为一个字符串xargsInput的,由换​​行分隔,然后...

echo $xargsInput | xargs -n 1 -P 3 -I {} sh -c {} 

然而,这有一些错误:

ls: xargs: No such file or directory 
ls: bash: No such file or directory 
ls: calculateMd5Sums: No such file or directory 
ls: 20140107: No such file or directory 
ls: _: No such file or directory 
ls: {}: No such file or directory 

回答

1

每一行后只需添加一个&,和可选的wait在最后等待他们全部完成:

ls filenames_for_bld*_all | xargs -i bash -c 'calculateMd5Sums "[email protected]"' _ {} & 
cat filenames_for_bld*_all | awk -F'|' '{print $2}' | xargs -n 1 -P 10 -I {} remotecmd -q -n server1 md5sum {} 2>/dev/null| tee -a ${dir}/md5sums_for_server1 & 
cat filenames_for_bld*_all | awk -F'|' '{print $2}' | xargs -n 1 -P 23 -I {} remotecmd -q -n server2 md5sum {} 2>/dev/null | tee -a ${dir}/md5sums_for_server2 & 
wait 

这适用于不需要进行交互的所有单独的shell命令。