格式化文本文件中使用bash脚本

-1

我有一个文件（exOut.txt）按以下格式由数千行文字CSV：格式化文本文件中使用bash脚本

[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482 
[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482 
[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405, score=0.497312, total=11.0min 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405, score=0.499232, total=11.0min 
[Parallel(n_jobs=-2)]: Done 2 out of 6 | elapsed: 11.0min remaining: 22.0min 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405, score=0.499762, total=11.1min 
[Parallel(n_jobs=-2)]: Done 3 out of 6 | elapsed: 11.1min remaining: 11.1min 
[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482, score=0.449309, total=19.6min 
[Parallel(n_jobs=-2)]: Done 4 out of 6 | elapsed: 19.6min remaining: 9.8min 
[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482, score=0.449831, total=19.7min 
[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482, score=0.451609, total=19.7min 
[Parallel(n_jobs=-2)]: Done 6 out of 6 | elapsed: 19.7min remaining: 0.0s 
[Parallel(n_jobs=-2)]: Done 6 out of 6 | elapsed: 19.7min finished 
...

我想写一个shell脚本会拿这个文件并重新格式化，以csv格式创建一个新文件，只记录具有“score”属性的行。这应该看起来像这样：

solver,penalty,multi_class,max_iter,C,score 
sag,l2,multinomial,187.638,0.312,0.497 
sag,l2,multinomial,187.638,0.312,0.499 
sag,l2,multinomial,187.638,0.312,0.500 
newton-cg,l2,ovr,187.638,0.779,0.449 
newton-cg,l2,ovr,187.638,0.779,0.450 
newton-cg,l2,ovr,187.638,0.779,0.450

如果可能的话，所有值四舍五入到最接近的第1000位。

最终我想借此CSV和通过识别与记录，除了“分数”等各个领域，并与给出的参数的平均得分来取代这一个纪录做一个浓缩版。例如：

solver,penalty,multi_class,max_iter,C,avg_score 
sag,l2,multinomial,187.638,0.312,0.499 
newton-cg,l2,ovr,187.638,0.779,0.450

任何帮助表示赞赏！我不是正则表达式的专家，主要是为什么我问。

编辑1个感谢您的反馈，这里有更多的一些信息：

用grep，awk的迄今为止我已经试过各种脚本和sed，包括grep '=.*,' exOut.txt只承认一个大发生的模式，而不是多个字段，以及仅清理每行的第一部分的sed 's/^[^\=]*\=//g' exOutput.txt > firstCSV.csv。

来源

2017-06-06 flpp

欢迎堆栈溢出。这不是一个代码编写服务，您可以在其中发布您的要求和选择的语言，并且有人为您编写代码。我们非常乐意提供帮助，但我们希望您先努力自己解决问题，并将您的努力包括在您的问题中。请在这里询问之前[编辑]显示你自己尝试过的代码。如果您需要更多信息，请参阅[问]。 –

'awk'应该很简单。试一试，如果你仍然无法得到它，你可以发布你的代码和它给你的输出的例子。（而'awk'非常值得学习。） – Jack

如果你想要一个bash脚本，为什么用python标记？使用scikit学习由Python程序生成 –

1.问题，你可以参考parseRawDataFile功能。

2.问题，你可以参考THR parseCsvDataFile功能。

3.In代码有一些硬代码，请注意。

import codecs 
datausage = r"e:\temp\111.txt" 
storagePath = r"e:\temp\222.csv" 
storagePath_New = r"e:\temp\222_New.csv" 
strStr = "score" 

def parseRawDataFile(path): 
    sumResult = "solver,penalty,multi_class,max_iter,C,score\r\n" 
    # [CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405, score=0.497312, total=11.0min 

    with open(path) as f: 

     lines = f.readlines() 
     for line in lines: 
      if strStr in line: 

       tempList = line.replace(",", "=").split("=") 
       # print(tempList) 
       sumResult += "%s,%s,%s,%.03f,%.03f,%.03f\r\n" % (tempList[1], tempList[3], tempList[5], float(tempList[7]), float(tempList[9]), float(tempList[11])) 

    # print(sumResult) 
    return sumResult 

def parseCsvDataFile(path): 
    sumResult = "solver,penalty,multi_class,max_iter,C,score\r\n" 
    # sag,l2,multinomial,187.638,0.312,0.497 

    nameList = [] 
    with open(path) as f: 
     lines = f.readlines() 
     temptSum = 0.0 
     tempNum = 0 
     for line in lines: 
      if "solver" in line: 
       continue 
      tempList = line.split(",") 
      if tempList[0] not in nameList: 
       if temptSum != 0: 
        sumResult += "%.03f\r\n" % (temptSum/tempNum) 
       sumResult += "%s,%s,%s,%.03f,%.03f," % (tempList[0], tempList[1], tempList[2], float(tempList[3]), float(tempList[4])) 

       tempNum = 0 
       temptSum = 0.0 
       nameList.append(tempList[0]) 
      if tempList[0] in nameList: 
       tempNum += 1 
       temptSum += float(tempList[-1]) 

     sumResult += "%.03f\r\n" % (temptSum/tempNum) 
    print(sumResult) 
    return sumResult 

def writeLog(f, d): 
    try: 
     f = codecs.open(f, 'w', 'utf-8') 
     f.write(d) 
     f.close() 
    except: 
     print("file is not exist.") 

result = parseRawDataFile(datausage) 
writeLog(storagePath, result) 
result = parseCsvDataFile(storagePath) 
writeLog(storagePath_New, result)

来源

2017-06-06 02:44:16 saul

@ cricket_007谢谢。修复了错误代码格式。 – saul

格式化文本文件中使用bash脚本

回答

相关问题