Powershell CSV脚本更快

我正在寻找使我的脚本更快，给我更快的输出。我正在处理csv中的大量数据，如果我将整个文件放入脚本中，需要大约一个小时。Powershell CSV脚本更快

$csv = Import-Csv 'U:\Local Group Members.csv' | 
Where-Object {($_.Name0 -eq "administrators") -and ($_.caption0 -match "Microsoft Windows 10 Enterprise|Microsoft Windows 7 Enterprise|Microsoft Windows 7 Professional|Microsoft Windows 8 Enterprise|Microsoft Windows 8 Pro|Microsoft Windows 8.1 Enterprise|Microsoft Windows 8.1 Pro")} | 
    Select-Object "Netbios_name0", "Name0", "Account0","category0","Domain0","Unique Account Name","Type0","caption0", "Excluded" 
#Modify each line 
Foreach ($row in $csv) { 
If ($row.Type0 -eq 'Domain') { 
    $row."Unique Account Name" = "$($row.Domain0) - $($row.Account0)" 
    If ($row."Unique Account Name" -in @('ACCOUNTS - DODSCAN'.'ACCOUNTS - Domain Admins','ACCOUNTS - LADM_WS_Admins','ACCOUNTS - Tech Enterprise')) 
     {$row."Excluded" = "True"} 
    Else {$row."Excluded" = "False"}   
} 
Else { 
    $row."Unique Account Name" = "$($row.Netbios_name0) - $($row.Account0)" 
    If ($row."Account0" -in @('esrxadm1n_esi','#Update','medco_tech','medco_admin')) 
     {$row."Excluded" = "True"} 
    Else {$row."Excluded" = "False"} 
} 
    Write-Host $row."Unique Account Name" 
    Write-Host $row."Excluded"  
} 

#Export CSV 
$csv | Export-Csv U:\$(get-date -f yyyy-MM-dd-hh-mm)-TestOutput1.csv -NoTypeInformation

任何人都可以帮助我更快吗？

来源

2017-06-15 Brady Sprenger

这将是一个更好的CodeReview，而不是StackOverflow。 – BenH

好的，谢谢@BenH –

这个文件有多少行？ –

而不是让2个Foreach块在同一个对象上循环（$ csv中的$行），为什么不把它们合并？如果您的CSV很大，为什么要经过两次？

Foreach ($row in $csv) { 
If ($row.Type0 -eq 'Domain') { 
    $row."Unique Account Name" = "$($row.Domain0) - $($row.Account0)" 
    Write-Host $row."Unique Account Name" 
} Else { 
    $row."Unique Account Name" = "$($row.Netbios_name0) - $($row.Account0)" 
} 
If (($row.Type0 -eq 'Local') -and ($row.Account0 -in @('esrxadm1n_esi','#Update','medco_tech','medco_admin')) -and ($row."Unique Account Name" -in @('ACCOUNTS - DODSCAN'.'ACCOUNTS - Domain Admins','ACCOUNTS - LADM_WS_Admins','ACCOUNTS - Tech Enterprise'))) { 
    $row."Excluded" = "True" 
    Write-Host $row."Excluded" 
} Else { 
    $row."Excluded" = "False" 
} 
}

而且，当你在上面做的这场比赛：

($_.caption0 -match "Microsoft Windows 10 Enterprise|Microsoft Windows 7 Enterprise|Microsoft Windows 7 Professional|Microsoft Windows 8 Enterprise|Microsoft Windows 8 Pro|Microsoft Windows 8.1 Enterprise|Microsoft Windows 8.1 Pro")

你可以把它降低到$_.caption0 -match "Microsoft Windows \d{1,2}"没有它拿起你不想行，或者是说出来的照片？我在评论谈论

来源

2017-06-15 16:43:42

这是一种超乎想象的情况，因为我们不希望Microsoft Windows服务器在那里 –

例子：

$t = New-Object System.Timers.Timer 
$t.Start() 

$arr1 = @('ACCOUNTS - DODSCAN'.'ACCOUNTS - Domain Admins','ACCOUNTS - LADM_WS_Admins','ACCOUNTS - Tech Enterprise') 
$arr2 = @('esrxadm1n_esi','#Update','medco_tech','medco_admin') 

$csv = Import-Csv 'U:\Local Group Members.csv' | Where-Object {($_.Name0 -eq "administrators") -and ($_.caption0 -match "Microsoft Windows 10 Enterprise|Microsoft Windows 7 Enterprise|Microsoft Windows 7 Professional|Microsoft Windows 8 Enterprise|Microsoft Windows 8 Pro|Microsoft Windows 8.1 Enterprise|Microsoft Windows 8.1 Pro")} | Select-Object "Netbios_name0", "Name0", "Account0","category0","Domain0","Unique Account Name","Type0","caption0", "Excluded" 
Write-Host "Took $($t.Elapsed)ms to load and filter the CSV" 

#Modify each line 
Foreach ($row in $csv) { 
If ($row.Type0 -eq 'Domain') { 
    $row."Unique Account Name" = "$($row.Domain0) - $($row.Account0)" 
    If ($row."Unique Account Name" -in $arr1) 
     {$row."Excluded" = "True"} 
    Else {$row."Excluded" = "False"}   
} 
Else { 
    $row."Unique Account Name" = "$($row.Netbios_name0) - $($row.Account0)" 
    If ($row."Account0" -in $arr2) 
     {$row."Excluded" = "True"} 
    Else {$row."Excluded" = "False"} 
} 
    Write-Host $row."Unique Account Name" 
    Write-Host $row."Excluded"  
} 

#Export CSV 
$csv | Export-Csv U:\$(get-date -f yyyy-MM-dd-hh-mm)-TestOutput1.csv -NoTypeInformation

来源

2017-06-15 17:29:06 thepip3r

我试过了你，当我运行它时，整个文件仍然花费了54分钟。我做的一个49 –

文件加载的发射时间是多少？ – thepip3r

如从创建文件到脚本结束？ –

如果你把所有这样在同一行？

$tab1 = @('ACCOUNTS - DODSCAN', 'ACCOUNTS - Domain Admins','ACCOUNTS - LADM_WS_Admins','ACCOUNTS - Tech Enterprise') 
$tab2 = @('esrxadm1n_esi','#Update','medco_tech','medco_admin') 

Import-Csv 'U:\Local Group Members.csv' | Select "administrators", "Netbios_name0", "Name0", "Account0","category0","Domain0","Unique Account Name","Type0","caption0", "Excluded" | %{ 

if ($_.Name0 -eq "administrators" -and $_.caption0 -match "Microsoft Windows 10 Enterprise|Microsoft Windows 7 Enterprise|Microsoft Windows 7 Professional|Microsoft Windows 8 Enterprise|Microsoft Windows 8 Pro|Microsoft Windows 8.1 Enterprise|Microsoft Windows 8.1 Pro") 
{ 
    If ($_.Type0 -eq 'Domain') 
    { 
     $_."Unique Account Name" = "$($_.Domain0) - $($_.Account0)" 
     $_."Excluded" = $_."Unique Account Name" -in $tab1 
    } 
    Else 
    { 
     $_."Unique Account Name" = "$($_.Netbios_name0) - $($_.Account0)" 
     $_."Excluded" = $_."Account0" -in $tab2 
    } 

    $_ 

} 

} | Export-Csv U:\$(get-date -f yyyy-MM-dd-hh-mm)-TestOutput1.csv -NoTypeInformation

来源

2017-06-15 17:35:22 Esperento57

不幸的是，当我把它放到powershell中时，它并没有运行。 –

什么是错误？ – Esperento57

我在唯一帐户名称中添加，所以它给我以下错误，因为它说它不在那里。异常设置“唯一帐户名称”：“该属性'唯一帐户名称'无法找到此对象上。验证该属性是否存在并可以设置。“ At line：10 char：55 + $ _。“Unique Account Name”=“$（$ _。Domain0） - $（$ _。Account0）” + ~~~~~~~~~~~ + CategoryInfo：NotSpecified：（:) []，SetValueInvocationException –

我有个〜60万线CSV事件日志这里，我可以测试：

import-csv events.csv | select * | foreach { $_ } | Export-csv out.csv

和需要5分15秒（〜180MB /上的SSD）。随着write-host它推高9分钟36秒。所以这可能是关于你的CSV变得很大（每行很多信息？），一次加载到内存中？或写主机线。

尝试将其更改为流媒体的方式：

Import-Csv 'U:\Local Group Members.csv' | 
    Where-Object { 

     ($_.Name0 -eq "administrators") -and 
     ($_.caption0 -match "Microsoft Windows (10 Enterprise|7 Enterprise|7 Professional|8 Enterprise|8 Pro|8.1 Enterprise|8.1 Pro)") 

    } | ForEach-Object { 

     If ($row.Type0 -eq 'Domain') 
     { 
      $row."Unique Account Name" = $row.Domain0 + ' - ' + $row.Account0 
      $row."Excluded" = ($row."Unique Account Name" -in @('ACCOUNTS - DODSCAN'.'ACCOUNTS - Domain Admins','ACCOUNTS - LADM_WS_Admins','ACCOUNTS - Tech Enterprise')) 
     } 
     Else 
     { 
      $row."Unique Account Name" = $row.Netbios_name0 + ' - ' + $row.Account0 
      $row."Excluded" = ($row."Account0" -in @('esrxadm1n_esi','#Update','medco_tech','medco_admin')) 
     } 

    } | Select-Object "Netbios_name0", "Name0", "Account0", 
         "category0","Domain0","Unique Account Name", 
         "Type0","caption0", "Excluded" | 
    Export-Csv U:\$(get-date -f yyyy-MM-dd-hh-mm)-TestOutput1.csv -NoTypeInformation

我也周围的一些多余的，“如果（事测试结果为真），那么（真）否则，如果（事测试是假的），那么洗牌（假）“只是”（东西测试）“，删除了子表达式，缩短了正则表达式，并且将选择移到了”过滤器提前过早选择迟到“的末尾。

但是使它比PS方法快得多，包括将CSV处理从PowerShell下降到.Net，或将它视为文本文件，并以某种方式首先进行某种过滤。

来源

2017-06-15 19:17:41 TessellatingHeckler

Powershell CSV脚本更快

回答

相关问题