当比较大型数据集时使用嵌套循环时速度变慢

Question

我正在检索两个使用Linq到实体的数据集列表。他们都在同一个数据库中，但我需要将一个表转换为我的任务表，因为它已集成到我的日历中。在这里不值得详细讨论，但我很想加快匹配id和创建新Task对象的过程。这是一次完成的片段，即使速度很慢，我也可以让程序在一夜之间运行。但是，为了将来的参考，我想提高一些提高效率的建议。

var accounts = data.Accounts.ToList().OrderBy(a => a.ID); 
Incidents[] Incidents = data.Incidents.ToArray(); 

     for (int i=0;i<Incidents.Length;i++) 
     { 
      foreach (var a in accounts) 
      { 
       if (a.Acct_CID == Incidents[i].CustomerID) 
       { 
        Tasks t = new Tasks(); 
        t.creator_id = a.ID; 
        t.start_date = Incidents[i].DateOpened; 
        t.end_date = Incidents[i].DateCLosed; 
        t.product_code = Incidents[i].ProductCode; 
        t.install_type = Incidents[i].InstallType; 
        t.os = Incidents[i].OSType; 
        t.details = Incidents[i].Description; 
        t.solution = Incidents[i].Solution; 
        t.creator_name = Incidents[i].TechID; 
        t.category = Incidents[i].Title; 
        t.text = "Ticket for" + " " + Incidents[i].Name; 
        if (t.end_date == DateTime.MinValue || t.end_date == null) 
         t.status_id = 6; 
        else t.status_id = 7; 
        data.Tasks.Add(t); 
        break; 
       } 
      } 
     } 
     data.SaveChanges(); 

Answer 1

更换

foreach (var a in accounts) 
     { 
      if (a.Acct_CID == Incidents[i].CustomerID) 
      { 

为什么不加入表和动态创建的任务？

var tasks = from i in data.Incidents 
      join a in data.Accounts on i.CustomerID equals a.Acct_CID 
      select new Tasks() 
      { 
       creator_id = a.ID, 
       start_date = i.DateOpened, 
       end_date = i.DateCLosed 
       // ... 
      }; 

顺便说一句，我不认为排序在这里是有意义的，因此它应该无关紧要你将创建的任务添加到数据库。

// Query will not be executed until here 
foreach(var task in tasks) 
    data.Tasks.Add(task); 
data.SaveChanges(); 

Answer 2

替换该行

var accounts = data.Accounts.ToList().OrderBy(a => a.ID); 

与此

var accounts = data.Accounts.OrderBy(a => a.ID).ToList(); 

这将让数据库做排序，然后缓存排序结果。你现在拥有什么，然后在每次到达foreach循环时对它们进行排序（accounts重新枚举）。

我不能说它会做出巨大的改进，但是如果你的数据集足够大，重新排列一个大的列表很多次肯定会减慢你的速度。

---

在第二眼，你不是每次只选accounts，但你似乎在寻找中只用于记录的一小部分，但你遍历整个数组。考虑

 foreach (var a in accounts.Where(acct => acct.Acct_CID == Incidents[i].CustomerID)) 
     { 

Answer 3

我会Join在DB

var joinedResult = data.Accounts.Join(data.Incidents, 
             a => a.Acct_CID, 
             i => i.CustomerID, 
             (a, i) => new { Account = a, Incident = i }); 

foreach (var item in joinedResult) 
{ 
    Tasks t = new Tasks(); 
    t.creator_id = item.Account.ID; 
    t.start_date = item.Incident.DateOpened; 
    ........ 

} 

Answer 4

创建帐户

var accountsLookup = data.Accounts.ToLookup(a => a.Acct_CID); 
foreach (var incident in data.Incidents) 
{ 
    foreach (var a in accountsLookup[incident.CustomerID]) 
    { 
     Tasks t = new Tasks(); 
     t.creator_id = a.ID; 
     ... 
    } 
} 
data.SaveChanges(); 

---

如果该帐户是唯一的，你还可以创建词典的查找结果

var accountsDict = data.Accounts.ToDictionary(a => a.Acct_CID); 
foreach (var incident in data.Incidents) 
{ 
    Account a; 
    if (accountsDict.TryGetValue(incident.CustomerID, out a) 
    { 
     Tasks t = new Tasks(); 
     t.creator_id = a.ID; 
     ... 
    } 
} 
data.SaveChanges(); 

这会比第一个变种更快。请注意，词典具有不依赖其大小的恒定查找时间。因此你基本上得到了循环的O（n）执行时间。您的原始实现具有O（n^2）执行时间。

Answer 5

var tasks = (from i in data.Incidents 
        join a in data.Accounts on i.CustomerID equals a.Acct_CID 
        select new 
        { 
         creator_id = a.ID, 
         start_date = i.DateOpened, 
         end_date = i.DateCLosed, 
         product_code = i.ProductCode, 
         install_type = i.InstallType, 
         os = i.OSType, 
         details = i.Description, 
         solution = i.Solution, 
         creator_name = i.TechID, 
         category = i.Title, 
         text = "Ticket for" + " " + i.Name, 
         status_id = 7 
        }).AsEnumerable().Select(x => new 
         { 
          x.creator_id, 
          x.start_date, 
          x.end_date, 
          x.product_code, 
          x.os, 
          x.details, 
          x.solution, 
          x.creator_name, 
          x.category, 
          x.text, 
          x.install_type, 
          x.status_id 
         }); 


     foreach (var item in tasks) 
     { 
      Tasks t = new Tasks(); 
      t.os = item.os; 
      t.id = item.creator_id; 
      t.install_type = item.install_type; 
      t.start_date = item.start_date; 
      t.end_date = item.end_date; 
      t.solution = item.solution; 
      t.details = item.details; 
      t.creator_name = item.creator_name; 
      t.category = item.category; 
      t.text = item.text; 
      t.product_code = item.product_code; 
      if (t.end_date == DateTime.MinValue || t.end_date == null) 
       t.status_id = 6; 
      else t.status_id = 7; 
      data.Tasks.Add(t); 
     } 
     data.SaveChanges();