.NET - 阻塞主线程，直到有任何可用线程

Question

我有一个进程，其中我的主线程正在读取文件并将其分解成多个部分。那些部分则需要进一步处理。我想利用任何可用的线程，以便下游处理尽可能多地利用CPU（或更多的核心）。我不想从主线程创建过多积压，所以我需要主线程等待添加到队列中，直到有另一个可用线程。

我看到像VB.NET 4.0: Looking to execute multiple threads, but wait until all threads are completed before resuming很多文章，但他们都在等待所有线程完成，而我只需要任何线程可用

这是不是我可以用Task Parallel Library解决，还是应该我手动创建线程和监视线程池？

Using Reader As New StreamReader(FileName) 
    Do 
     CurrentBlockSize = Reader.ReadBlock(CurrentBuffer, 0, BufferSize) 

     RunningBuffer &= New String(CurrentBuffer) 

     If RunningBuffer.Contains(RowDelimiter) Then 
      LineParts = RunningBuffer.Split(RowDelimiter) 

      For I As Integer = 0 To LineParts.Count - 1 
       If I < LineParts.Count - 1 Then 

        'Make synchronous call that blocks until' 
        'another thread is available to process the line' 
        AddLineToTheProcessingQueue(CurrentLine) 

       Else 
        RunningBuffer = LineParts(I) 
       End If 
      Next 
     End If 

    Loop While CurrentBlockSize = BufferSize 
End Using 

Answer 1

将此代码粘贴到新的控制台应用程序中。

Imports System.Threading 

Module Module1 

    ' I just picked 6 randomly, not sure what is a good strategy for picking this number 
    ' also, not sure what is the difference between a Worker Thread and a Completion thread 
    Const MaxWorkerThreads As Integer = 6 
    Const MaxCompletionPortThreads As Integer = 6 

    Sub Main() 

     ThreadPool.SetMaxThreads(MaxWorkerThreads, MaxCompletionPortThreads) 

     Dim availableWorkerThreads As Integer 
     Dim availableCompletionPortThreads As Integer 

     For i As Integer = 0 To 100 

      ' GetAvailableThreads returns results via output parameters 
      ThreadPool.GetAvailableThreads(availableWorkerThreads, availableCompletionPortThreads) 

      Dim tries As Integer = 0 

      Do While (availableWorkerThreads = 0) 
       ' this loop does not execute if there are available threads 
       ' you may want to add a fail-safe to check "tries" in case the child threads get stuck 
       tries += 1 

       Console.WriteLine(String.Format("waiting to start item {0}, attempt {1}, available threads: {2}, {3}", i, tries, availableWorkerThreads, availableCompletionPortThreads)) 

       ' failure to call Sleep will make your program unresponsive 
       Thread.Sleep(1000) 

       ' call GetAvailableThreads again for the next test at the top of the loop 
       ThreadPool.GetAvailableThreads(availableWorkerThreads, availableCompletionPortThreads) 
      Loop 

      ' this is how you pass parameters to a thread created through QueueUserWorkItem 
      Dim parameters As Object() = {i} 
      ThreadPool.QueueUserWorkItem(AddressOf DoWork, parameters) 
      ' According to MSDN, you must Sleep after calling QueueUserWorkItem, or else the current thread will just exit 
      Thread.Sleep(500) 

     Next 

    End Sub 

    Sub DoWork(parameters As Object()) 
     Dim itemNumber = parameters(0) 
     Dim sleepLength = itemNumber * 1000 
     Console.WriteLine(String.Format("Item: {0} - sleeping for {1} miliseconds.", itemNumber, sleepLength)) 
     Thread.Sleep(sleepLength) 
     Console.WriteLine(String.Format("Item: {0} - done sleeping.", itemNumber)) 
    End Sub 

End Module 

Answer 2

我不知道到底为什么做你想做的事，但你可以实现通过BlockingCollection或BoundedCapacity集的数据流块非常类似的东西。

例如，如果您将容量设置为1并且此时您的消费者正忙，您将无法向队列中添加第二个项目，直到其中一个消费者完成其当前工作并移除该项目从队列中。这两个版本都可以让您等待，直到您可以将其他项添加到队列中。