如何发送并行GET请求并等待结果响应？

Question

我在春天mvc 3.2.2使用apache http客户端发送5请求同步如图所示。

如何将所有这些异步发送（并行）并等待请求返回以便从所有GET请求返回已解析的有效负载字符串？

public String myMVCControllerGETdataMethod() 
{ 
    // Send 1st request 
    HttpClient httpclient = new DefaultHttpClient(); 
    HttpGet httpget = new HttpGet("http://api/data?type=1"); 
    ResponseHandler<String> responseHandler = new BasicResponseHandler(); 
    String responseBody = httpclient.execute(httpget, responseHandler); 

    // Send 2st request 
    HttpClient httpclient2 = new DefaultHttpClient(); 
    HttpGet httpget2 = new HttpGet("http://api/data?type=2"); 
    ResponseHandler2<String> responseHandler2 = new BasicResponseHandler(); 
    String responseBody2 = httpclient.execute(httpget, responseHandler2); 

    // o o o more gets here 

    // Perform some work here...and wait for all requests to return 
    // Parse info out of multiple requests and return 
    String results = doWorkwithMultipleDataReturned(); 

    model.addAttribute(results, results); 
    return "index"; 

} 

Answer 1

只是一般，你需要封装你的工作单位在Runnable或java.util.concurrent.Callable，并通过java.util.concurrent.Executor（或org.springframework.core.task.TaskExecutor）执行。这允许每个工作单元分开执行，通常以异步方式执行（取决于Executor的实现）。

因此，对于您的特定问题，你可以做这样的事情：

import java.util.ArrayList; 
import java.util.Iterator; 
import java.util.List; 
import java.util.concurrent.Callable; 
import java.util.concurrent.Executor; 
import java.util.concurrent.FutureTask; 
import org.apache.http.client.methods.HttpGet; 
import org.apache.http.impl.client.BasicResponseHandler; 
import org.apache.http.impl.client.DefaultHttpClient; 
import org.springframework.stereotype.Controller; 
import org.springframework.ui.Model; 
import org.springframework.web.bind.annotation.RequestMapping; 

@Controller 
public class MyController { 
    //inject this 
    private Executor executor; 

    @RequestMapping("/your/path/here") 
    public String myMVCControllerGETdataMethod(Model model) { 
     //define all async requests and give them to injected Executor 
     List<GetRequestTask> tasks = new ArrayList<GetRequestTask>(); 
     tasks.add(new GetRequestTask("http://api/data?type=1", this.executor)); 
     tasks.add(new GetRequestTask("http://api/data?type=2", this.executor)); 
     //... 
     //do other work here 
     //... 
     //now wait for all async tasks to complete 
     while(!tasks.isEmpty()) { 
      for(Iterator<GetRequestTask> it = tasks.iterator(); it.hasNext();) { 
       GetRequestTask task = it.next(); 
       if(task.isDone()) { 
        String request = task.getRequest(); 
        String response = task.getResponse(); 
        //PUT YOUR CODE HERE 
        //possibly aggregate request and response in Map<String,String> 
        //or do something else with request and response 
        it.remove(); 
       } 
      } 
      //avoid tight loop in "main" thread 
      if(!tasks.isEmpty()) Thread.sleep(100); 
     } 
     //now you have all responses for all async requests 

     //the following from your original code 
     //note: you should probably pass the responses from above 
     //to this next method (to keep your controller stateless) 
     String results = doWorkwithMultipleDataReturned(); 
     model.addAttribute(results, results); 
     return "index"; 
    } 

    //abstraction to wrap Callable and Future 
    class GetRequestTask { 
     private GetRequestWork work; 
     private FutureTask<String> task; 
     public GetRequestTask(String url, Executor executor) { 
      this.work = new GetRequestWork(url); 
      this.task = new FutureTask<String>(work); 
      executor.execute(this.task); 
     } 
     public String getRequest() { 
      return this.work.getUrl(); 
     } 
     public boolean isDone() { 
      return this.task.isDone(); 
     } 
     public String getResponse() { 
      try { 
       return this.task.get(); 
      } catch(Exception e) { 
       throw new RuntimeException(e); 
      } 
     } 
    } 

    //Callable representing actual HTTP GET request 
    class GetRequestWork implements Callable<String> { 
     private final String url; 
     public GetRequestWork(String url) { 
      this.url = url; 
     } 
     public String getUrl() { 
      return this.url; 
     } 
     public String call() throws Exception { 
      return new DefaultHttpClient().execute(new HttpGet(getUrl()), new BasicResponseHandler()); 
     } 
    } 
} 

请注意，此代码没有经过测试。

对于您的Executor实施，请检查Spring's TaskExecutor和task:executor namespace。你可能需要一个可重用的线程池来用于这种用例（而不是每次创建一个新线程）。

Answer 2

您应该使用AsyncHttpClient。您可以提出任意数量的请求，并在收到回复时回复您。您可以配置它可以创建多少个连接。所有线程都由库处理，因此比自己管理线程要容易得多。

看看这里的例子：https://github.com/AsyncHttpClient/async-http-client