的HttpClient 4.3阻塞连接池

，当我使用的HttpClient 4.3如下的HttpClient 4.3阻塞连接池

static { 
    try { 
     SSLContextBuilder builder = new SSLContextBuilder(); 
     builder.loadTrustMaterial(null, new TrustSelfSignedStrategy()); 
     SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(builder.build()); 

     CookieSpecProvider easySpecProvider = new CookieSpecProvider() { 

      public CookieSpec create(HttpContext context) { 
       return new BrowserCompatSpec() { 
        @Override 
        public void validate(Cookie cookie, CookieOrigin origin) throws MalformedCookieException { 
         // Oh, I am easy 
        } 
       }; 
      } 

     }; 
     Registry<CookieSpecProvider> r = RegistryBuilder.<CookieSpecProvider> create() 
       .register(CookieSpecs.BEST_MATCH, new BestMatchSpecFactory()) 
       .register(CookieSpecs.BROWSER_COMPATIBILITY, new BrowserCompatSpecFactory()) 
       .register("easy", easySpecProvider).build(); 
     RequestConfig requestConfig = RequestConfig.custom().setConnectionRequestTimeout(5000) 
       .setSocketTimeout(10000).setConnectTimeout(10000).setCookieSpec("easy").setRedirectsEnabled(false) 
       .build(); 

     PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(); 
     cm.setMaxTotal(100); 
     cm.setDefaultMaxPerRoute(10); 

     client = HttpClients.custom().setConnectionManager(cm).setDefaultCookieSpecRegistry(r) 
       .setSSLSocketFactory(sslsf).setDefaultRequestConfig(requestConfig).build(); 
    } catch (Exception e) { 
     logger.error("http client init fail!", e); 
    } 
} 

public static String execute(HttpRequest httpRequest) { 
    CloseableHttpResponse response = null; 
    HttpGet httpGet = null; 
    HttpEntity httpEntity = null; 

    try { 
     httpGet = new HttpGet(httpRequest.getUrl()); 

     httpGet.setHeader("Connection", "close"); 
     if (httpRequest.isUseGzip()) { 
      httpGet.addHeader("Accept-Encoding", "gzip,deflate,sdch"); 
     } 
     if (!StringUtils.isEmpty(httpRequest.getContentType())) { 
      httpRequest.setContentType(httpRequest.getContentType()); 
     } 
     httpGet.addHeader("User-Agent", 
       "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63"); 

     response = client.execute(httpGet); 
     httpEntity = response.getEntity(); 

     byte[] bytes = null; 
     try { 
      bytes = EntityUtils.toByteArray(httpEntity); 
     } catch (Exception e) { 
      return null; 
     } 

     if (response.getStatusLine().getStatusCode() != 200) { 
      logger.warn("error! StatusCode: " + response.getStatusLine().getStatusCode() + ", url: " 
        + httpRequest.getUrl()); 
      return null; 
     } 

     @SuppressWarnings("deprecation") 
     String charset = EntityUtils.getContentCharSet(httpEntity); 
     if (StringUtils.isEmpty(charset)) { 
      Matcher match = charsetPatterm.matcher(new String(bytes)); 

      if (match.find()) { 
       charset = match.group(1); 
      } 
     } 
     if (!StringUtils.isEmpty(charset)) { 
      String strUtf8 = new String(new String(bytes, charset).getBytes(), GlobalConfig.ENCODING); 

      return StringEscapeUtils.unescapeHtml4(strUtf8); 
     } 
    } catch (Exception e) { 
     logger.error("error! url [" + httpRequest.getUrl() + "]", e); 
    } finally { 
     try { 
      if (httpEntity != null) { 
       EntityUtils.consume(httpEntity); 
      } 
      if (response != null) { 
       response.close(); 
      } 
      if (httpGet != null) { 
       httpGet.abort(); 
      } 
     } catch (Exception e) { 
      // ignore 
     } 
    } 

    return null; 
}

线程将阻塞.. jstack表明这样的。我只是用它来抓取一些网站。它发生在statusCode是404时。
Using Java Apache PoolingClientConnectionManager leaks the Memory,How to solve it? 我的问题与此类似。

"pool-1-thread-10" prio=10 tid=0x00007f7168003000 nid=0x3e4d waiting on condition [0x00007f717c398000] 
    java.lang.Thread.State: WAITING (parking) 
     at sun.misc.Unsafe.park(Native Method) 
     - parking to wait for <0x00000000e69d7350> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) 
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) 
     at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) 
     at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:133) 
     at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:282) 
     at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64) 
     at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:177) 
     at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:170) 
     at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:102) 
     at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.jav 
a:244) 
     at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:231) 
     at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:173) 
     at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195) 
     at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86) 
     at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108) 
     at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) 
     at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) 
     at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)

如何解决？

来源

2014-11-21 Darren Nizna

请将此标记为Java。 – 2014-11-21 14:32:33

线程正在等待并且您不希望它等待的问题？ – 2014-11-21 14:38:54

尝试使用实体正文以获取40x响应，因为流未关闭，因此不会释放连接，因此需要完全使用它（流） – Ironluca 2014-11-21 14:43:08

我有同样的问题，这是第一个答复。我用Ironluca的评论来解决我的问题，但觉得这需要一个完整的答案。

quick start guide有一个如何设置和使用基本HttpClient的简单例子。

// In order to ensure correct deallocation of system resources 
// the user MUST call CloseableHttpResponse#close() from a finally clause. 
// Please note that if response content is not fully consumed the underlying 
// connection cannot be safely re-used and will be shut down and discarded 
// by the connection manager. 
try { 
    System.out.println(response1.getStatusLine()); 
    HttpEntity entity1 = response1.getEntity(); 
    // do something useful with the response body 
    // and ensure it is fully consumed 
    EntityUtils.consume(entity1); 
} finally { 
    response1.close(); 
}

回想起你的问题，你似乎有密切和消费的代码。
您还有一个自定义连接池。这是默认的相同，但我想你有不同的配置。

来源

2015-01-16 00:03:47 pimaster

我遇到过类似的错误。看起来我们需要在返回之前使用HttpEntity。就你的情况而言，当你遇到200个不答复时，我发现你没有这样做。你只是返回null。你可能需要在返回之前使用它。另外，我会推荐使用HttpClientUtils.closeQuietly（response）;这是EntityUtils.consume的包装

来源

2015-09-29 15:24:48

的HttpClient 4.3阻塞连接池

回答

相关问题