Hadoop地图作业失败，出现com.datastax.driver.core.exceptions.NoHostAvailableException

Question

我正在尝试使用hadoop map-reduce在cassandra中存储的数据运行分析。为此，我使用可通过maven依赖项cassandra-all获得的类CqlInputFormat。目前，我们在生产环境中一直使用这种依赖关系的2.0.10版本。另外，我们使用版本为2.1.1的caassandra-driver-core。

现在，当我向我的jobtracker提交一个简单的map-reduce作业时，我的所有mapper任务都会失败，并显示以下异常。

这里需要注意的另一件重要的事情是，如果我使用cassandra-all 2.0.6中的CqlPagingInputFormat，那么一切正常。但是这个输入格式在后面的版本中已经被删除了。

com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: 10.40.242.174:9042 (com.datastax.driver.core.exceptions.DriverException: Timeout while trying to acquire available connection (you may want to increase the driver number of per-host connections))) 
    at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:65) 
    at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:258) 
    at com.datastax.driver.core.Cluster.connect(Cluster.java:267) 
    at org.apache.cassandra.hadoop.cql3.CqlRecordReader.initialize(CqlRecordReader.java:137) 
    at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:521) 
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) 
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) 
    at org.apache.hadoop.mapred.Child.main(Child.java:249) 
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: 10.40.242.174:9042 (com.datastax.driver.core.exceptions.DriverException: Timeout while trying to acquire available connection (you may want to increase the driver number of per-host connections))) 
    at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:103) 
    at com.datastax.driver.core.SessionManager.execute(SessionManager.java:446) 
    at com.datastax.driver.core.SessionManager.executeQuery(SessionManager.java:482) 
    at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:88) 
    at com.datastax.driver.core.AbstractSession.executeAsync(AbstractSession.java:60) 
    at com.datastax.driver.core.Cluster.connect(Cluster.java:260) 
    ... 9 more 

任何输入是最受欢迎的。

Answer 1

我在使用cassandra-all-2.0.10时遇到了完全相同的问题，但问题已通过使用版本cassandra-all-2.0.12得到解决。

我想修正这一承诺： Potentially use non-local hosts in CqlConfigHelper