2016-08-15 57 views
3

我使用tar球安装了Cassandra-2.2.7。 Cassandra工作正常,我在一个keyspace中创建了表。现在我想将这个表格数据转换成HDFS。我正在使用Sqoop-1.4.6。我一直在下面的库文件在$ SQOOP_HOME/lib中/,尝试使用Sqoop将数据从Cassandra传输到Hadoop时发生错误

apache-cassandra-2.2.7.jar 
apache-cassandra-thrift-2.2.7.jar 
cassandra-jdbc-1.2.5.jar 
cassandra-all-1.2.0.jar 
libthrift-0.8.0.jar 
thrift-server-0.3.7.jar 

然后我跑的命令,

bin/sqoop list-tables --driver org.apache.cassandra.cql.jdbc.CassandraDriver --connect jdbc:cassandra://127.0.0.1:7199/sample --username cassandra --password cassandra 

当我运行此命令时,我得到以下错误,

Warning: /usr/lib/sqoop/../hcatalog does not exist! HCatalog jobs will fail. 
 
Please set $HCAT_HOME to the root of your HCatalog installation. 
 
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail. 
 
Please set $ACCUMULO_HOME to the root of your Accumulo installation. 
 
16/08/15 16:52:41 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6 
 
16/08/15 16:52:41 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 
 
16/08/15 16:52:41 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 
 
16/08/15 16:52:41 INFO manager.SqlManager: Using default fetchSize of 1000 
 
SLF4J: Class path contains multiple SLF4J bindings. 
 
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/hadoop-1.2.1/lib/sf2hadoop.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
 
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/hadoop-1.2.1/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
 
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 
 
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder] 
 
16/08/15 16:52:41 ERROR manager.SqlManager: Error reading database metadata: java.sql.SQLNonTransientConnectionException: org.apache.thrift.transport.TTransportException 
 
java.sql.SQLNonTransientConnectionException: org.apache.thrift.transport.TTransportException 
 
\t at org.apache.cassandra.cql.jdbc.CassandraConnection.<init>(CassandraConnection.java:159) 
 
\t at org.apache.cassandra.cql.jdbc.CassandraDriver.connect(CassandraDriver.java:92) 
 
\t at java.sql.DriverManager.getConnection(DriverManager.java:664) 
 
\t at java.sql.DriverManager.getConnection(DriverManager.java:247) 
 
\t at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:885) 
 
\t at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) 
 
\t at org.apache.sqoop.manager.SqlManager.listTables(SqlManager.java:520) 
 
\t at org.apache.sqoop.tool.ListTablesTool.run(ListTablesTool.java:49) 
 
\t at org.apache.sqoop.Sqoop.run(Sqoop.java:143) 
 
\t at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) 
 
\t at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) 
 
\t at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) 
 
\t at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) 
 
\t at org.apache.sqoop.Sqoop.main(Sqoop.java:236) 
 
Caused by: org.apache.thrift.transport.TTransportException 
 
\t at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) 
 
\t at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) 
 
\t at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129) 
 
\t at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101) 
 
\t at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) 
 
\t at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378) 
 
\t at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297) 
 
\t at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204) 
 
\t at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) 
 
\t at org.apache.cassandra.thrift.Cassandra$Client.recv_describe_cluster_name(Cassandra.java:1247) 
 
\t at org.apache.cassandra.thrift.Cassandra$Client.describe_cluster_name(Cassandra.java:1235) 
 
\t at org.apache.cassandra.cql.jdbc.CassandraConnection.<init>(CassandraConnection.java:130) 
 
\t ... 13 more 
 
Could not retrieve tables list from server 
 
16/08/15 16:52:41 ERROR tool.ListTablesTool: manager.listTables() returned null

我以为这是某种节俭的rror,但无法纠正它,即使我用其他几个版本替换了lib-thrift jar。

回答

2

Cassandra JDBC驱动程序非常旧。上次我尝试使用它,我需要手动编译它对付最新的cassandra thrift库并修复一些编译错误。

我会建议使用一个Spark工作来传输数据,或者如果你必须使用sqoop,这取决于你正在做的DataStax企业什么包含自定义sqoop对Cassandra的驱动程序。

相关问题