Sqoop错误而载入数据从蜂巢到MySQL

我正在sqoop错误而从蜂巢到MySQLSqoop错误而载入数据从蜂巢到MySQL

加载数据

错误信息是：

java.lang.NumberFormatException：对于输入字符串

= =

hive > CREATE EXTERNAL TABLE IF NOT EXISTS test (
id int, 
name string 
) 
ROW FORMAT DELIMITED 
FIELDS TERMINATED BY '\t' 
LINES TERMINATED BY '\n' 
STORED AS TEXTFILE 
LOCATION 
'/user/cloudera/test';

VI试验：

1  a 
2  b

hadoop fs -put test /user/cloudera

mysql> CREATE TABLE `foo` (`id` int(11) , `name` varchar(30))

sqoop出口--connect的jdbc：mysql的：//本地主机/测试--table FOO - m 1 --export-dir/user/cloudera/test

日志：

14/05/13 07:18:52 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 
14/05/13 07:18:52 INFO tool.CodeGenTool: Beginning code generation 
14/05/13 07:18:53 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `foo` AS t LIMIT 1 
14/05/13 07:18:53 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `foo` AS t LIMIT 1 
14/05/13 07:18:53 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-0.20-mapreduce 
14/05/13 07:18:53 INFO orm.CompilationManager: Found hadoop core jar at: /usr/lib/hadoop-0.20-mapreduce/hadoop-core.jar 
Note: /tmp/sqoop-cloudera/compile/e6582e332bf9e0eedfb641f14d866599/foo.java uses or overrides a deprecated API. 
Note: Recompile with -Xlint:deprecation for details. 
14/05/13 07:18:56 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/e6582e332bf9e0eedfb641f14d866599/foo.jar 
14/05/13 07:18:56 INFO mapreduce.ExportJobBase: Beginning export of foo 
14/05/13 07:18:59 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 
14/05/13 07:19:00 INFO input.FileInputFormat: Total input paths to process : 1 
14/05/13 07:19:00 INFO input.FileInputFormat: Total input paths to process : 1 
14/05/13 07:19:00 INFO mapred.JobClient: Running job: job_201405081447_0046 
14/05/13 07:19:01 INFO mapred.JobClient: map 0% reduce 0% 
14/05/13 07:19:14 INFO mapred.JobClient: Task Id : attempt_201405081447_0046_m_000000_0, Status : FAILED 
java.io.IOException: Can't export data, please check task tracker logs 
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) 
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) 
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) 
    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) 
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) 
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:396) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) 
    at org.apache.hadoop.mapred.Child.main(Child.java:262) 
Caused by: java.lang.NumberFormatException: For input string: "1 a" 
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) 
    at java.lang.Integer.parseInt(Integer.java:458) 
    at java 
14/05/13 07:19:20 INFO mapred.JobClient: Task Id : attempt_201405081447_0046_m_000000_1, Status : FAILED 
java.io.IOException: Can't export data, please check task tracker logs 
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) 
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) 
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) 
    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) 
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) 
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:396) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) 
    at org.apache.hadoop.mapred.Child.main(Child.java:262) 
Caused by: java.lang.NumberFormatException: For input string: "1 a" 
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) 
    at java.lang.Integer.parseInt(Integer.java:458) 
    at java 
14/05/13 07:19:28 INFO mapred.JobClient: Task Id : attempt_201405081447_0046_m_000000_2, Status : FAILED 
java.io.IOException: Can't export data, please check task tracker logs 
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) 
    at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) 
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) 
    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) 
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) 
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:396) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) 
    at org.apache.hadoop.mapred.Child.main(Child.java:262) 
Caused by: java.lang.NumberFormatException: For input string: "1 a" 
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) 
    at java.lang.Integer.parseInt(Integer.java:458) 
    at java

任何帮助吗？

谢谢！

来源

2014-05-13 Rio

之前执行导出你检查蜂箱表显示正确的结果。你可以检查通过从测试表中选择ID显示只有ID？它看起来像字段分隔符没有在输入文件中正确设置。 – Arun

是的，我检查了蜂巢表，并有数据。我从hive表中选择了id。任何建议？ – Rio

当您检查时，请不要使用从测试中选择** \ ***。而是从测试中选择id，name。 – javadba

放置文件的位置看起来不正确。对于“测试”表，您应该在目录测试下放置一个文件。但是，你的命令

hadoop fs -put test /user/cloudera

创建文件称为测试。

您可能会发现更多的成功如下：

hadoop fs -mkdir /user/cloudera/test 
hadoop dfs -put test /user/cloudera/test

来源

2014-05-13 21:15:30 javadba

Sqoop错误而载入数据从蜂巢到MySQL

回答

相关问题