2017-08-10 78 views
0

当我尝试将配置单元分区导出到Oracle数据库时,导出失败,我无法弄清楚为什么...有人可以帮助我吗?Sqoop导出失败 - 未找到ExportBatchOutputFormat

的My Oracle表:

CREATE TABLE "GFPROGNOSE"."HADOOP_TEST" 
    ("COLUMN1" VARCHAR2(255), 
    "COLUMN2" VARCHAR2(255), 
    "COLUMN3" VARCHAR2(255), 
    "COLUMN4" VARCHAR2(255), 
    "DS_YEAR" VARCHAR2(255), 
    "DS_MONTH" VARCHAR2(255) 
    ) 

我的蜂巢表: 拥有Oracle表中作为字符串中的所有列和其他列

我东张西望出口COMAND:

sudo -u hdfs sqoop-export --connect jdbc:oracle:thin:@192.168.145.210:1521:dmine 
--table HADOOP_TEST --username USERNAME --password xxxxxx --num-mappers 1 
--update-mode allowinsert --skip-dist-cache --hcatalog-database default 
--hcatalog-table adobe_archive --hcatalog-partition-keys ds_year,ds_month,ds_day 
--hcatalog-partition-values 2017,06,01 

命令行输出:

17/08/10 10:03:56 INFO mapreduce.Job: map 0% reduce 0% 
17/08/10 10:03:56 INFO mapreduce.Job: Job job_1501840909803_1851 failed with state FAILED due to: Application application_1501840909803_1851 failed 2 times due to AM Container for appattempt_1501840909803_1851_000002 exited with exitCode: 1 
For more detailed output, check the application tracking page: http://master.joinplus.local:8088/cluster/app/application_1501840909803_1851 Then click on links to logs of each attempt. 
Diagnostics: Exception from container-launch. 
Container id: container_e17_1501840909803_1851_02_000001 
Exit code: 1 
Stack trace: ExitCodeException exitCode=1: 
     at org.apache.hadoop.util.Shell.runCommand(Shell.java:944) 
     at org.apache.hadoop.util.Shell.run(Shell.java:848) 
     at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1142) 
     at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:237) 
     at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:317) 
     at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:83) 
     at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
     at java.lang.Thread.run(Thread.java:745) 

应用程序日志:

2017-08-10 10:03:55,839 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null 
2017-08-10 10:03:55,880 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found 
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found 
     at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:520) 
     at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:500) 
     at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1601) 
     at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:500) 
     at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:286) 
     at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) 
     at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1559) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:422) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) 
     at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1556) 
     at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1487) 
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found 
     at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2241) 
     at org.apache.hadoop.mapreduce.task.JobContextImpl.getOutputFormatClass(JobContextImpl.java:222) 
     at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:516) 
     ... 11 more 
Caused by: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found 
     at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2147) 
     at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2239) 
     ... 13 more 

回答

0

现在,它的工作原理。 问题在于sqoop命令中的参数--skip-dist-cache
它在那里,因为我复制了来自falcon生成的oozie工作流的命令,用于调试目的。