Oozie和sqoop配置问题

我想设置一个oozie和sqoop工作流（我想将mySql数据备份到我的hdfs中）。但是当我尝试启动我的工作时，我陷入了困境。Oozie和sqoop配置问题

我正在使用hadoop2（工作hdfs节点），oozie的最后一个版本。我在计算机上安装了oozie服务器（我想在部署它之前对其进行测试），在oozie conf上使用hdfs配置（core-site.xml，hdfs-site.xml，yarn-site.xml，mapred-site.xml）/haddop-conf dir）和我的hdfs在服务器上。

我已经做了基本工作流程（测试目的，我只是想看看是否sqoop工作）这样的：

<workflow-app name="Sqoop" xmlns="uri:oozie:workflow:0.4"> 
    <start to="Sqoop"/> 
    <action name="Sqoop"> 
     <sqoop xmlns="uri:oozie:sqoop-action:0.2"> 
     <job-tracker>yarn.resourcemanager.address:8040</job-tracker> 
      <name-node>hdfs://hdfs-server:54310</name-node> 
      <command>job --list</command> 
     </sqoop> 
     <ok to="end"/> 
     <error to="kill"/> 
    </action> 
    <kill name="kill"> 
     <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> 
    </kill> 
    <end name="end"/> 
</workflow-app>

我把这个工作流进我的HDFS。我已经做了Java代码开始我的工作：

OozieClient wc = new OozieClient("http://localhost:11000/oozie"); 

     Properties conf = wc.createConfiguration(); 
     conf.setProperty(OozieClient.APP_PATH, "hdfs://hdfs_server:54310/hive/testSqoop/sqoop-workflow.xml"); 
     conf.setProperty("queueName", "default"); 

     try { 
      String jobId = wc.run(conf); 
      System.out.println("Workflow job submitted"); 

      while (wc.getJobInfo(jobId).getStatus() == WorkflowJob.Status.RUNNING) { 
       System.out.println("Workflow job running ..."); 
       System.out.println("..." + wc.getJobInfo(jobId).getStatus().toString()); 
       Thread.sleep(10 * 1000); 
      } 
      System.out.println("Workflow job completed ..."); 
      System.out.println(wc.getJobInfo(jobId)); 
     } catch (Exception r) { 
      r.printStackTrace(); 
     }

在Oozie的webinterface我可以看到我的工作运行

2013-05-28 12:42:30,004 INFO ActionStartXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]:start:] Start action [[email protected]:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2013-05-28 12:42:30,008 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action status=DONE 
2013-05-28 12:42:30,009 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action updated in DB! 
2013-05-28 12:42:30,192 INFO ActionStartXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2013-05-28 12:42:31,389 WARN SqoopActionExecutor:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] credentials is null for the action 
2013-05-28 12:42:42,942 INFO SqoopActionExecutor:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] checking action, external ID [job_1369126414383_0003] status [RUNNING] 
2013-05-28 12:42:42,945 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] [***[email protected]***]Action status=RUNNING 
2013-05-28 12:42:42,946 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] [***[email protected]***]Action updated in DB! 
2013-05-28 12:47:43,034 INFO KillXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] STARTED WorkflowKillXCommand for jobId=0000000-130528124140043-oozie-anth-W 
2013-05-28 12:47:43,328 WARN CoordActionUpdateXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100 
2013-05-28 12:47:43,328 INFO KillXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] ENDED WorkflowKillXCommand for jobId=0000000-130528124140043-oozie-anth-W

当我检查纱webinterface，我可以看到我的工作，但与状态失败

Application application_1369126414383_0003 failed 1 times due to AM Container for appattempt_1369126414383_0003_000001 exited with exitCode: 1 due to: .Failing this attempt.. Failing the application.

我真的不知道什么是错的。我需要你的建议。

谢谢〜

来源

2013-05-29 AnthonyC

在Web界面中找到失败的任务尝试日志，您将得到任务失败的异常。 – SNeumann

你必须检查作业日志：

$ oozie job -log <coord_job_id>

明白发生了什么。

来源

2016-09-13 19:31:45 ozw1z5rd

Oozie和sqoop配置问题

回答

相关问题