在Hadoop 2上的猪15上运行时嵌入的猪错误

每当我从终端运行任何apache猪代码everythig进行顺利，我得到的结果。所以我得出结论说我的Pig 0.15.0和Hadoop 2.7.0的安装是可以的。问题是，当我从Java代码中运行pigServer：在Hadoop 2上的猪15上运行时嵌入的猪错误

PigServer pigServer = new PigServer(ExecType.MAPREDUCE, conf); 
pigServer.setBatchOn(); 
pigServer.debugOff(); 
pigServer.setJobName(JobId); 
pigServer.registerScript(scriptUrl, params); 
pigServer.executeBatch();

我的Maven的依赖关系是：

 <dependency> 
      <groupId>org.apache.pig</groupId> 
      <artifactId>pig</artifactId> 
      <version>0.15.0</version> 
     </dependency> 
     <dependency> 
      <groupId>org.apache.hadoop</groupId> 
      <artifactId>hadoop-client</artifactId> 
      <version>2.7.0</version> 
     </dependency>

我碰到下面的错误。

WARN org.apache.pig.backend.hadoop20.PigJobControl - falling back to default JobControl (not using hadoop 0.20 ?) 
java.lang.NoSuchFieldException: runnerState 
    at java.lang.Class.getDeclaredField(Class.java:1948) 
    at org.apache.pig.backend.hadoop20.PigJobControl.<clinit>(PigJobControl.java:51) 
    at org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:100) 
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:313) 
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:199) 
    at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:277) 
    at org.apache.pig.PigServer.launchPlan(PigServer.java:1367) 
    at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1352) 
    at org.apache.pig.PigServer.execute(PigServer.java:1341) 
    at org.apache.pig.PigServer.executeBatch(PigServer.java:392) 
    at org.apache.pig.PigServer.executeBatch(PigServer.java:375) 
    at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:170) 
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:232) 
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:203) 
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) 
    at org.apache.pig.Main.run(Main.java:479)

我曾经在Hadoop 1上运行上面的代码，它正在工作，但现在不是。

来源

2015-08-13 Abdulrahman

默认情况下，猪使用Hadoop的0.20版本，所以在运行猪假设您正在使用Hadoop 0.20所以你得到的是错误

您可以通过设置HADOOP_HOME与不同版本的Hadoop运行猪指向目录你已经安装了Hadoop。如果不设置HADOOP_HOME，默认情况下，猪将与

Run Configurations-->ClassPath-->User Entries-->Advanced-->Add ClassPath Variables-->New-->Name(HADOOP_HOME)-->Path(You Hadoop directory path)

Maven依赖需要

的嵌入式版本，目前的Hadoop 0.20.2 .--写在Apache的猪网站 https://pig.apache.org/docs/r0.9.2/start.html

设置HADOOP_HOME在Eclipse中运行

<dependencies> 
    <dependency> 
    <groupId>org.apache.hadoop</groupId> 
    <artifactId>hadoop-hdfs</artifactId> 
    <version>2.7.1</version> 
</dependency> 
<dependency> 
    <groupId>org.apache.hadoop</groupId> 
    <artifactId>hadoop-client</artifactId> 
    <version>2.7.1</version> 
</dependency> 

<dependency> 
    <groupId>commons-io</groupId> 
    <artifactId>commons-io</artifactId> 
    <version>2.4</version> 
</dependency> 
<dependency> 
    <groupId>log4j</groupId> 
    <artifactId>log4j</artifactId> 
    <version>1.2.16</version> 
</dependency> 
<dependency> 
    <groupId>org.apache.pig</groupId> 
    <artifactId>pig</artifactId> 
    <version>0.15.0</version> 
</dependency> 

<dependency> 
    <groupId>org.antlr</groupId> 
    <artifactId>antlr-runtime</artifactId> 
    <version>3.4</version> 
</dependency> 
</dependencies>

来源

2016-03-17 14:41:23

在Hadoop 2上的猪15上运行时嵌入的猪错误

回答

相关问题