猪输出位置验证失败

在执行下面给出的脚本时，我收到如下所述的错误。我尝试使用现有的目录以及不存在的目录，但我仍然得到相同的错误。正如我刚刚开始写猪脚本，请帮我解决这个问题。我试图加载文件，并在应用快速转换后存储在不同的位置。猪输出位置验证失败

代码

SET output.compression.enabled true; 
SET mapred.map.output.compression.codec org.apache.hadoop.io.compress.SnappyCodec; 
A = LOAD '/abc/def/' using PigStorage(); 
Store A into '/abc/def/snappy/' using PigStorage();

错误

Pig Stack Trace 
--------------- 
ERROR 2116: 
<file snappy_conversion.pig, line 4, column 0> Output Location Validation Failed for: '/abc/def/snappy/ 

org.apache.pig.impl.plan.VisitorException: ERROR 2116: 
<file snappy_conversion.pig, line 4, column 0> Output Location Validation Failed for: '/abc/def/snappy/ 
     at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:75) 
     at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:66) 
     at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64) 
     at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66) 
     at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53) 
     at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52) 
     at org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45) 
     at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:285) 
     at org.apache.pig.PigServer.compilePp(PigServer.java:1324) 
     at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249) 
     at org.apache.pig.PigServer.execute(PigServer.java:1241) 
     at org.apache.pig.PigServer.executeBatch(PigServer.java:335) 
     at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:137) 
     at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198) 
     at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170) 
     at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84) 
     at org.apache.pig.Main.run(Main.java:604) 
     at org.apache.pig.Main.main(Main.java:157) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
     at java.lang.reflect.Method.invoke(Method.java:597) 
     at org.apache.hadoop.util.RunJar.main(RunJar.java:208) 
Caused by: java.lang.NullPointerException 
     at java.lang.Class.forName0(Native Method) 
     at java.lang.Class.forName(Class.java:169) 
     at org.apache.pig.builtin.PigStorage.setStoreLocation(PigStorage.java:396) 
     at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:68) 
     ... 22 more 
================================================================================

来源

2014-04-08 Piyush Jindal

的问题是，你设置了错误的配置。正确的设置是

SET output.compression.codec org.apache.hadoop.io.compress.SnappyCodec;

mapred.map.output.compression.codec用于设置其是由映射器输出的中间文件的压缩格式。

你可以在这里阅读代码来获得更多的信息：http://www.grepcode.com/file/repo1.maven.org/maven2/org.apache.pig/pig/0.12.0/org/apache/pig/builtin/PigStorage.java?av=h#440

来源

2014-04-08 05:35:19 zsxwing

感谢@zsxwing –

猪输出位置验证失败

回答

相关问题