图减少工作失败

我运行图减少了Cloudera工作图减少工作失败

代码：

public class WordCount { 
    public static class Map extends MapReduceBase implements Mapper<LongWritable,Text,Text,IntWritable>{ 
     private final static IntWritable one = new IntWritable(1); 
     private Text word = new Text(); 

     public void map(LongWritable key, Text value, OutputCollector<Text,IntWritable> output, Reporter reporter) throws IOException{ 
      String line = value.toString(); 
      StringTokenizer tokenizer = new StringTokenizer(line); 

      while(tokenizer.hasMoreTokens()){ 
       word.set(tokenizer.nextToken()); 
       output.collect(word, one); 
      } 
     } 
    } 

    public static class Reduce extends MapReduceBase implements Reducer<Text,IntWritable,Text,IntWritable>{ 

     public void reduce (Text key, Iterator<IntWritable> values,OutputCollector<Text,IntWritable> output,Reporter reporter)throws IOException { 
      int sum = 0; 
      while(values.hasNext()){ 
       sum += values.next().get(); 
      } 
      output.collect(key , new IntWritable(sum)); 

     } 
    } 

    public static void main(String[] args) throws Exception{ 
     JobConf conf = new JobConf(WordCount.class); 
     conf.setJobName("WordCount"); 
     conf.setJarByClass(WordCount.class); 
     conf.setOutputKeyClass(Text.class); 
     conf.setOutputValueClass(IntWritable.class); 

     conf.setMapperClass(Map.class); 
     conf.setCombinerClass(Reduce.class); 
     conf.setReducerClass(Reducer.class); 


     conf.setInputFormat(TextInputFormat.class); 
     conf.setOutputFormat(TextOutputFormat.class); 

     FileInputFormat.setInputPaths(conf, new Path(args[0])); 
     FileOutputFormat.setOutputPath(conf, new Path (args[1])); 

     JobClient.runJob(conf); 


    } 

}

命令：

[[email protected] bin]$ hadoop jar /home/cloudera/workspace/Test/bin/WordCount.jar WordCount /user/cloudera/input /user/cloudera/output

错误：

14/02/28 22:49:34 INFO mapred.JobClient: Task Id : attempt_201402281818_0016_r_000000_0, Status : FAILED 
java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.mapred.Reducer.<init>() 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131) 
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:469) 
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:447) 
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:396) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) 
    at org.apache.hadoop.mapred.Child.main(Child.java:262) 
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.mapred.Reducer.<init>() 
    at java.lang.Class.getConstructor0(Class.java:2706) 
    at java.lang.Class.getDeclaredConstructor(Class.java:1985) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125) 
    ... 7 more

请帮助。

来源

2014-03-01 Brainchild

只是提醒你不应该使用不赞成使用的库。避免使用org.apache.hadoop.mapred包。反而使用来自org.apache.hadoop.mapreduce –

也许你应该尝试解决此行代码：

conf.setReducerClass(Reducer.class);

到

conf.setReducerClass(Reduce.class);

因为我注意到你的减速机类的名字是减少。

来源

2014-03-01 07:10:24 iamdvd09

的库谢谢。接得好。 – Brainchild

图减少工作失败

回答

相关问题