在Ubuntu上设置单节点Hadoop集群14.04

-4

我是Hadoop的新手，并尝试在我的Ubuntu 14.04机器上设置单节点集群。我遵循了Michel Nall的教程。这个群集有一个问题......有时它会运行，有时会在没有任何原因的情况下停止。群集正在运行时...它不会允许我运行Java映射器和reducer。在Ubuntu上设置单节点Hadoop集群14.04

找不到问题。任何人都可以帮助我一步一步的安装和使用帮助。这里的代码和错误：

package org.myorg; 

import java.io.IOException; 
import java.util.*; 

import org.apache.hadoop.fs.Path; 
import org.apache.hadoop.conf.*; 
import org.apache.hadoop.io.*; 
import org.apache.hadoop.mapred.*; 
import org.apache.hadoop.util.*; 

public class WordCount { 

    public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> { 
    private final static IntWritable one = new IntWritable(1); 
    private Text word = new Text();  
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { 
     String line = value.toString(); 
     StringTokenizer tokenizer = new StringTokenizer(line); 
     while (tokenizer.hasMoreTokens()) { 
     word.set(tokenizer.nextToken()); 
     output.collect(word, one); 
     } 
    } 
    } 

    public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> { 
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { 
     int sum = 0; 
     while (values.hasNext()) { 
     sum += values.next().get(); 
     } 
     output.collect(key, new IntWritable(sum)); 
    } 
    } 

    public static void main(String[] args) throws Exception { 
    JobConf conf = new JobConf(WordCount.class); 
    conf.setJobName("wordcount"); 

    conf.setOutputKeyClass(Text.class); 
    conf.setOutputValueClass(IntWritable.class); 

    conf.setMapperClass(Map.class); 
    conf.setCombinerClass(Reduce.class); 
    conf.setReducerClass(Reduce.class); 

    conf.setInputFormat(TextInputFormat.class); 
    conf.setOutputFormat(TextOutputFormat.class); 

    FileInputFormat.setInputPaths(conf, new Path(args[0])); 
    FileOutputFormat.setOutputPath(conf, new Path(args[1])); 

    JobClient.runJob(conf); 
    } 
}

错误是：

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory 
    at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:139) 
    at WordCount.main(WordCount.java:36) 
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 
    ... 2 more

来源

2015-04-01 Meenal borkar

日志显示什么？你怎么知道它没有运行？您在所需的服务中是否有异常？ – 2015-04-01 10:19:38

Meenal，欢迎来到stackoverflow。为了提高获得答案的机会，请尽可能多地发布错误日志等详细信息，以支持您所描述的每个问题。 – CKing 2015-04-01 10:21:20

这是我的代码 – 2015-04-01 11:00:23

你得到的连接除外，它的原因是：抛出java.lang.ClassNotFoundException：org.apache.commons.logging

上级LogFactory不存在org/apache/commons/logging/path。你必须确定它是否存在。

来源

2015-04-01 12:05:55

在Ubuntu上设置单节点Hadoop集群14.04

回答

相关问题