地图降低输出不正确

Question

我有一个输入文件

UserId|TrackId|Shared|Radio|Skip 
111115|222|0|1|0 
111113|225|1|0|0 
111117|223|0|1|1 
111115|225|1|0|0 

我需要添加共享以及无线列所有轨道ID 输出应该

222,1 
223,1 
225,2 

通过以下程序我写的，我得到

222,1 
223,1 
225,1 
225,2. 

不知道错误是什么

这是我的计划

public class Total { 

public static class ListenMap extends Mapper<LongWritable, Text, Text, IntWritable> 
{ 
    public void map(LongWritable key, Text values, Context context) throws IOException, InterruptedException 
    { 
     String slt= values.toString(); 
     String arr[]= slt.split("[|]"); 
     String trackid= arr[1]; 
     String shared= arr[2]; 
     String radio= arr[3]; 
     int sharenum= Integer.parseInt(shared); 
     int radionum= Integer.parseInt(radio); 
     int total= sharenum+radionum; 
     context.write(new Text(trackid), new IntWritable(total)); 
    } 
} 


public static class ListenReduce extends Reducer<Text, IntWritable, Text, IntWritable> 
{ 
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException 
    { 
     int sum=0; 
     for(IntWritable x: values) 
     { 
      sum+=x.get(); 
      context.write(key, new IntWritable(sum)); 

     } 
    } 
} 
public static void main(String[] args) throws IOException, InterruptedException, ClassNotFoundException{ 
    Configuration conf= new Configuration(); 
    Job job= new Job(conf, "listen"); 

    job.setJarByClass(Total.class); 
    job.setMapperClass(ListenMap.class); 
    job.setReducerClass(ListenReduce.class); 

    job.setOutputKeyClass(Text.class); 
    job.setOutputValueClass(IntWritable.class); 

    job.setInputFormatClass(TextInputFormat.class); 
    job.setOutputFormatClass(TextOutputFormat.class); 

    FileInputFormat.addInputPath(job, new Path(args[0])); 
    FileOutputFormat.setOutputPath(job, new Path(args[1])); 

    System.exit(job.waitForCompletion(true)? 1:0); 

    } 
} 

Answer 1

在循环外部移动context.write(key, new IntWritable(sum));，除非您希望在增加它之后打印每个总和值。

我打算假设这段时间在提问时是错字，因为您的代码没有添加该代码。

Answer 2

你写出来的结果在for循环。将它移动到外部：

public static class ListenReduce extends Reducer<Text, IntWritable, Text, IntWritable> 
{ 
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException 
    { 
     int sum=0; 
     for(IntWritable x: values) 
     { 
      sum+=x.get(); 
     } 
     context.write(key, new IntWritable(sum)); 
    } 
} 

Answer 3

你正在为循环写你的上下文对象，这就是为什么你可以看到重复的键。

取而代之它应该只为每个键写入一次。

public static class ListenReduce extends Reducer<Text, IntWritable, Text, IntWritable> 
{ 
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException 
    { 
     int sum=0; 
     for(IntWritable x: values) 
     { 
      sum+=x.get(); 
     } 
     // Write it here 
     context.write(key, new IntWritable(sum)); 
    } 
}