为什么Windows7中的hadoop引发了RuntimeException：配置对象时出错

我试图在windows7中以伪分布模式运行hadoop程序，并且引发了RuntimeException。我不知道为什么以及如何解决它。为什么Windows7中的hadoop引发了RuntimeException：配置对象时出错

我使用python进行mapreduce工作，而不是使用hadoop-streaming实用程序的java（https://hadoop.apache.org/docs/current/hadoop-streaming/HadoopStreaming.html）。我跑hadoop jar %HADOOP_HOME%\share\hadoop\tools\lib\hadoop-streaming-2.6.2.jar -mapper mapper.py -reducer reducer.py -input myinput -output myoutput -file mapper.py -file reducer.py和java.lang.RuntimeException: Error in configuring object已被提出和工作失败。

我的hadoop版本是2.6.2，hadoop建立在windows7 x64上，而不是在虚拟机或cygwin上。

首先，我使用hadoop的标准测试程序来测试hadoop是否安装成功。我跑yarn jar %HADOOP_HOME%/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.2.jar pi 16 1000和输出是正确的。 Hadoop和MapReduce的工作。

其次，mapper.py和reducer.py已经在命令提示符下使用命令cat data.txt | python mapper.py | sort | python reducer.py进行了测试，并且输出是正确的。因此，它不会是Python程序的错误。

我尝试将'hadoop.tmp.dir'属性更改为本地目录和/ tmp，但都不起作用。我不知道为什么引发异常以及如何解决它。非常感谢任何人都可以帮助我！

完整的异常日志如下：

16/03/25 18:47:46 WARN streaming.StreamJob: -file option is deprecated, please use generic option -files instead. 
packageJobJar: [mapper.py, reducer.py, /C:/Users/chensk/AppData/Local/Temp/hadoop-unjar5079915111177028733/] [] C:\Users\chensk\AppData\Local\Temp\streamjob6081649198401076264.jar tmpDir=null 
16/03/25 18:47:47 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 
16/03/25 18:47:48 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 
16/03/25 18:47:50 INFO mapred.FileInputFormat: Total input paths to process : 1 
16/03/25 18:47:50 INFO mapreduce.JobSubmitter: number of splits:2 
16/03/25 18:47:50 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1458902651843_0002 
16/03/25 18:47:51 INFO impl.YarnClientImpl: Submitted application application_1458902651843_0002 
16/03/25 18:47:51 INFO mapreduce.Job: The url to track the job: http://skyfalls:8088/proxy/application_1458902651843_0002/ 
16/03/25 18:47:51 INFO mapreduce.Job: Running job: job_1458902651843_0002 
16/03/25 18:47:59 INFO mapreduce.Job: Job job_1458902651843_0002 running in uber mode : false 
16/03/25 18:47:59 INFO mapreduce.Job: map 0% reduce 0% 
16/03/25 18:48:07 INFO mapreduce.Job: Task Id : attempt_1458902651843_0002_m_000001_0, Status : FAILED 
Error: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:446) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) 
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656) 
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 9 more 
Caused by: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) 
    ... 14 more 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 17 more 
Caused by: java.lang.RuntimeException: configuration exception 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222) 
    at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66) 
    ... 22 more 
Caused by: java.io.IOException: Cannot run program "G:\Programs\hadoop\tmp\nm-local-dir\usercache\chensk\appcache\application_1458902651843_0002\container_1458902651843_0002_01_000003\.\mapper.py": CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209) 
    ... 23 more 
Caused by: java.io.IOException: CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessImpl.create(Native Method) 
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:385) 
    at java.lang.ProcessImpl.start(ProcessImpl.java:136) 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) 
    ... 24 more 

16/03/25 18:48:08 INFO mapreduce.Job: map 50% reduce 0% 
16/03/25 18:48:08 INFO mapreduce.Job: Task Id : attempt_1458902651843_0002_m_000000_0, Status : FAILED 
Error: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:446) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) 
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656) 
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 9 more 
Caused by: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) 
    ... 14 more 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 17 more 
Caused by: java.lang.RuntimeException: configuration exception 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222) 
    at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66) 
    ... 22 more 
Caused by: java.io.IOException: Cannot run program "G:\Programs\hadoop\tmp\nm-local-dir\usercache\chensk\appcache\application_1458902651843_0002\container_1458902651843_0002_01_000002\.\mapper.py": CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209) 
    ... 23 more 
Caused by: java.io.IOException: CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessImpl.create(Native Method) 
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:385) 
    at java.lang.ProcessImpl.start(ProcessImpl.java:136) 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) 
    ... 24 more 

16/03/25 18:48:09 INFO mapreduce.Job: map 0% reduce 0% 
16/03/25 18:48:14 INFO mapreduce.Job: Task Id : attempt_1458902651843_0002_m_000001_1, Status : FAILED 
Error: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:446) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) 
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656) 
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 9 more 
Caused by: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) 
    ... 14 more 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 17 more 
Caused by: java.lang.RuntimeException: configuration exception 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222) 
    at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66) 
    ... 22 more 
Caused by: java.io.IOException: Cannot run program "G:\Programs\hadoop\tmp\nm-local-dir\usercache\chensk\appcache\application_1458902651843_0002\container_1458902651843_0002_01_000004\.\mapper.py": CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209) 
    ... 23 more 
Caused by: java.io.IOException: CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessImpl.create(Native Method) 
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:385) 
    at java.lang.ProcessImpl.start(ProcessImpl.java:136) 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) 
    ... 24 more 

16/03/25 18:48:16 INFO mapreduce.Job: Task Id : attempt_1458902651843_0002_m_000000_1, Status : FAILED 
Error: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:446) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) 
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656) 
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 9 more 
Caused by: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) 
    ... 14 more 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 17 more 
Caused by: java.lang.RuntimeException: configuration exception 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222) 
    at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66) 
    ... 22 more 
Caused by: java.io.IOException: Cannot run program "G:\Programs\hadoop\tmp\nm-local-dir\usercache\chensk\appcache\application_1458902651843_0002\container_1458902651843_0002_01_000005\.\mapper.py": CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209) 
    ... 23 more 
Caused by: java.io.IOException: CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessImpl.create(Native Method) 
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:385) 
    at java.lang.ProcessImpl.start(ProcessImpl.java:136) 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) 
    ... 24 more 

16/03/25 18:48:23 INFO mapreduce.Job: Task Id : attempt_1458902651843_0002_m_000001_2, Status : FAILED 
Error: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:446) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) 
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656) 
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 9 more 
Caused by: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) 
    ... 14 more 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 17 more 
Caused by: java.lang.RuntimeException: configuration exception 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222) 
    at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66) 
    ... 22 more 
Caused by: java.io.IOException: Cannot run program "G:\Programs\hadoop\tmp\nm-local-dir\usercache\chensk\appcache\application_1458902651843_0002\container_1458902651843_0002_01_000006\.\mapper.py": CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209) 
    ... 23 more 
Caused by: java.io.IOException: CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessImpl.create(Native Method) 
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:385) 
    at java.lang.ProcessImpl.start(ProcessImpl.java:136) 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) 
    ... 24 more 

16/03/25 18:48:25 INFO mapreduce.Job: Task Id : attempt_1458902651843_0002_m_000000_2, Status : FAILED 
Error: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:446) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) 
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656) 
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 9 more 
Caused by: java.lang.RuntimeException: Error in configuring object 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) 
    ... 14 more 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) 
    ... 17 more 
Caused by: java.lang.RuntimeException: configuration exception 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222) 
    at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66) 
    ... 22 more 
Caused by: java.io.IOException: Cannot run program "G:\Programs\hadoop\tmp\nm-local-dir\usercache\chensk\appcache\application_1458902651843_0002\container_1458902651843_0002_01_000007\.\mapper.py": CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047) 
    at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209) 
    ... 23 more 
Caused by: java.io.IOException: CreateProcess error=193, %1 ??????完?? Win32 ??車 
    at java.lang.ProcessImpl.create(Native Method) 
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:385) 
    at java.lang.ProcessImpl.start(ProcessImpl.java:136) 
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) 
    ... 24 more 

16/03/25 18:48:32 INFO mapreduce.Job: map 100% reduce 100% 
16/03/25 18:48:34 INFO mapreduce.Job: Job job_1458902651843_0002 failed with state FAILED due to: Task failed task_1458902651843_0002_m_000001 
Job failed as tasks failed. failedMaps:1 failedReduces:0 

16/03/25 18:48:34 INFO mapreduce.Job: Counters: 13 
    Job Counters 
     Failed map tasks=7 
     Killed map tasks=1 
     Launched map tasks=8 
     Other local map tasks=6 
     Data-local map tasks=2 
     Total time spent by all maps in occupied slots (ms)=52274 
     Total time spent by all reduces in occupied slots (ms)=0 
     Total time spent by all map tasks (ms)=52274 
     Total vcore-seconds taken by all map tasks=52274 
     Total megabyte-seconds taken by all map tasks=53528576 
    Map-Reduce Framework 
     CPU time spent (ms)=0 
     Physical memory (bytes) snapshot=0 
     Virtual memory (bytes) snapshot=0 
16/03/25 18:48:34 ERROR streaming.StreamJob: Job not successful! 
Streaming Command Failed!

来源

2016-03-25 Chen Shengkun

的-mapper和-reducer命令行参数的值需要是可执行文件。在不支持shebang行的Windows上，Python脚本不可执行。

尝试指定的-mapper和-reducer参数如下：

-mapper "python mapper.py" -reducer "python reducer.py"

请注意，我没有这个运行自己，所以我不能保证它会成功。它可能会抱怨它找不到名称为python mapper.py的文件。

如果这不起作用，请编写短批处理脚本mapper.bat和reducer.bat，它们只运行相应的Python脚本。例如，mapper.bat将包含以下行：

@echo off 
python mapper.py %*

然后，你-mapper和-reducer参数指定这些批处理脚本。另外，添加命令行参数-file mapper.bat和-file reducer.bat以确保将这些批处理脚本复制到所有计算节点。

来源

2016-03-25 13:33:59

我尝试指定-mapper和-reducer参数为'python mapper.py'和'python reducer.py'，并且工作正常！非常感谢！ –

为什么Windows7中的hadoop引发了RuntimeException：配置对象时出错

回答

相关问题