2015-06-11 107 views
1

我目前正在尝试在Amazon EC2实例(多节点群集)上运行Hadoop 2.6.0。我启动了两个Ubuntu 14.04实例。其中一人是主人,另一人是奴隶。下面是我的配置:Hadoop NameNode无法启动

-core-site.xml中

<configuration> 
     <property> 
       <name>fs.defaultFS</name> 
       <value>hdfs://hadoopmaster:9000</value> 
     </property> 
</configuration> 

-hdfs-site.xml中

<configuration> 
     <property> 
       <name>dfs.replication</name> 
       <value>1</value> 
     </property> 
     <property> 
       <name>dfs.namenode.name.dir</name> 
       <value>file:/home/ubuntu/hadoop-2.6.0/hadoop_data/hdfs/namenode</value> 
     </property> 
</configuration> 

-yarn-site.xml中

<property> 
     <name>yarn.nodemanager.aux-services</name> 
     <value>mapreduce_shuffle</value> 
</property> 
<property> 
     <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> 
     <value>org.apache.hadoop.mapred.ShuffleHandler</value> 
</property> 
<property> 
     <name>yarn.resourcemanager.resource-tracker.address</name> 
     <value>hadoopmaster:8025</value> 
</property> 
<property> 
     <name>yarn.resourcemanager.scheduler.address</name> 
     <value>hadoopmaster:8030</value> 
</property> 
<property> 
     <name>yarn.resourcemanager.address</name> 
     <value>hadoopmaster:8050</value> 
</property> 

-mapred-site.xml中

<configuration> 
     <property> 
       <name>mapred.job.tracker</name> 
       <value>hadoopmaster:54311</value> 
     </property> 
</configuration> 

-masters

hadoopmaster 

-slaves

hadoopslave1 

从属

-hdfs-site.xml中

<configuration> 
     <property> 
       <name>dfs.replication</name> 
       <value>1</value> 
     </property> 
     <property> 
       <name>dfs.datanode.data.dir</name> 
       <value>file:/home/ubuntu/hadoop-2.6.0/hadoop_data/hdfs/datanode</value> 
     </property> 
</configuration> 

其他人和主人一样。

当我运行hdfs namenode -format它看起来很好,并与状态0退出当我运行start-all.sh它提供了以下的输出:

[email protected]:~/hadoop-2.6.0$ sbin/start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh 
Starting namenodes on [hadoopmaster] 
hadoopmaster: starting namenode, logging to /home/ubuntu/hadoop-2.6.0/logs/hadoop-ubuntu-namenode-hadoopmaster.out 
hadoopslave1: starting datanode, logging to /home/ubuntu/hadoop-2.6.0/logs/hadoop-ubuntu-datanode-hadoopslave1.out 
Starting secondary namenodes [0.0.0.0] 
0.0.0.0: starting secondarynamenode, logging to /home/ubuntu/hadoop-2.6.0/logs/hadoop-ubuntu-secondarynamenode-hadoopmaster.out 
starting yarn daemons 
starting resourcemanager, logging to /home/ubuntu/hadoop-2.6.0/logs/yarn-ubuntu-resourcemanager-hadoopmaster.out 
hadoopslave1: starting nodemanager, logging to /home/ubuntu/hadoop-2.6.0/logs/yarn-ubuntu-nodemanager-hadoopslave1.out 

这听起来不错,没有错误的报道。然而,当我运行jps主站点上,它提供了以下的输出:

[email protected]:~/hadoop-2.6.0$ jps 
3640 ResourceManager 
3501 SecondaryNameNode 
3701 Jps 

的NameNode会丢失!当我从网站上运行jps,我得到的NameNode以下

[email protected]:~/hadoop-2.6.0$ jps 
1686 DataNode 
1870 Jps 
1817 NodeManager 

以下是日志文件:

2015-06-11 04:16:18,987 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: 
/************************************************************ 
STARTUP_MSG: Starting NameNode 
STARTUP_MSG: host = hadoopmaster/54.172.40.127 
STARTUP_MSG: args = [] 
STARTUP_MSG: version = 2.6.0 

.out文件:

ulimit -a for user ubuntu 
core file size   (blocks, -c) 0 
data seg size   (kbytes, -d) unlimited 
scheduling priority    (-e) 0 
file size    (blocks, -f) unlimited 
pending signals     (-i) 13357 
max locked memory  (kbytes, -l) 64 
max memory size   (kbytes, -m) unlimited 
open files      (-n) 1024 
pipe size   (512 bytes, -p) 8 
POSIX message queues  (bytes, -q) 819200 
real-time priority    (-r) 0 
stack size    (kbytes, -s) 8192 
cpu time    (seconds, -t) unlimited 
max user processes    (-u) 13357 
virtual memory   (kbytes, -v) unlimited 
file locks      (-x) unlimited 

我已经重复了这个几次,我得到了同样的结果。 NameNode总是缺失。任何人都可以给我一些关于这个问题的建议吗?非常感谢!

回答

0

你检查该文件夹的权限的路径下 文件:/home/ubuntu/hadoop-2.6.0/hadoop_data/hdfs/

+0

是的,它应该可以读/写这个文件夹,否则'hdfs namenode -format'会失败。 – user3019792

1

我猜你已经想通了解决方案,但这个是任何人都会遇到同样的问题(像我一样)。

首先使用stop-yarn.shstop-dfs.sh(按此顺序)将您的hadoop群集关闭。现在,您只需转到hadoop临时目录即可。如果你没有配置你的用户,那么它将在/usr/local/hadoop/tmp/

其他从的core-site.xml找到它。 然后只需键入:

室射频*

现在启动群集起来,中提琴,的Namenode启动。