Hadoop集群中的节点数量

当我开始我的群，这里是我的控制台响应..

[email protected]:/usr/local/hadoop$ /usr/local/hadoop/sbin/start-dfs.sh 
Starting namenodes on [hadoop-master] 
hadoop-master: starting namenode, logging to /usr/local/hadoop/logs/hadoop-hduser-namenode-hadoop-master.out 
hadoop-master: starting datanode, logging to /usr/local/hadoop/logs/hadoop-hduser-datanode-hadoop-master.out 
hadoop-child: starting datanode, logging to /usr/local/hadoop/logs/hadoop-hduser-datanode-hadoop-child.out 
Starting secondary namenodes [0.0.0.0] 
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-hduser-secondarynamenode-hadoop-master.out 
[email protected]:/usr/local/hadoop$ jps 
21079 NameNode 
21258 DataNode 
21479 SecondaryNameNode 
21600 Jps 
[email protected]:/usr/local/hadoop$ /usr/local/hadoop/sbin/start-yarn.sh 
starting yarn daemons 
starting resourcemanager, logging to /usr/local/hadoop/logs/yarn-hduser-resourcemanager-hadoop-master.out 
hadoop-child: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-hduser-nodemanager-hadoop-child.out 
hadoop-master: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-hduser-nodemanager-hadoop-master.out 
[email protected]:/usr/local/hadoop$ jps 
21079 NameNode 
21258 DataNode 
22117 Jps 
21815 NodeManager 
21479 SecondaryNameNode 
21658 ResourceManager

你可以看到一个数据节点正在启动在Hadoop的子机。

现在，当我试图获取所有的节点信息。我没有看到所有正在显示的节点。

[email protected]:/usr/local/hadoop$ bin/hdfs dfsadmin -report 
Configured Capacity: 21103243264 (19.65 GB) 
Present Capacity: 17825124352 (16.60 GB) 
DFS Remaining: 17821085696 (16.60 GB) 
DFS Used: 4038656 (3.85 MB) 
DFS Used%: 0.02% 
Under replicated blocks: 0 
Blocks with corrupt replicas: 0 
Missing blocks: 0 
Missing blocks (with replication factor 1): 0 

------------------------------------------------- 
Live datanodes (1): 

Name: 127.0.0.1:50010 (localhost) 
Hostname: localhost 
Decommission Status : Normal 
Configured Capacity: 21103243264 (19.65 GB) 
DFS Used: 4038656 (3.85 MB) 
Non DFS Used: 3278118912 (3.05 GB) 
DFS Remaining: 17821085696 (16.60 GB) 
DFS Used%: 0.02% 
DFS Remaining%: 84.45% 
Configured Cache Capacity: 0 (0 B) 
Cache Used: 0 (0 B) 
Cache Remaining: 0 (0 B) 
Cache Used%: 100.00% 
Cache Remaining%: 0.00% 
Xceivers: 1 
Last contact: Sun Feb 26 17:13:04 UTC 2017

我需要在这里查看主人和孩子的信息。我有1个主节点和1个子节点。

********************修复*******

根据弗兰克，这是它是如何固定..

编辑/ etc/config文件，并提供主和子ip地址。

改变了低于2线

127.0.0.1 localhost hadoop-master 
961.118.98.183 hadoop-child

至（在两个节点）

127.0.0.1 localhost 
961.118.99.251 hadoop-master 
961.118.98.183 hadoop-child

2.Restart在以下order..the数据节点应被重新格式化群集。

format will remove only the meta, the datanode's data directories will still be using the old namenode's identity which wil cause the datanode to fail(so delete the directories). 

Can you please this order.. 
1) stop the cluster 
2) rm -rf /path/to/datanode/data/dir (in both nodes) 
3) hadoop namenode -format 
4) START cluste

来源

2017-02-26 user1050619

你的一个datanodes没有运行！ – franklinsijo

当我尝试在子节点中的jps命令..它显示其[email protected]：/ home/ubuntu $ jps 3286 NodeManager 3134 DataNode 3429 Jps – user1050619

核心站点中设置的值是什么。 xml的两个节点中的属性fs.defaultFS？同时，目前节点报告的主机名为'localhost'。 – franklinsijo

当建立一个多节点环境中，namenode和resourcemanager地址必须明确定义。

这个属性添加到两个节点core-site.xml，

<property> 
    <name>fs.defaultFS</name> 
    <value>hdfs://hadoop-master:8020</value> 
</property>

而且这yarn-site.xml，

<property> 
    <name>yarn.resourcemanager.hostname</name> 
    <value>hadoop-master</value> 
</property>

确保在/etc/hosts文件中的所有节点，IP地址和主机名映射完成。

注意：如果服务正在运行，请停止它们并在再次启动它们之前添加这些属性。

来源

2017-02-26 18:08:32 franklinsijo

谢谢..当我运行dfsadmin报告时，仍然得到相同的报告...只报告一个节点。另外，前面提到的主机名仍然是本地主机 – user1050619

并且报告的datanode具有名称localhost？ – franklinsijo

那是正确的..它与我上面显示的完全一样的报告。 – user1050619

Hadoop集群中的节点数量

回答

相关问题