2015-02-23 166 views
1

我想要使用神经节监控Hadoop(Hadoop版本-0.20.2)多节点群集。我Hadoop是工作properly.I已安装神经节阅读以下的博客后---Ganglia监控Hadoop多节点群集

http://hakunamapdata.com/ganglia-configuration-for-a-small-hadoop-cluster-and-some-troubleshooting/

http://hokamblogs.blogspot.in/2013/06/ganglia-overview-and-installation-on.html

我也研究了Ganglia.pdf(附录B 神经节和Hadoop/HBase的)监测。

I have modified only the following lines in **Hadoop-metrics.properties**(same on all Hadoop Nodes)==> 



// Configuration of the "dfs" context for ganglia 
dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext 
dfs.period=10 
dfs.servers=192.168.1.182:8649 

// Configuration of the "mapred" context for ganglia 
mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext 
mapred.period=10 
mapred.servers=192.168.1.182:8649:8649 


// Configuration of the "jvm" context for ganglia 
jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext 
jvm.period=10 
jvm.servers=192.168.1.182:8649 


**gmetad.conf** (Only on Hadoop master Node) 


data_source "Hadoop-slaves" 5 192.168.1.182:8649 
RRAs "RRA:AVERAGE:0.5:1:302400" //Because i want to analyse one week data. 



**gmond.conf** (on all the Hadoop Slave nodes and Hadoop Master) 

globals { 
    daemonize = yes 
    setuid = yes 
    user = ganglia 
    debug_level = 0 
    max_udp_msg_len = 1472 
    mute = no 
    deaf = no 
    allow_extra_data = yes 
    host_dmax = 0 /*secs */ 
    cleanup_threshold = 300 /*secs */ 
    gexec = no 
    send_metadata_interval = 0 
} 

cluster { 
    name = "Hadoop-slaves" 
    owner = "Sandeep Priyank" 
    latlong = "unspecified" 
    url = "unspecified" 
} 

/* The host section describes attributes of the host, like the location */ 
host { 
    location = "CASL" 
} 

/* Feel free to specify as many udp_send_channels as you like. Gmond 
    used to only support having a single channel */ 
udp_send_channel { 
    host = 192.168.1.182 
    port = 8649 
    ttl = 1 
} 
/* You can specify as many udp_recv_channels as you like as well. */ 
udp_recv_channel { 
    port = 8649 

} 

/* You can specify as many tcp_accept_channels as you like to share 
    an xml description of the state of the cluster */ 
tcp_accept_channel { 
    port = 8649 
} 

现在Ganglia只给所有节点的系统指标(mem,disk等)。但它并未在Web界面上显示Hadoop指标(如jvm,地图指标 等)。我该如何解决这个问题?

回答

0

感谢大家,如果你正在使用Hadoop的旧版本然后把以下文件(从Hadoop的新版本)==>

  1. GangliaContext31.java

  2. GangliaContext.java

In path ==> hadoop/src/core/org/apache/hadoop/metrics/ganglia 从Hadoop的新版本开始。

使用ant编译您的Hadoop(并在编译时设置适当的代理)。 如果它发生错误,如函数定义丢失,则将该函数定义(来自新版本)放在适当的java文件中,然后再次编译Hadoop。它会工作。

0

我的工作Hadoop与Ganglia,是的,我在Ganglia看到很多Hadoop指标(容器,地图任务,vmem)。实际上,Hadoop向Ganglio提供了更多的一百个指标。

hokamblogs Post对此已足够。

主节点和内容上我编辑hadoop-metrics2.properties是:

namenode.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31 
namenode.sink.ganglia.period=10 
namenode.sink.ganglia.servers=gmetad_hostname_or_ip:8649 

resourcemanager.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31 
resourcemanager.sink.ganglia.period=10 
resourcemanager.sink.ganglia.servers=gmetad_hostname_or_ip:8649 

,我还可以编辑在奴隶一样的文件:你还记得重启Hadoop的

datanode.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31 
datanode.sink.ganglia.period=10 
datanode.sink.ganglia.servers=gmetad_hostname_or_ip:8649 

nodemanager.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31 
nodemanager.sink.ganglia.period=10 
nodemanager.sink.ganglia.servers=gmetad_hostname_or_ip:8649 

和Ganglia在更改文件后。

我希望这对你有所帮助。