2013-09-22 31 views
1

http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.htmlContainerLaunchContext.setResource()的Hadoop纱的缺失

我尽量让这个例子从上面link.but工作得很好,我不能编译如下

Resource capability = Records.newRecord(Resource.class); 
capability.setMemory(512); 
amContainer.setResource(capability); 

// Set the container launch content into the 
// ApplicationSubmissionContext 
appContext.setAMContainerSpec(amContainer); 

amContainer代码ContainerLaunchContext我的hadoop版本是2.1.0-beta。 我做了一些调查。我发现ContainerLaunchContext中没有方法“setResource”

我对此有3个问题
1)该方法已被删除或什么?
2)如果方法已被删除,我现在该怎么办?
3)有没有关于纱线的任何文档,因为我发现网站上的doc很容易,我希望我可以得到一个手册或其他东西。例如, capability.setMemory(512); 根据代码中的注释我不知道它是512k或512M。

回答

0

您可以通过commend为ApplicationMaster设置可用内存。因此:

// Set the necessary command to execute the application master 
Vector<CharSequence> vargs = new Vector<CharSequence>(30); 
... 
vargs.add("-Xmx" + amMemory + "m"); // notice "m" indicating megabytes, you can use also -Xms combined with -Xmx 
... // transform vargs to String commands 
amContainer.setCommands(commands); 

这应该可以解决您的问题。至于3个问题。纱线是快速发展的软件。我的建议忘记文档,获取源代码并阅读它。这将回答你很多问题。

+0

这不适合我。 – Dyin

+0

'资源能力= Records.newRecord(Resource.class); capability.setMemory(amMemory); appContext.setResource(capability); // ApplicationSubmissionContext' 试试这个 – alien01

+0

'ContainerLaunchContext'没有函数'setResource'。 – Dyin

1

这实际上是对问题的恰当解决方案。先前的回答可能会导致执行不正

@Dyin我可能不适合在注释;)验证为AppMaster 2.2.0和2.3.0

驱动程序设置资源:

ApplicationSubmissionContext appContext = app.getApplicationSubmissionContext(); 
ApplicationId appId = appContext.getApplicationId(); 
appContext.setApplicationName(this.appName); 

// Set up the container launch context for the application master 
ContainerLaunchContext amContainer = Records.newRecord(ContainerLaunchContext.class); 

Resource capability = Records.newRecord(Resource.class); 
capability.setMemory(amMemory);   
appContext.setResource(capability); 

appContext.setAMContainerSpec(amContainer); 

Priority pri = Records.newRecord(Priority.class); 
pri.setPriority(amPriority); 
appContext.setPriority(pri); 

appContext.setQueue(amQueue); 

// Submit the application to the applications manager 
yarnClient.submitApplication(appContext); // this.yarnClient = YarnClient.createYarnClient(); 

在ApplicationMaster你这是怎么应该为容器(工人)指定资源。

private AMRMClient.ContainerRequest setupContainerAskForRM() { 
     // setup requirements for hosts 
     // using * as any host will do for the distributed shell app 
     // set the priority for the request 
     Priority pri = Records.newRecord(Priority.class); 
     pri.setPriority(requestPriority); 

     // Set up resource type requirements 
     // For now, only memory is supported so we set memory requirements 
     Resource capability = Records.newRecord(Resource.class); 
     capability.setMemory(containerMemory); 

     AMRMClient.ContainerRequest request = new AMRMClient.ContainerRequest(capability, null, null, 
       pri); 
     return request; 
    } 

在AppMaster

AMRMClientAsync.CallbackHandler allocListener = new RMCallbackHandler(); 
resourceManager = AMRMClientAsync.createAMRMClientAsync(1000, allocListener); 
resourceManager.init(conf); 
resourceManager.start(); 

for (int i = 0; i < numTotalContainers; ++i) { 
    AMRMClient.ContainerRequest containerAsk = setupContainerAskForRM(); 
    resourceManager.addContainerRequest(containerAsk); // 
} 

启动容器 您可以使用原来的答案溶液(java的CMD)部分的run()或main()方法,但它只是在顶部的樱桃。无论如何,它应该工作。

+0

这帮了我!谢谢!另外,安装2​​.3.0。 – Dyin