我的EMR群集在哪里

我想在java上创建一个EMR群集，但是我既不能在EMR群集列表中找到它，也不能在EC2上看到请求的实例。我的EMR群集在哪里

EMR角色确实存在：

[email protected]:~$ aws iam list-roles | grep EMR 
      "RoleName": "EMR_DefaultRole", 
      "Arn": "arn:aws:iam::removed:role/EMR_DefaultRole" 
      "RoleName": "EMR_EC2_DefaultRole", 
      "Arn": "arn:aws:iam::removed:role/EMR_EC2_DefaultRole"

，现在我的java代码：

AWSCredentials awsCredentials = new BasicAWSCredentials(awsKey, awsKeySecret); 
AmazonElasticMapReduce emr = new AmazonElasticMapReduceClient(awsCredentials); 

StepFactory stepFactory = new StepFactory(); 
     StepConfig enabledebugging = new StepConfig() 
       .withName("Enable debugging") 
       .withActionOnFailure("TERMINATE_JOB_FLOW") 
       .withHadoopJarStep(stepFactory.newEnableDebuggingStep()); 

    HadoopJarStepConfig hadoopConfig1 = new HadoopJarStepConfig() 
    .withJar("s3://foo.bucket/hadoop_jar/2015-01-12/foo.jar") 
    .withMainClass("com.strackoverflow.DriverFoo") // optional main class, this can be omitted if jar above has a manifest 
    .withArgs("--input=s3://foo.bucket/logs/,s3://foo.bucket/morelogs/", "--output=s3://foo.bucket/myEMROutput" , "--inputType=text"); // i have custom java code to handle the --input, --output and --inputType parameters 


    StepConfig customStep = new StepConfig("Step1", hadoopConfig1); 

    Collection <StepConfig> steps = new ArrayList<StepConfig>(); 
    { 
     steps.add(enabledebugging); 
     steps.add(customStep); 
    } 


    JobFlowInstancesConfig instancesConfig = new JobFlowInstancesConfig() 
    .withEc2KeyName("fookey") //not fookey.pem 
    .withInstanceCount(2) 
    .withKeepJobFlowAliveWhenNoSteps(false) // on aws example is set to true 
    .withMasterInstanceType("m1.medium") 
    .withSlaveInstanceType("m1.medium"); 


    RunJobFlowRequest request = new RunJobFlowRequest() 
    .withName("java programatic request") 
    .withAmiVersion("3.3.1") 
    .withSteps(steps) // on the amazon example is lunched debug and hive, here is debug and a jar 
    .withLogUri("s3://devel.rui/emr_clusters/pr01/") 
    .withInstances(instancesConfig) 
    .withVisibleToAllUsers(true); 

     RunJobFlowResult result = emr.runJobFlow(request); 

     System.out.println("toString "+ result.toString()); 
     System.out.println("getJobFlowId "+ result.getJobFlowId()); 
     System.out.println("hashCode "+ result.hashCode());

哪里是我的集群？我无法在群集列表中看到它，输出文件夹未创建，日志文件夹保持为空，并且EC2上没有任何实例可见。

由程序输出该

toString {JobFlowId: j-2xxxxxxU} 
getJobFlowId j-2xxxxxU 
hashCode -1xxxxx4

我只好按照指令从这里创建群集 http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/calling-emr-with-java-sdk.html

而这款以创建Java工作 http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-common-programming-sample.html

来源

2015-02-10 SQL.injection

在亚马逊的例子，该区域未配置。

配置区域后，集群正常启动。

AmazonElasticMapReduce emr = new AmazonElasticMapReduceClient(awsCredentials); 
    emr.setRegion(Region.getRegion(Regions.EU_WEST_1));

来源

2015-02-11 11:48:00

我的EMR群集在哪里

回答

相关问题