2016-08-17 33 views
1

尽管我的研究告诉我这是内部到Amazon的错误,但我仍然一直在收到此错误。我不知道从哪里开始出现这个错误,或者是否有什么我可以做的来帮助它。AWS Cloud Formation错误:ElasticMapReduce集群未能稳定

事实上,我始终如一地获得它,这让我觉得这是我的脚本错了。那就是:

{ 
    "Description": "Demo pipeline.", 
    "Resources": { 
    "s3Demo": { 
     "Type" : "AWS::S3::Bucket", 
     "Properties" : { 
      "BucketName" : "example-dna-demo" 
     } 
    }, 

    "s3Access": { 
     "Type": "AWS::IAM::Role", 
     "Properties": { 
      "ManagedPolicyArns": [ 
       "arn:aws:iam::aws:policy/AmazonS3FullAccess" 
      ], 
      "AssumeRolePolicyDocument": { 
       "Version": "2012-10-17", 
       "Statement": [{ 
        "Effect": "Allow", 
        "Action": "sts:AssumeRole", 
        "Principal":{ 
         "Service": "firehose.amazonaws.com" 
        } 
       }] 
      }, 
      "RoleName": "kinesisS3Access" 
     }, 
     "DependsOn": "s3Demo" 
    }, 

    "kinesisDemo": { 
     "Type": "AWS::KinesisFirehose::DeliveryStream", 
     "Properties": { 
      "DeliveryStreamName": "Demo-Stream", 
      "S3DestinationConfiguration": { 
       "BucketARN" : "arn:aws:s3:::example-dna-demo", 
       "BufferingHints" : { 
        "IntervalInSeconds" : 300, 
        "SizeInMBs" : 5 
       }, 
       "CompressionFormat" : "UNCOMPRESSED", 
       "Prefix" : "twitter", 
       "RoleARN" : { "Fn::GetAtt": [ "s3Access", "Arn" ]} 
      } 
     }, 
     "DependsOn": "s3Access" 
    }, 

    "S3LambdaAccess":{ 
     "Type": "AWS::IAM::Role", 
     "Properties": { 
      "ManagedPolicyArns": [ 
       "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess" 
      ], 
      "AssumeRolePolicyDocument": { 
       "Version": "2012-10-17", 
       "Statement": [{ 
        "Effect": "Allow", 
        "Action": "sts:AssumeRole", 
        "Principal":{ 
         "Service": "lambda.amazonaws.com" 
        } 
       }] 
      }, 
      "RoleName": "lambdaS3Access" 
     } 
    }, 
    "LambdaDemo": { 
     "Type" : "AWS::Lambda::Function", 
     "Properties" : { 
      "Code" : { 
       "S3Bucket" : "example-dna-cloud-formation", 
       "S3Key" : "lambda_function.py.zip" 
      }, 
      "Description" : "Looks for S3 writes and loads them into another resource", 
      "FunctionName" : "DemoLambdaFunction", 
      "Handler" : "lambda-handler", 
      "Role" : { "Fn::GetAtt": [ "S3LambdaAccess", "Arn" ]}, 
      "Runtime" : "python2.7" 
     }, 
     "DependsOn": "S3LambdaAccess" 
    }, 
    "EMRClusterJobFlowRole": { 
     "Type": "AWS::IAM::Role", 
     "Properties": { 
      "AssumeRolePolicyDocument": { 
       "Version": "2012-10-17", 
       "Statement": [{ 
        "Effect": "Allow", 
        "Action": "sts:AssumeRole", 
        "Principal":{ 
         "Service": "ec2.amazonaws.com" 
        } 
       }] 
      }, 
      "RoleName": "ClusterRole" 
     } 
    }, 
    "EMRServiceRole": { 
     "Type": "AWS::IAM::Role", 
     "Properties": { 
      "AssumeRolePolicyDocument": { 
       "Version": "2012-10-17", 
       "Statement": [{ 
        "Effect": "Allow", 
        "Action": "sts:AssumeRole", 
        "Principal":{ 
         "Service": "ec2.amazonaws.com" 
        } 
       }] 
      }, 
      "RoleName": "EC2InstanceRole" 
     } 
    }, 
    "EMR":{ 
     "Type" : "AWS::EMR::Cluster", 
     "Properties" : { 
      "Applications": [ 
       { 
        "Name" : "Spark" 
       } 
      ], 
      "ReleaseLabel": "emr-5.0.0", 
      "Instances" : { 
       "CoreInstanceGroup" : { 
        "BidPrice": 0.06, 
        "InstanceCount" : 1, 
        "InstanceType" : "m4.large", 
        "Market": "SPOT" 
        }, 
       "MasterInstanceGroup" : { 
        "BidPrice": 0.06, 
        "InstanceCount" : 1, 
        "InstanceType" : "m4.large", 
        "Market": "SPOT" 
        } 
       }, 
      "JobFlowRole" : "EMRClusterJobFlowRole", 
      "Name" : "DemoEMR", 
      "ServiceRole" : "EMRServiceRole", 
      "LogUri":"s3://toyota-dna-cloud-formation/cf-logging" 
     }, 
     "DependsOn": ["EMRServiceRole", "EMRServiceRole"] 
    } 
    } 
} 

我想,你可能无法运行它,因为我有一个lambda函数从S3斗,我已经改变了这里的名字越来越代码。我只是在学习云形成脚本,而且我知道有很多东西我不在这里做,但我只想构建一个有用的小东西,然后再填充一点。

我知道我的脚本一直工作到两个IAM角色和EMR集群。提前致谢。

编辑:我指定了最近的实例版本并选择了一个ReleaseLabel属性。没有运气。同样的错误。

+0

将实例类型更改为m3.medium并尝试。 “ – helloV

+0

”实例类型'm3.medium'不受支持。“ – bwighthunter

+0

从当前代中指定一个实例并尝试:https://aws.amazon.com/emr/pricing/ 使用以前的gen实例时,我看到了这种稳定性错误。 – helloV

回答

1

这可能是因为您的帐户已达到您尝试部署到的区域的EC2限制。你尝试过不同的地区吗?

+0

神圣莫里。我不能相信那是真的。 DUMB AWS错误。 – bwighthunter

1

所以事实证明,我在运行脚本的地区没有默认的VPC,这就是我的EMR集群未能稳定的原因。

当我试着在不同的区域运行它时,它工作,但是因为该区域DID有一个默认的VPC。

相关问题