Error creating jobflow for EMR when trying to set roles

[Eran Eidinger]

I have the two roles EMR_DefaultRole and EMR_EC2_DefaultRole defined, and when I create a cluster that uses them through the AWS console, it works fine.

When using boto to launch a cluster, my cluster fails, as I try to set the IAM roles for the cluster, and for the EC2 instances:

My code:

    emr_jobid = emr_conn.run_jobflow(name='MyJob',

                                     log_uri = 's3://mylogs/',

                                     steps = steps,

                                     ami_version = HADOOP_AMI_VERSION,

                                     enable_debugging = True,

                                     ec2_keyname = 'MyKey',

                                     bootstrap_actions = [bootstrap_step],

                                     action_on_failure = 'TERMINATE_JOB_FLOW',

                                     instance_groups = instance_groups,

                                     job_flow_role = 'EMR_DefaultRole',

                                     service_role = 'EMR_EC2_DefaultRole',

                                     keep_alive = True

                                     )

I get the following errors:

* For the job_flow_role:

   Reason: Failed to provision ec2 instances because 'Value (EMR_DefaultRole) for parameter iamInstanceProfile.name is invalid. Invalid IAM Instance Profile name (Service: AmazonEC2;'

* For the service_role:

   Reason: EMR service role arn:aws:iam::123456789012:role/EMR_EC2_DefaultRole is invalid

the number 123456789012 is, of course, obfuscated.

What am I doing wrong?

BTW, if I leave those two as Null, I instead have a problem where the cluster launches, but cannot access the map and reduce scripts, stored on s3 (probably permissions..., but I don't know what the default roles it runs with)

[Adrian Klaver]

Well according to this:

http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-iam-roles-launch-jobflow.html

/elastic-mapreduce --create --alive --num-instances 3 \
--instance-type m1.large \
--name "myJobFlowName" \
--hive-interactive --hive-versions 0.8.1.6 \
--ami-version 2.3.0 \
--jobflow-role EMR_EC2_DefaultRole \
--service-role EMR_DefaultRole

you have them backwards, so:

job_flow_role = 'EMR_EC2_DefaultRole',
service_role = 'EMR_DefaultRole',

> BTW, if I leave those two as Null, I instead have a problem where the
> cluster launches, but cannot access the map and reduce scripts, stored
> on s3 (probably permissions..., but I don't know what the default roles
> it runs with)
>

> --


--
Adrian Klaver
adrian...@aklaver.com

[Eran Eidinger]

Awesome. thanks!

--
You received this message because you are subscribed to a topic in the Google Groups "boto-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/boto-users/hvlcGK40YM4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to boto-users+unsubscribe@googlegroups.com.
To post to this group, send email to boto-...@googlegroups.com.
Visit this group at http://groups.google.com/group/boto-users.
For more options, visit https://groups.google.com/d/optout.