Need to create dataproc cluster and get sqladmin scope added by default.

[Daryl Erwin]

When I create a VM it uses the default computer service account. The vm does not get created with the proper sqladmin scope that I need for sql-proxy.

I can get it to work via command line and passing --scop sql-admin *but* that is not an option when dataproc runs with Apache Airflow. It simply pulls in the defaults assigned to the service account.

My vm gets this assignment ..

serviceAccounts:
- email: 484XXXXX...@developer.gserviceaccount.com
  scopes:
  - https://www.googleapis.com/auth/devstorage.read_only
  - https://www.googleapis.com/auth/logging.write
  - https://www.googleapis.com/auth/monitoring.write
  - https://www.googleapis.com/auth/servicecontrol
  - https://www.googleapis.com/auth/service.management.readonly
  - https://www.googleapis.com/auth/trace.append

How do I make auth/sqladmin appear in that list??

I have tried a bunch of different things - like assigning the role via IAM .. it shows the serviceaccount having the perms but they dont get passed along to the new vm.

Any pointers out there?

Daryl.

[Dennis Huo]

Thanks for bringing this up, looks like we haven't yet plumbed through the serviceAccountScopes field to the Airflow DataprocClusterCreateOperator - the field would need to be added as cluster_data['config']['gcsClusterConfig']['serviceAccountScopes'] similarly to where things like metadata are added.

I went ahead and filed https://issues.apache.org/jira/browse/AIRFLOW-1535 to track this minor feature; in the meantime you could try extending the operator yourself to add the field you need.

In general, IAM permissions/roles are necessary but not sufficient by themselves for enabling access by service-accounts from VMs; the "scopes" are still necessary even in the context of a given service account to specify the subset of service scopes it can access in the context of a particular Dataproc cluster. If "scope" constraints aren't needed, you'd typically just use something like the "cloud-platform" scope, and then service-account access should be precisely equivalent to its raw IAM permissions.

[Daryl Erwin]

Thanks .. that is exactly what I did .. hard coded in the value I needed.  Works like a charm. I am not a python whiz so I just took the simple approach.