Announcing support for autoscaling!

[Karthik Palaniappan]

Hey everyone,

As announced at Google Cloud Next 2018, we have released support for Cloud Dataproc Cluster Autoscaling! Here is a link to the documentation https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/autoscaling.

This is an alpha release, so we are actively soliciting feedback. Please join cloud-dataproc-discuss and post your questions, issues, and feature requests. For private issues, please continue to reach out through regular support channels.

Best,

The Cloud Dataproc Team

[evgeniy...@gmail.com]

Hi, Team

As far as i understood using autoscaling I can only enable "graceful_decommission_timeout" and it's enough, don't you?Because, when i created cluster with autoscaling properties I didn't see in cluster settings on UI that it's enabled. I tested autoscaling and looks like it works for me, but I have some little doubts.

Thank you, Evgeniy

[mal...@google.com]

Hi Evgeniy,

The graceful_decommission_timeout enabled in autoscaling properties only applies to cluster scaling operations driven by autoscaling and will not appear in the UI. To verify that the timeout was set properly, run 'gcloud dataproc clusters describe <CLUSTER_NAME>', and the autoscaling properties should be visible under config.softwareConfig.properties.

In the UI, it is also possible to set graceful decommission timeout for a single cluster scaling operation (when editing the cluster configuration).

Hope this helps!