GPU support on preemtible workers VMs on dataproc

[Jakub Stransky]

Hello,

Are the GPUs supported on preemptible workers VMs? [This documentation](https://cloud.google.com/dataproc/docs/concepts/compute/gpus#before_you_begin) claims `GPUs cannot be attached to preemptible virtual machines in Dataproc clusters.`

but [latter section](https://cloud.google.com/dataproc/docs/concepts/compute/gpus#attaching_gpus_to_clusters_2) mentions: 

```

Attach GPUs to the master and primary and preemptible worker nodes in a Dataproc cluster when creating the cluster using the ‑‑master-accelerator, ‑‑worker-accelerator, and ‑‑preemptible-accelerator flags. 

```

And [gcloud dataproc SDK](https://cloud.google.com/sdk/gcloud/reference/dataproc/clusters/create#--preemptible-worker-accelerator) mentions that. 

It is super confusing, could someone clarify, please? 

[karth...@google.com]

You can attach GPUs to preemptible workers! The note at the top of that document is just out of date, and I'll just go ahead and remove it.

Thanks for reporting this!

Karthik

[karth...@google.com]

Actually it turns out the flag isn't wired correctly in the backend. If you use the --preemptible-worker-accelerator flag, it doesn't attach GPUs to the secondary workers. The flag in gcloud was prematurely made public before the feature was ready in the backend.

We'll try to have a fix out in the next few weeks -- you can watch the release notes to see when it rolls out: https://cloud.google.com/dataproc/docs/release-notes.

So for now, GPUs are not supported on preemptible workers. You can still use GPUs on primary workers. Sorry for the confusion.