Where are the dataproc images with Spark 3 releases?

61 views

Skip to first unread message

Daniel Solow

unread,

Mar 4, 2021, 9:47:47 AM3/4/21

to Google Cloud Dataproc Discussions

Every 2.0 image contains a Spark 3 release candidate: https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-2.0

Where are the actual spark 3 releases? I don't want to use a release candidate

Thanks.

Mich Talebzadeh

unread,

Mar 5, 2021, 5:07:30 AM3/5/21

to Google Cloud Dataproc Discussions

Hi Daniel,

My suggestion is that we should wait for Dataprocs to provide a Spark image built on a stable version Spark 3.1.1 (now in GA) and released a couple of days ago.

HTH

Daniel Solow

unread,

Mar 5, 2021, 8:20:17 AM3/5/21

to Google Cloud Dataproc Discussions

Spark 3 has been out since June 2020. What possible justification is there for not having a Dataproc image that supports the most recent major version after almost a year?

Reply all

Reply to author

Forward

0 new messages