Scio 0.9.3 released

[Neville Li]

Hi all,

We just released Scio 0.9.3. This bumps Beam SDK to 2.23.0 and includes a lot of improvements & bug fixes.

Cheers,

Neville

https://github.com/spotify/scio/releases/tag/v0.9.3

"Petrificus Totalus"

There are no breaking changes in this release, but some were introduced with v0.9.0:

See v0.9.0 Migration Guide for detailed instructions.

Improvements

• Allow user-supplied filename prefix for smb writes/reads (#3215)
• Refactor SortedBucketTransform into a BoundedSource + reuse merge logic (#3097)
• Add keyGroupFilter optimization to scio-smb (#3160)
• Add error message to BaseAsyncLookupDoFn preconditions check (#3176)
• Add Elasticsearch 5,6,7 add/update alias on multiple indices ops (#3134)
• Add initial update alias op to ES7(#2920)
• Add ScioContext#applyTransform (#3146)
• Allow SCollection#transform name override (#3142)
• Allow setting default name through SCollection#applyTransform (#3144)
• Update 0.9 migration doc and add Bigquery Type read schema documentation(#3148)

Bug Fixes

• AvroBucketMetadata should validate keyPath (fix #3038) (#3140)
• Allow union types in non leaf field for key (#3187)
• Fix issue with union type as non-leaf field of smb key (#3193)
• Fix ContextAndArgs#typed overloading issue (#3199)
• Fix error propagation on Scala Future onSuccess callback (#3178)
• Fix ByteBuffer should be readOnly (#3220)
• Fix compiler warnings (#3183)
• Fix JdbcShardedReadOptions.fetchSize description (#3209)
• Fix FAQ typo (#3194)
• Fix scalafix error in SortMergeBucketScioContextSyntax (#3158)
• Add scalafix ExplicitReturnType and ProcedureSyntax rules (#3179)
• Cleanup a few more unused and unchecked params (#3223)
• Use GcpOptions#getWorkerZone instead of deprecated GcpOptions#getZone (#3224)
• Use raw coder in SCollection#applyKvTransform (#3171)
• Add raw beam coder materializer (#3164)
• Avoid circular dep between SCollection and PCollectionWrapper (#3163)
• Remove unused param of internal partitionFn (#3166)
• Remove unused CoderRegistry (#3165)
• Remove defunct scio-bench (#3150)
• Reuse applyTransform (#3162)
• Make multijoin.py python3
• Use TextIO#withCompression (#3145)

Dependency Updates

• Update Beam SDK to 2.23.0 (#3197)
• Update dependencies to be inline with 2.23.0 (#3225)
• Update to scala 2.12.12 (#3157)
• Update auto-value to 1.7.4 (#3147)
• Update breeze to 1.1 (#3211)
• Update cassandra-all to 3.11.7 (#3186)
• Update cassandra-driver-core to 3.10.0 (#3195)
• Update commons-lang3 to 3.11 (#3161)
• Update commons-text to 1.9 (#3185)
• Update contributing guidelines with current tools (#3149)
• Update elasticsearch-rest-client, ... to 7.8.1 (#3192)
• Update elasticsearch, ... to 6.8.11 (#3188)
• Update jackson-module-scala to 2.10.5 (#3169)
• Update jna to 5.6.0 (#3156)
• Update magnolify to 0.2.2 (#3154)
• Update mysql-connector-java to 8.0.21 (#3153)
• Update pprint to 0.6.0 (#3203)
• Update protobuf version to 3.11.4 (#3200)
• Update sbt-scalafix to 0.9.18 (#3138)
• Update sbt-sonatype to 3.9.4 (#3136)
• Update scalafmt-core to 2.6.2 (#3139)
• Update scalafmt-core to 2.6.3 (#3152)
• Update scalafmt-core to 2.6.4 (#3167)
• Update sparkey to 3.1.0 (#3204)
• Fix conflicting gcsio dependency (#3180)

[Robert Bradshaw]

Thanks for the update!

[Marcin Kuthan]

Just upgraded to Scio 0.9.2 / Beam 2.22 and the stack is not up-to-date again ;)

Thanks for the update (Scio and Beam teams)!