Scio 0.9.3 released

1 view
Skip to first unread message

Neville Li

unread,
Aug 5, 2020, 2:46:11 PM8/5/20
to Scio Users, us...@beam.apache.org
Hi all,

We just released Scio 0.9.3. This bumps Beam SDK to 2.23.0 and includes a lot of improvements & bug fixes.

Cheers,
Neville


"Petrificus Totalus"

There are no breaking changes in this release, but some were introduced with v0.9.0:

See v0.9.0 Migration Guide for detailed instructions.

Improvements

  • Allow user-supplied filename prefix for smb writes/reads (#3215)
  • Refactor SortedBucketTransform into a BoundedSource + reuse merge logic (#3097)
  • Add keyGroupFilter optimization to scio-smb (#3160)
  • Add error message to BaseAsyncLookupDoFn preconditions check (#3176)
  • Add Elasticsearch 5,6,7 add/update alias on multiple indices ops (#3134)
  • Add initial update alias op to ES7(#2920)
  • Add ScioContext#applyTransform (#3146)
  • Allow SCollection#transform name override (#3142)
  • Allow setting default name through SCollection#applyTransform (#3144)
  • Update 0.9 migration doc and add Bigquery Type read schema documentation(#3148)

Bug Fixes

  • AvroBucketMetadata should validate keyPath (fix #3038) (#3140)
  • Allow union types in non leaf field for key (#3187)
  • Fix issue with union type as non-leaf field of smb key (#3193)
  • Fix ContextAndArgs#typed overloading issue (#3199)
  • Fix error propagation on Scala Future onSuccess callback (#3178)
  • Fix ByteBuffer should be readOnly (#3220)
  • Fix compiler warnings (#3183)
  • Fix JdbcShardedReadOptions.fetchSize description (#3209)
  • Fix FAQ typo (#3194)
  • Fix scalafix error in SortMergeBucketScioContextSyntax (#3158)
  • Add scalafix ExplicitReturnType and ProcedureSyntax rules (#3179)
  • Cleanup a few more unused and unchecked params (#3223)
  • Use GcpOptions#getWorkerZone instead of deprecated GcpOptions#getZone (#3224)
  • Use raw coder in SCollection#applyKvTransform (#3171)
  • Add raw beam coder materializer (#3164)
  • Avoid circular dep between SCollection and PCollectionWrapper (#3163)
  • Remove unused param of internal partitionFn (#3166)
  • Remove unused CoderRegistry (#3165)
  • Remove defunct scio-bench (#3150)
  • Reuse applyTransform (#3162)
  • Make multijoin.py python3
  • Use TextIO#withCompression (#3145)

Dependency Updates

  • Update Beam SDK to 2.23.0 (#3197)
  • Update dependencies to be inline with 2.23.0 (#3225)
  • Update to scala 2.12.12 (#3157)
  • Update auto-value to 1.7.4 (#3147)
  • Update breeze to 1.1 (#3211)
  • Update cassandra-all to 3.11.7 (#3186)
  • Update cassandra-driver-core to 3.10.0 (#3195)
  • Update commons-lang3 to 3.11 (#3161)
  • Update commons-text to 1.9 (#3185)
  • Update contributing guidelines with current tools (#3149)
  • Update elasticsearch-rest-client, ... to 7.8.1 (#3192)
  • Update elasticsearch, ... to 6.8.11 (#3188)
  • Update jackson-module-scala to 2.10.5 (#3169)
  • Update jna to 5.6.0 (#3156)
  • Update magnolify to 0.2.2 (#3154)
  • Update mysql-connector-java to 8.0.21 (#3153)
  • Update pprint to 0.6.0 (#3203)
  • Update protobuf version to 3.11.4 (#3200)
  • Update sbt-scalafix to 0.9.18 (#3138)
  • Update sbt-sonatype to 3.9.4 (#3136)
  • Update scalafmt-core to 2.6.2 (#3139)
  • Update scalafmt-core to 2.6.3 (#3152)
  • Update scalafmt-core to 2.6.4 (#3167)
  • Update sparkey to 3.1.0 (#3204)
  • Fix conflicting gcsio dependency (#3180)

Robert Bradshaw

unread,
Aug 5, 2020, 8:09:01 PM8/5/20
to user, Scio Users
Thanks for the update!

Marcin Kuthan

unread,
Aug 6, 2020, 3:43:35 AM8/6/20
to us...@beam.apache.org, Scio Users
Just upgraded to Scio 0.9.2 / Beam 2.22 and the stack is not up-to-date again ;)

Thanks for the update (Scio and Beam teams)!
Reply all
Reply to author
Forward
0 new messages