Delta Lake Q1, 2023 Contribution Report

[Carly Akerly]

Check out the Delta Lake Q1, 2023 Contribution Report including community issues, latest releases, community health, and more.

Description: Delta Lake is an open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs for Scala, Java, Rust, Ruby, and Python.   Community: Noritaka Sekiyama, Kyle Duong, and Sandeep Adwankar published Introducing native Delta Lake table support with AWS Glue crawlers on the AWS Big Data blog. Matthew Powers, Chitral Verma published Reading Delta Lake Tables into Polars DataFrames. Mehdi Ouazza published Python Devs, It's Time To Get On The Rust Bandwagon! Simplify Delta Lake Complexity with mack: A great blog post by Daniel Beach on working with mack which provides a variety of helper methods that make it easy for you to perform common Delta Lake operations. Contributions to mack and jodie: Thanks to Robert Kossendey and Souvik Pratiher for their contributions to mack (python) and Brayan Jules for contributions to jodie (scala) Delta Lake helper functions. Incubation of deltarray: The deltarray library provides a Delta Lake table reader for the Ray open-source ML toolkit. Coinbase published their second part to their blog Optimizations in SOON (Spark cOntinuOus iNgestion) - Part 2 which calls out their merge optimizations in Delta Lake. Thanks to efforts from the community and spearheaded by Nick Karpov, delta-spark is now available on conda. Brayan Jules Jacques published HIO: a library that provides elegant functions to manage HDFS filesystem and cloud object stores. Vítor Teixeira published Delta Lake — Automatic Schema Evolution and contributed PR 1645. Vítor also published Delta Lake: Keeping It Fast and Clean on how to improve your Delta tables’ performance. Bryan Cafferky published Understanding Delta File Logs - The Heart of Delta Lake. Latest releases: Delta Rust We released Python-v0.8.1 on March 13, 2023. Please see here. Credit for this release includes Will Jones and gruuya. We released Rust-v0.8.0 on March 11, 2023. Please see here. Credit for this release includes Will Jones. We released Python-v0.8.0 on March 11, 2023. Please see here. Credit for this release includes Will Jones, QP Hou, Tommy Guy, spebern, Ilia Moshkov, Rachel Bushrian, Marijn Valk, Robert Pack. We released Rust-v0.7.0 on February 12, 2023. Please see here. Credit for this release includes Will Jones. We released Python-v0.7.0 on January 25, 2023. Please see here. Credit for this release includes Alexandre Bury, John Batty, Fazzani, Andrei Ionescu, Will Jones, Ilia Moshkov, Marijn Valk, Greg Hayes, Ian Alexander Joiner, Florian Valeye, byteink, Robert Pack, Marko Grujic, damiondoesthings. Delta Spark We released Delta Lake 2.0.2 on January 13, 2023. Please see here. Credit for this release includes Helge Brügner, Jiaheng Tang, Mitchell Riley, Ryan Johnson, Scott Sandre, Venki Korukanti, Jintao Shen, Yann Byron. Delta Sharing We released Delta Sharing 0.6.3 on February 17, 2023. Please see here. Credit for this release includes Abhijit Chakankar, Lin Zhou, Xiaotong Sun. Contributors and Committers/Maintainers: connectors - 18, delta-rs - 72, delta-sharing - 21, delta-spark - 61, website - 25 Community Health: The community continues to have steady growth. In Q1 of 2023, Delta Lake had 1.99K total contributors and 3.37K total commits. There has been a growth of 51% in total contributors and 64% in the total commits over the last year. (source: LFX Insights) This email was sent by: Delta Lake 548 Market St, PMB 57274, San Francisco, CA 94104-5401, United States Subscription Center

--

Carly Akerly

cak...@contractor.linuxfoundation.org