Delta Sharing 0.1.0 released!

45 views
Skip to first unread message

Shixiong(Ryan) Zhu

unread,
May 26, 2021, 12:26:00 PM5/26/21
to Delta Lake Users and Developers

We are excited to announce the release of Delta Sharing 0.1.0.

Delta Sharing is an open protocol for secure real-time exchange of large datasets, which enables organizations to share data in real time regardless of which computing platforms they use. It is a simple REST protocol that securely shares access to part of a cloud dataset and leverages modern cloud storage systems, such as S3, ADLS, or GCS, to reliably transfer data.

With Delta Sharing, a user accessing shared data can directly connect to it through pandas, Tableau, Apache Spark, Rust, Python, or dozens of other systems that support the open protocol, without having to deploy a specific compute platform first. This makes life simpler for both data providers and consumers. Data providers can share a dataset once to reach a broad range of consumers on any platform, and data consumers can get started using the data in minutes on their existing computing tools.

This repo includes the following components:

  • Delta Sharing protocol specification.
  • Python Connector: A Python library that implements the Delta Sharing Protocol to read shared tables as pandas DataFrame or Apache Spark DataFrames.
  • Apache Spark Connector: An Apache Spark connector that implements the Delta Sharing Protocol to read shared tables from a Delta Sharing Server. The tables can then be accessed in SQL, Python, Java, Scala, or R.
  • Delta Sharing Server: A reference implementation server for the Delta Sharing Protocol for development purposes. Users can deploy this server to share existing tables in Delta Lake and Apache Parquet format on modern cloud storage systems.

See the documentation for more details.

Best Regards,

Ryan

Jacek Laskowski

unread,
May 27, 2021, 4:25:18 AM5/27/21
to Shixiong(Ryan) Zhu, Delta Lake Users and Developers
Hi Ryan,

Thanks for another great tool!

Great to see Matei in the dev team! Having had a look at the [1] leaves no doubt he's really into it! ;-)


--
You received this message because you are subscribed to the Google Groups "Delta Lake Users and Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to delta-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/delta-users/CAGeYWGLFQBs1uP3P2d19pgYTKJExzy8CKSfJ9Ys547FqdTuzoA%40mail.gmail.com.

Shixiong(Ryan) Zhu

unread,
Jun 1, 2021, 7:44:41 PM6/1/21
to Jacek Laskowski, Delta Lake Users and Developers
Hey Jacek,

Thanks! Are you going to write a new "The Internals Of” online book? :P

Best Regards,

Ryan

Jacek Laskowski

unread,
Jun 2, 2021, 9:22:44 AM6/2/21
to Shixiong(Ryan) Zhu, Delta Lake Users and Developers
Hehe...not really this week but am not so sure about the weeks to come! :P
Reply all
Reply to author
Forward
0 new messages