Delta Standalone, Delta-rs data skipping support

25 views
Skip to first unread message

COCOCAT TACOCOC

unread,
Oct 24, 2023, 7:32:13 PM10/24/23
to Delta Lake Users and Developers
Hello,

I'm new to Delta. I saw that Delta-rs supports ZOrdering, but I'm unsure if queries made using Delta-rs perform any data skipping with their queries. I have a similar question about Delta Standalone. Standalone does not provide a way to organize table data using z-ordering, but when it reads data that has been been organized with z-ordering, will it utilize resulting opportunities for data skipping?

Sorry if I missed this in the docs of either project. I couldn't find anything about data skipping in connection to either one.

Thanks! 

Matthew Powers

unread,
Oct 24, 2023, 8:16:47 PM10/24/23
to COCOCAT TACOCOC, Delta Lake Users and Developers
delta-rs performs file skipping, see this presentation for benchmarks to see how much Z Ordering can reduce query times with delta-rs: https://www.youtube.com/watch?v=A8bvJlG6phk

Thanks for the question!

--
You received this message because you are subscribed to the Google Groups "Delta Lake Users and Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to delta-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/delta-users/20a0ba15-cefe-43e9-848e-012a23b60c82n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages