Delta Kernel vs Delta Standalone

139 views
Skip to first unread message

COCOCAT TACOCOC

unread,
Oct 28, 2023, 5:53:45 PM10/28/23
to Delta Lake Users and Developers
I'm having trouble understanding the difference between Delta Kernel and Delta Standalone. At the end of this video (https://youtu.be/KVUMFv7470I?si=8etJHwQuIk3eo2Ti), it is shown that Delta Standalone will eventually migrate to use Delta kernel. That implies that they serve different purposes.

But then, my understanding of Delta kernel is that it's supposed to make it easier for third parties to develop connectors for their projects, which is what Delta Standalone is supposed to accomplish. Early in the video (https://youtu.be/KVUMFv7470I?si=No9GV4PbNSk4Mmp4), some of the current limitations of using Delta Standalone for this purpose are highlighted, namely that users of Standalone must understand the  details of Delta's protocol. 

So, it then sounds like kernel is intended to have the same purpose as standalone, but kernel is supposed to improve upon standalone. But, if so, what is the purpose of using the kernel within standalone? Is it just to enable those already using Standalone to easily benefit from Kernel while using their existing Standalone implementations?

Relatedly, for new projects that can choose between using the kernel and using standalone, is the intention that they'll use Kernel (if not now, at least once Kernel leaves preview)? If so, is there an approximate timeline for when kernel will leave preview and include write support, thereby achieving parity with Standalone?

Thanks! 


Reply all
Reply to author
Forward
0 new messages