[substrait-java] Proposal for handling google.protobuf.Any

[Victor Barua]

Hello!

I’m using Isthmus with a use case that requires access to Advanced Extensions.

Looking at substrait-java, I realized there wasn’t support for extracting google.protobuf.Any values from the protos to the POJOs, so I thought I’d try and flesh out a proposal for handling this.

Proposal (Google Docs because better code handling):

Substrait: Proposal for Handling google.protobuf.Any

Associated PR with an (incomplete) version of what these changes would look like:

https://github.com/substrait-io/substrait-java/pull/133

I’m curious to hear people's thoughts on this, and also to see if anyone else has a different approach that they think would work for this problem.

[Weston Pace]

I'm not working in Java so I haven't used substrait-java much but I glanced through the proposal.  Tag interfaces sound reasonable to me and I don't know if you'll be able to get anything simpler (e.g. I don't think you can avoid the inevitable cast that users will need to do when visiting the deserialized structure later).

Perhaps you could add a utility to do this cast for the user?  E.g. something like

// Returns empty if no extension is present OR if it doesn't match the requested type

public <T extends Extension.Enhancement> Optional<T> getEnhancementAs();

--
You received this message because you are subscribed to the Google Groups "substrait" group.
To unsubscribe from this group and stop receiving emails from it, send an email to substrait+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/substrait/7db88096-9e9f-4e70-8261-38584adbce0cn%40googlegroups.com.

[Victor Barua]

> I don't think you can avoid the inevitable cast that users will need to do when visiting the deserialized structure later.

That was effectively the conclusion I reached as well. I introduced the tag interfaces to help track data provenance (i.e. there are lots of different types of Any, which specific field Any is this Java class coming from).

> Perhaps you could add a utility to do this cast for the user?

I think it makes sense to have this in the long-term, but I'd like to hold off on it for now. This works effectively gives us access to Any data in the POJOs. Once that's in, I'll be able to implement various features internally, and while doing that hopefully figure out good patterns for working with the tagged data.

[Victor Barua]

I've gone ahead and implemented full support for:

• AdvancedExtension messages
• ReadRel.ExtensionTable messages
• ExtensionLeafRel, ExtensionSingleRel and ExtensionMultiRel message

In the following PR
https://github.com/substrait-io/substrait-java/pull/135