substrait-java JoinRel conversion

[chaojun zhang]

Hi , I have question about the JoinRel.   I found that substrait-java does not use the postJoinFilter attribute,  but writes the join condition to the expression attribute,  at the same time, I did not see that the join condition was written into the joinExpression

Is that a bug or do I have to follow this conversion?

For example I have sql statement below:

select
    p.*
from
   "part" p
inner join
   "lineitem" l
on p.p_partkey = l.l_partkey
where
     p.p_partkey = 2;

and got the  substrait json as below:

[chaojun zhang]

attach the substrait json file converted from sql statment "select

p.*
from
"part" p
inner join
"lineitem" l
on p.p_partkey = l.l_partkey
where
p.p_partkey = 2;"

[Jeroen van Straten]

I'm not sure what the problem is here. I didn't trace the field indices back to the schema, but the join condition is written to the join expression correctly, and the filter condition is written to a FilterRel. It's true that it could also have written that filter condition to the post-join filter; that's functionally the same thing, it's just maybe not as optimal. AFAIK, substrait-java aka Isthmus doesn't do any optimizations whatsoever (not yet anyway), so I'm not surprised by this.

--
You received this message because you are subscribed to the Google Groups "substrait" group.
To unsubscribe from this group and stop receiving emails from it, send an email to substrait+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/substrait/524cc8ae-647b-4d34-bc65-e3ecc1896dc3n%40googlegroups.com.

[chaojun zhang]

 Reading this document , There is no true distinction between logical and physical operations in Substrait.

 So the question is come to mind,  how can  I distinguish Hash/Nested Loop/Merge Join physical plan with Substrait JoinRel? 

[Jeroen van Straten]

That whole physical relations section is more of a draft than anything else; I believe it still needs a lot of work that simply no one has picked up yet. The different physical join types in particular simply don't have corresponding protobuf messages defined yet, so right now the answer to your question is "you can't," except via advanced extensions, I suppose. It still needs to be defined and added.

I believe the intention of "there is no true distinction between logical and physical operations in Substrait" is just to say that you can mix the logical and (hypothetical) physical relation types, and that some relations use the same relation type in both domains. A filter relation for example wouldn't look any different, except probably that it'd be more restricted in the physical domain. But I'm not 100% sure on that, honestly. I also find it a confusing statement.

To view this discussion on the web visit https://groups.google.com/d/msgid/substrait/94b1f058-f5ed-491a-bf12-d6c35fc7c7c2n%40googlegroups.com.

[Jacques Nadeau]

No one has spent the time creating protobufs for some of the physical relations that are defined in the spec. See here [1] for how they are specified. If you need one or more, please propose protobuf objects that match the specification.

[1] https://substrait.io/relations/physical_relations/

To view this discussion on the web visit https://groups.google.com/d/msgid/substrait/94b1f058-f5ed-491a-bf12-d6c35fc7c7c2n%40googlegroups.com.