IQueryable<IAbstract> and StreamedSequenceInfo

[Chris Eldredge]

I'm working on a project where I start with an IQueryable of a
concrete type, then pass that off to some other code that uses an
IQueryable of an interface that my concrete type implements. Here's a
sample test case to make this more clear:

https://github.com/chriseldredge/Lucene.Net.Linq/blob/test-case/re-linq/StreamedSequenceInfo/Lucene.Net.Linq.Tests/Integration/AbstractQueryableTests.cs

The test throws an exception with this message:

'System.Linq.IQueryable`1[Lucene.Net.Linq.Tests.Integration.AbstractQueryableTests
+IAbstract]' cannot be used as
the data type for a sequence with an ItemExpression of type
'Lucene.Net.Linq.Tests.Integration.AbstractQueryableTests+Concrete'

Is this a limitation of re-linq or am I doing something wrong? The
exception only seems to be triggered by ResultOperators that need to
adjust the data type on the StreamedSequenceInfo. If the Take() is
removed, the query succeeds.

Thanks for your help,

Chris

[Michael Ketting]

Hi Chris!

 

Will have to defer to Fabian for the definitive answer.

For now: there are some requirements about type consistency in re-linq or expression trees in general that you might have come across here.

e.g. this one here: https://groups.google.com/d/msg/re-motion-users/tmCDQyRHUnk/rhAJDXSj--0J

 

Anyhow, from the URL and a very quick peek at your github page, are you working on creating an/the official LiNQ frontend for Lucene.NET or is it more of a POC? Have been playing around with Lucene.NET a bit in the past and been thinking about such a match based on re-linq for a long while now. Just never really had the time to do something about it and that would be really cool. Both for Lucene.NET and for re-linq :)

 

Regards, Michael

[Chris Eldredge]

My project is somewhere beyond POC but nowhere close to feature
complete. I started it because I wanted to speed up NuGet.Server (the
part of nuget that lets you host a private internal feed). Nuget uses
OData under the covers, so I needed a pretty rich LINQ provider. I
couldn't find one that fit the bill so I started writing my own. The
library is in the official gallery on nuget.org, but it isn't an
official project as far as Apache is concerned. Actually I haven't
even reached out to that community for feedback.

Anyway, please kick the tires and contribute new features if you have
the inclination. The nunit tests in the Integration namespace probably
do the best job of documenting what I've implemented so far.

Chris

On May 31, 4:22 pm, Michael Ketting <michael.kett...@rubicon.eu>
wrote:

> Hi Chris!
>
> Will have to defer to Fabian for the definitive answer.
> For now: there are some requirements about type consistency in re-linq or
> expression trees in general that you might have come across here.
> e.g. this one here:https://groups.google.com/d/msg/re-motion-users/tmCDQyRHUnk/rhAJDXSj--0J
>
> Anyhow, from the URL and a very quick peek at your github page, are you
> working on creating an/the official LiNQ frontend for Lucene.NET or is it
> more of a POC? Have been playing around with Lucene.NET a bit in the past
> and been thinking about such a match based on re-linq for a long while now.
> Just never really had the time to do something about it and that would be
> really cool. Both for Lucene.NET and for re-linq :)
>
> Regards, Michael
>
>
>
>
>
>
>
> On Thursday, May 31, 2012 8:14:29 PM UTC+2, Chris Eldredge wrote:
> > I'm working on a project where I start with an IQueryable of a
> > concrete type, then pass that off to some other code that uses an
> > IQueryable of an interface that my concrete type implements. Here's a
> > sample test case to make this more clear:
>

> >https://github.com/chriseldredge/Lucene.Net.Linq/blob/test-case/re-li...

[Fabian Schmied]

> For now: there are some requirements about type consistency in re-linq or
> expression trees in general that you might have come across here.
> e.g. this one here:
> https://groups.google.com/d/msg/re-motion-users/tmCDQyRHUnk/rhAJDXSj--0J

Michael is correct: you're running into a type check perfomed by
re-linq in order to ensure that the expected query result type (which
is IQueryable<IAbstract>) matches the items returned by its Select
expression and result operators (which are of type Concrete). Since
typeof (Concrete) != typeof (IAbstract), you're getting this
exception.

We already have a feature request for allowing the items returned by
the select clause to be more specific than the actual enumerable
result type: "https://www.re-motion.org/jira/browse/RM-4482". I've
sketched a possible implementation in the last comment to the linked
issue. If you can take the time to implement it (including unit tests
and one or two integration tests), I'd gladly accept a patch :)

For now, you can try one of the following workarounds:

- Instead of implicitly converting the IQueryable<Concrete> to
IQueryable<IAbstract>, append ".Select (c => (IAbstract) c)" or
".Cast<IAbstract>()", if your LINQ provider supports any of these.
(Careful: Cast will probably lead to subqueries being created. If you
don't have good subquery support, use the Select approach instead.)
This will cause re-linq to see matching result item and sequence
types.

- If you really need to support implicit covariant IQueryable<T>
conversions, you can derive from DefaultQueryProvider and override
GenerateQueryModel. Check if the item type of the
QueryModel.ResultTypeOverride (if non-null) doesn't match the actual
item type ("ResultOperators.Aggregate ((IStreamedDataInfo)
SelectClause.GetOutputDataInfo (), (current, resultOperator) =>
resultOperator.GetOutputDataInfo (current))" - see source code of
QueryModel.GetOutputDataInfo). If so, append a CastResultOperator to
the QueryModel. Also check the QueryModel for SubQueryExpressions and
(recursively) perform the same modification on sub-QueryModel's if
necessary.

Cheers,
Fabian

> --
> You received this message because you are subscribed to the Google Groups
> "re-motion Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/re-motion-users/-/LdBxy2gGhFQJ.
>
> To post to this group, send email to re-moti...@googlegroups.com.
> To unsubscribe from this group, send email to
> re-motion-use...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/re-motion-users?hl=en.

[Chris Eldredge]

Wow, thanks again for the detailed response. I've confirmed that
".Select (c => (IAbstract) c)" side steps the issue but my provider
doesn't know what to do with the ".Cast<IAbstract>()" subquery.

The work to implement RM-4482 looks straightforward but a little
daunting.
If I find some time I'll take a crack at it.

Chris

> >>https://github.com/chriseldredge/Lucene.Net.Linq/blob/test-case/re-li...

[Chris Eldredge]

This didn't turn out to be that difficult to implement. I've attached
a patch
on https://www.re-motion.org/jira/browse/RM-4482.

There's one new unit test and an ignored integration test. As it turns
out,
it's not even possible to assign IQueryable<Concrete> to a variable of
type
IQueryable<IAbstract> until c#/.net 4.0. There's probably another way
to
test this functionality but I didn't come up with anything.

Cheers,

Chris

On Jun 1, 4:15 am, Fabian Schmied <fabian.schm...@gmail.com> wrote:

> >>https://github.com/chriseldredge/Lucene.Net.Linq/blob/test-case/re-li...

[Fabian Schmied]

> This didn't turn out to be that difficult to implement. I've attached
> a patch
> on https://www.re-motion.org/jira/browse/RM-4482.

Wow, thanks! I'll review it some time this week.

Cheers,
Fabian

[Michael Ketting]

Thanks for the invite. I'm afraid my plate's full ATM, but I'll definitely keep it in mind when I'm actually doing stuff with Lucene.NET the next time.
Regards, Michael

[Chris Eldredge]

I just tried to confirm this issue is resolved in 1.13.161, but unfortunately I found otherwise.

Updated issue with comment:  https://www.re-motion.org/jira/browse/RM-4482?focusedCommentId=14990&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14990 

It looks like the issue was closed but the branch was never merged into trunk.

Thanks,

Chris

[Chris Eldredge]

Oops, looks like I'm completely wrong. I just tried running my test case again and it appears that something didn't refresh correctly the first time. Sorry for any confusion I may have caused.

Chris

[Fabian Schmied]

> Oops, looks like I'm completely wrong. I just tried running my test case
> again and it appears that something didn't refresh correctly the first time.
> Sorry for any confusion I may have caused.

That's very good, cause I very distinctly remember merging that branch ;)

Cheers,
Fabian

> https://groups.google.com/d/msg/re-motion-users/-/tshm1_mMdXQJ.