Strange issue with dynamic query/index (v3.5)
38 views
Skip to first unread message
Nick
Jan 12, 2018, 4:51:57 PM1/12/18
to RavenDB - 2nd generation document database
Hi,
I have a simple query that doesn't use an index (so it relies on a dynamic index behind the scene):
RavenSession.Query<Booking>()
.Where(x => x.UserId == LoggedInUser.Id)
.SingleOrDefault();
In the current state of my DB, it should return a single result but instead it gives a null result.
So I went to Studio and queried the dynamic/bookings index. Same thing: 0 result.
I hit the Query Stats button, a popup appears, and Oh Surprise, the Index line shows the name of another index in my DB.
This index's Map has 2 fields that do some calculations on date fields, like:
MonthId1 = 12 * (this2.this1.this0.start.Year - 1) + this2.this1.this0.start.Month
and this eliminates the result I was expecting, simply because its Date field is null.
Note that if I remove the UserId property in the query (therefore requesting all the entities), when I click the Query Stats button, the Index line equals "Raven/DocumentsByEntityName" this time, and I can see my expected result in the returned list.
So my question is: why the dynamic feature chose one of my existing indexes, an incorrect one, which is something I did not expect at all? I thought dynamic was creating a new index with only the fields I was querying...
This is the first time I meet such a case.
Thanks
Oren Eini (Ayende Rahien)
Jan 13, 2018, 6:52:08 AM1/13/18
to ravendb
This is the default behavior in 3.5 (in 4.0 we changed it so there is no possibility of a mix).
If there is a static index that can match the query, we'll use it.
But it should know that it is not a valid option there. Can you post the full index def?
Hibernating Rhinos Ltd
Oren Eini l CEO l Mobile: + 972-52-548-6969
Office: +972-4-622-7811 l Fax: +972-153-4-622-7811
--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Nick
Jan 13, 2018, 11:04:58 AM1/13/18
to RavenDB - 2nd generation document database
Here it is:
As expected, if I replace the MonthIdx fields by
Map = bookings => from booking in bookings
let start = booking.StartDate.Value
let end = booking.EndDate.Value
let game = LoadDocument<Game>(booking.GameId)
select new
{
GameId = booking.GameId,
MonthId1 = 12 * (start.Year - 1) + start.Month,
MonthId2 = 12 * (end.Year - 1) + end.Month,
Status = booking.Status,
UserId = booking.UserId,
StartDate = booking.StartDate.Value,
EndDate = booking.EndDate.Value,
CodeFrom = booking.PassCode.CodeFrom,
CodeTo = booking.PassCode.CodeTo,
TokenAssigned = game.TokenUser > 0,
HasTheGame = game.GameAtHomeUser == booking.UserId
};
MonthId2 = (end != null ? 12 * (end.Year - 1) + end.Month : (int?)null)
then it works when the dynamic feature chooses this index.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
Oren Eini (Ayende Rahien)
Jan 13, 2018, 2:22:15 PM1/13/18
to ravendb
Your query in the original post was on UserID ?
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
Nick
Jan 13, 2018, 2:27:19 PM1/13/18
to RavenDB - 2nd generation document database
Actually, my final query is some AND/OR on UserId, GameId and Status.
But for the simplicity of the test, I also tried with UserId only and the issue is here in both cases.
Oren Eini (Ayende Rahien)
Jan 13, 2018, 2:40:39 PM1/13/18
to ravendb
From the information provided, it seems like it is a good decision on the query optimizer part.
You are querying on booking.UserId, and that looks like it is a valid field.
Can you create a failing test for this?
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
Nicolas Cadilhac
Jan 13, 2018, 2:52:40 PM1/13/18
to rav...@googlegroups.com
So, if you think this is a VALID behaviour (but which makes my code not "just work"), why treat it as a FAIL in a failing test? I whould just supply a specific static index or use the workaround I mentioned, no?
--
You received this message because you are subscribed to a topic in the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ravendb/zx1spc81etw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ravendb+unsubscribe@googlegroups.com.
Message has been deleted
Oren Eini (Ayende Rahien)
Jan 13, 2018, 2:56:08 PM1/13/18
to ravendb
This looks like a valid behavior, and it shouldn't cause your query to not work.
I don't see _how_ that can happen, that is why I asked for a test.
Nick
Jan 13, 2018, 2:57:33 PM1/13/18
to RavenDB - 2nd generation document database
Do you have some kind of right template I should use for the test? And where should I put it? Just send it by email to you?
In any case, I will do it when I can allocate some time for it.
--
You received this message because you are subscribed to a topic in the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ravendb/zx1spc81etw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Oren Eini (Ayende Rahien)
Jan 13, 2018, 3:05:03 PM1/13/18
to ravendb
Yes, see here:
You can send it here or to sup...@ravendb.net
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
Nicolas Cadilhac
Jan 13, 2018, 3:06:04 PM1/13/18
to rav...@googlegroups.com
Perfect. Thanks.
To unsubscribe from this group and all its topics, send an email to ravendb+unsubscribe@googlegroups.com.
Nick
Jan 13, 2018, 4:34:02 PM1/13/18
to RavenDB - 2nd generation document database
Failing test sent by email (zip with complete solution). Here is the code:
namespace RavenDbFailingTest
{
public class Booking
{
public string GameId { get; set; } // We could get rid of that for the test
public string UserId { get; set; }
public DateTime? Date { get; set; }
}
public class MyFailingTest : RavenTestBase
{
public class BookingIndex : AbstractIndexCreationTask<Booking>
{
public BookingIndex()
{
Map = bookings => from booking in bookings
let date = booking.Date.Value
select new
{
GameId = booking.GameId,
MonthId = 12 * (date.Year - 1) + date.Month,
UserId = booking.UserId
};
}
}
[Fact]
public void SampleTestMethod1()
{
using (DocumentStore store = NewRemoteDocumentStore())
{
new BookingIndex().Execute(store);
using (IDocumentSession session = store.OpenSession())
{
session.Store(new Booking
{
UserId = "users/1",
Date = null,
GameId = "games/33"
});
session.SaveChanges();
}
WaitForIndexing(store);
using (IDocumentSession session = store.OpenSession())
{
var booking = session
.Query<Booking>()
.Where(x => x.UserId == "users/1")
.SingleOrDefault();
Assert.NotNull(booking);
}
}
}
[Fact]
public void SampleTestMethod2()
{
using (DocumentStore store = NewRemoteDocumentStore())
{
using (IDocumentSession session = store.OpenSession())
{
session.Store(new Booking
{
UserId = "users/1",
Date = null,
GameId = "games/33"
});
session.SaveChanges();
}
using (IDocumentSession session = store.OpenSession())
{
var booking = session
.Query<Booking>()
.Where(x => x.UserId == "users/1")
.SingleOrDefault();
Assert.NotNull(booking);
}
}
}
[Fact]
public void SampleTestMethod3()
{
using (DocumentStore store = NewRemoteDocumentStore())
{
new BookingIndex().Execute(store);
using (IDocumentSession session = store.OpenSession())
{
session.Store(new Booking
{
UserId = "users/1",
Date = null,
GameId = "games/33"
});
session.SaveChanges();
}
WaitForIndexing(store);
using (IDocumentSession session = store.OpenSession())
{
var bookings = session
.Query<Booking>()
.ToList();
Assert.Single(bookings);
}
}
}
}
}
On Saturday, 13 January 2018 15:06:04 UTC-5, Nick wrote:
Perfect. Thanks.
On 13 January 2018 at 15:04, Oren Eini (Ayende Rahien) <aye...@ayende.com> wrote:
Yes, see here:You can send it here or to sup...@ravendb.net
Hibernating Rhinos Ltd
Oren Eini l CEO l Mobile: + 972-52-548-6969
Office: +972-4-622-7811 l Fax: +972-153-4-622-7811
Do you have some kind of right template I should use for the test? And where should I put it? Just send it by email to you?
In any case, I will do it when I can allocate some time for it.
On Saturday, 13 January 2018 14:56:08 UTC-5, Oren Eini wrote:
This looks like a valid behavior, and it shouldn't cause your query to not work.I don't see _how_ that can happen, that is why I asked for a test.
Hibernating Rhinos Ltd
Oren Eini l CEO l Mobile: + 972-52-548-6969
Office: +972-4-622-7811 l Fax: +972-153-4-622-7811
Oren Eini (Ayende Rahien)
Jan 14, 2018, 2:30:47 AM1/14/18
to ravendb
Okay, now I know what is going on.
This is intersection of a few very different things.
First, the query optimizer will correctly select the BookingIndex for you.
However, the BookingIndex will error on a particular record if the date is null.
The why is interesting, you have this:
MonthId = 12 * (this0.date.Year - 1) + this0.date.Month,
But date is null, so this is translated, eventually, to:
ExpressionType arg_32_1 = ExpressionType.Add;
CallSite<Func<CallSite, int, object, object>>.Create(Microsoft.CSharp.RuntimeBinder.Binder.BinaryOperation(arg_32_0, arg_32_1, arg_32_2, expr_28));
If you care to, this is the full thing:
The problem is that this is binding on the left, which is the int, which we can't access.
If you would change the index to have the value on the right:
MonthId = ((this0.date.Year - 1) *12) + this0.date.Month,
It would work properly because it would bind things.
This is a bug, because the query optimizer in this case obviously select an index with errors, which it can detect and avoid, but I'm not sure whatever we can/should fix it for backward compatability issues.
Nick
Jan 14, 2018, 9:48:17 AM1/14/18
to RavenDB - 2nd generation document database
Your choice. On my side, I have my workaround. But who knows if I will again meet this bug in the future without realizing this is this issue again...
Reply all
Reply to author
Forward
0 new messages