Query Limit of 128

[Devin Bialo]

I am having two separate areas where my queries are not returning
anything past the 128th entry

Firstly, this is only returning 128. Considering that the db is
supposed end up containing thousands of entries, not being able to
count past 128 is pretty troubling.

using (var session = documentStore.OpenSession())
{
return
session.Advanced.LuceneQuery<Conversation>().Count();
}


The documentation says that adding a "Take" clause will fix the issue,
but it did not. The following code does not return any values past the
128th

//startIndex = 120
//maxConversations = 10
using (var session = documentStore.OpenSession())
{
//Returns only 8 values, even though the DB currently
has more than 200 conversations
return session.Advanced.LuceneQuery<Conversation>()
.OrderByDescending(c =>
c.LastUpdated).ThenByDescending(c => c.Id)
.Skip(startIndex)
.Take(maxConversations)
.ToArray();
}

If you call it with a startIndex of 130 and a Take(10), it returns
nothing at all.

What is the correct way to implement Count and paging?

[Matt Warren]

The problem is that the Take/Skip come after the OrderBy/ThenBy and so
aren't done as part of the RavenDB IQueryable<T> provider.

So basically you're always getting the 1st 128 records from the server
(based on the orderby and thenby clauses), then the skip/take is done
in memory.

So you're really doing skip/take on 128 items (the most RavenDB will
give you in 1 go) which explains what you're seeing.

I'm note sure if this is a RavenDB bug, or just down to LINQ providers
in general. This page has some info about a possible fix and it would
mean some changes to our LINQ provider. See
http://blogs.msdn.com/b/mattwar/archive/2007/10/09/linq-building-an-iqueryable-provider-part-viii.aspx.

So unless I'm missing something there's not an easy fix, I'll wait and
see what anyone else says before looking at it some more.

[Matt Warren]

I'm not entirely sure, but I think it's a bug in Raven. It just
happens that your query shows it!!

Raven can internally handle sorting and paging, so it's just a problem
with the LINQ provider.

As a workaround you can drop down to the "magic" strings that the LINQ
provider uses internally. See the section title "Performing a low-
level Lucene query directly" here http://ravendb.net/documentation/client-api/querying
and http://www.ravendb.net/documentation/docs-http-indexes-querying
for more info.

But I'd only recommend this as a tempoary thing, it'll be far better
for the LINQ provider to handle this.

> Thanks Matt.
>
> Are you saying that this is a problem with RavenDB paging in general
> past 128 items? Or just with my query?
>
> The goal is only to implement paging.

On Feb 8, 2:53 pm, Matt Warren <mattd...@gmail.com> wrote:
> The problem is that the Take/Skip come after the OrderBy/ThenBy and so
> aren't done as part of the RavenDB IQueryable<T> provider.
>
> So basically you're always getting the 1st 128 records from the server
> (based on the orderby and thenby clauses), then the skip/take is done
> in memory.
>
> So you're really doing skip/take on 128 items (the most RavenDB will
> give you in 1 go) which explains what you're seeing.
>
> I'm note sure if this is a RavenDB bug, or just down to LINQ providers
> in general. This page has some info about a possible fix and it would

> mean some changes to our LINQ provider. Seehttp://blogs.msdn.com/b/mattwar/archive/2007/10/09/linq-building-an-i....

[Ayende Rahien]

This could would now fail to compile, which is what one would expect it to do.

[ThomasZ]

Any updates on this issue?

I searched for a couple of hours and found out that there should be a server settings to get rid of the 128 search results limit. Unfortunately I wasn't able to solve it  by changing anything in the server .config file.

What however worked was to use Take(numOfResults); 

But this would also not be a good solution as one could easily forget to use the Take method.

Any idea on how to solve this, ideally in the server settings? 

[Oren Eini (Ayende Rahien)]

The server setting is 1024 by default, but you can raise this.

The client setting is 128 by default, and you have to use Take() to modify it.

This is by design. 

The reasoning is that you could easily forget to use the Take method.