GetDocumentsWithIdStartingWith optimization

[Jesús López]

Hi,

 

I have written a new overload of GetDocumentsWithIdStartingWith method which is faster than the original. This overload takes lastReadId instead of start parameter:

 

public IEnumerable<JsonDocument> GetDocumentsWithIdStartingWith(string idPrefix, string lastReadId, int take)

 

Reading 200000 documents with a page size of 1000 documents using ESENT takes nearly half time. The culprit of such a improvement is this piece of code that can be avoided:

 

 

while (start > 0)
{
 if (Api.TryMoveNext(session, Documents) == false)
  return Enumerable.Empty<JsonDocument>();
 start--;
}

 

 

I added the new overload to IDocumentStorageActions interface, implemented the method in both Raven.Storage.Managed.DocumentsStorageActions and Raven.Storage.Esent.StorageActions. I also added the method to Raven.Database.DocumentDatabase class.

 

Questions:

 

Would you accept a pull request?

Which additional classes would I need to modify to expose the new overload at the client side?

 

Thanks.

 

 

[Oren Eini (Ayende Rahien)]

Can you take a look at the latest unstable?

I fixed this issue already.

[Jesús López]

I thought I were using the latest unstable. It isn't this?:

 

https://github.com/ayende/ravendb/blob/1.2/Raven.Database/Storage/Esent/StorageActions/Documents.cs

 

I see the same culprit code.

 

El miércoles, 1 de agosto de 2012 15:39:18 UTC+2, Oren Eini escribió:

Can you take a look at the latest unstable?

I fixed this issue already.

[Oren Eini (Ayende Rahien)]

Sorry, it got lost in the merging, fixed in the next build.

[Oren Eini (Ayende Rahien)]

Oh, I see, I made the fix for another function, not this.

Resolved anyway.

[Jesús López]

Oren,

 

Now this method seems buggy.

 

do

{

Api.MakeKey(session, Documents, idPrefix, Encoding.Unicode, MakeKeyGrbit.NewKey | MakeKeyGrbit.SubStrLimit);

if (Api.TrySetIndexRange(session, Documents, SetIndexRangeGrbit.RangeUpperLimit | SetIndexRangeGrbit.RangeInclusive) == false)

return Enumerable.Empty<JsonDocument>();

if (TryMoveDocumentRecords(start, backward: false))

return Enumerable.Empty<JsonDocument>();

optimizer.Add();

} while (Api.TryMoveNext(session, Documents) && optimizer.Count < take);

 

 

You are moving on each iteration which seems wrong.

You are also making the key on each iteration which also seems wrong.

 

 

I also would like to point out that TryMoveDocumentRecords returns false when it success which is odd.

[Oren Eini (Ayende Rahien)]

Correct on both counts, thanks for that.

I'll have a fix soon.