Very long indexing operation after high write rate

[Jahmai Lay]

If we spam writes to a collection (say, 20k in a minute), it takes a very long time for an index to catch up.

I notice in on the statistics page of the database something, under the indexes, something like;

Operation: Index

Input: 1

Output: 1

Duration: 00:00:00.0213412

Duration in ms: 21

It does this over and over (1 at a time) well after we have stopped writing to the database.

Is this right? I'm expecting that this should be batched, or am I not reading it right?

[Jahmai Lay]

Screenshot

[Jahmai Lay]

Interestingly if I kill the server while it is indexing, and then restart, the indexing completes almost immediately, and the stats look much more sensible (attached).

This is build 2360 btw.

[Jahmai Lay]

OK so I decided to go get my hands dirty and debug the issue.

There seems to be something fishy about this code;

private bool TryGetInMemoryJsonDocuments(Guid nextDocEtag, List<JsonDocument> items)

{

if (context.Configuration.DisableDocumentPreFetchingForIndexing)

return false;

JsonDocument result;

bool hasDocs = false;

while (inMemoryDocs.TryPeek(out result)  &&

ComparableByteArray.CompareTo(nextDocEtag.ToByteArray(),result.Etag.Value.ToByteArray()) >= 0)

{

// safe to do peek then dequeue because we are the only one doing the dequeues

// and here we are single threaded

inMemoryDocs.TryDequeue(out result);

if (result.Etag.Value != nextDocEtag)

continue;

items.Add(result);

hasDocs = true;

nextDocEtag = Etag.Increment(nextDocEtag, 1);

}

return hasDocs;

}

It will only ever return the document(s) involved in a single transaction.

I set "Raven/DisableDocumentPreFetchingForIndexing" to true and now it is indexing as you'd expect (attached).

Please note that setting the property InMemoryRavenConfiguration.DisableDocumentPreFetchingForIndexing to true had no effect - that value appears to be ignored or overwritten.

[Oren Eini (Ayende Rahien)]

Do you have the versioning bundle in there?

--
You received this message because you are subscribed to the Google Groups "ravendb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Jahmai Lay]

No versioning bundle.

The production system has Replication bundle, but this is also happening even on local system without any bundles.

[Oren Eini (Ayende Rahien)]

Hm,

Can you send me some code to try to repro that?
Also, can you test 2.5? We did a major rewrite of that portion of the code.

[Jahmai Lay]

Yup will try get to that today