Error - Version store for this instance (1) has reached its maximum size

[John]

I am running build #888
Whenever I run indexing getting the following error :

Raven (13148) C:\Raven\Rolls-RavenDB\packages\RavenDB.1.0.888\server
\Tenants\Proquest\Datac4951621-3c3f-485a-870b-3a60fc29a2ce: The
version store for this instance (1) has reached its maximum size of
1023Mb. It is likely that a long-running transaction is preventing
cleanup of the version store and causing it to build up in size.
Updates will be rejected until the long-running transaction has been
completely committed or rolled back.
Possible long-running transaction:
SessionId: 0x00000000012E1100
Session-context: 0x00000000
Session-context ThreadId: 0x0000000000002AB4
Cleanup: 1

[Oren Eini (Ayende Rahien)]

As the error says, you are doing _something_ big in a transaction?

We usually see this when we have users saving hundreds of thousands of documents in a single transaction.

What are you doing to cause this?

[John Selvaraj]

I am just running the following index from management console app. This will create distinct index for the fieds specified.

 

My database is huge, so far 2 million unique rows have been created. But getting that error... why ?

 

 

map:

 

from doc in docs.Rolls
from docRollIndexListItem in (IEnumerable<dynamic>)doc.RollIndexList from docBookIndexListItem in (IEnumerable<dynamic>)doc.BookIndexList
select new {DocumentType = docRollIndexListItem.Index.DocumentType, Year = docBookIndexListItem.Index.Year, Month = docBookIndexListItem.Index.Month, Day = docBookIndexListItem.Index.Day, Title = docBookIndexListItem.Index.Title}

 

 

reduce:

 

from result in results
group result by new {result.DocumentType, result.Year, result.Month, result.Day, result.Title} into g
select new { g.Key.DocumentType,  g.Key.Year,  g.Key.Month,  g.Key.Day,  g.Key.Title}

[Matt Warren]

from doc in docs.Rolls
from docRollIndexListItem in (IEnumerable<dynamic>)doc.RollIndexList
from docBookIndexListItem in (IEnumerable<dynamic>)doc.BookIndexList

select new {.....}

You've got 2 million Roll items, how many sub items does each one have in RollIndexList and then in BookIndexList?

For example if each RollIndexList has 10 items and each BookIndexList inside that has a further 10 items that 2 million X 10 X 10 = 200 million items that are being processed in the Map statement, that's huge!!!

You might need to change the batch size setting, RavenDB indexes 2500 docs at a time, with that many sub-items per doc it's having problems. See "Raven/MaxNumberOfItemsToIndexInSingleBatch" 'on this page http://ravendb.net/docs/server/administration/configuration

[Matt Warren]

If you set "Raven/MaxNumberOfItemsToIndexInSingleBatch" to 250 or 100 does that solve the issue?

[Oren Eini (Ayende Rahien)]

You are generating a single map result for each Roll, multiple by the  BookIndexList.

That might be causing it to generate the error.

Configure Raven/MaxNumberOfItemsToIndexInSingleBatch to be a relatively low number (1,024 or so), does this helps?