Stale Indexes

[Paul Hinett]

Why would my indexes always become stale, I just deleted a temporary index in the management studio because it wasn’t required due to a bug in my code…

 

But right after I done that, all my indexes became stale and no results are returning in my web app from my existing indexes, I understand that they need to become stale to see if any changes were made to them but thought the results that were already indexed should still be returned…is that correct?

 

I’m having real problems at the moment with my indexes, they are fairly large and every time I make a minute change to any index it seems to kill all my other indexes for a good while until they are indexed again, surely this isn’t what is supposed to happen and i must be me doing something wrong?

[Oren Eini (Ayende Rahien)]

What build are you using? Do you have modification to the database coming in?

[Paul Hinett]

Build 700.

 

I think is what I did was actually rename a field in one my Audio index late last night, but didn’t run my web app again until today, I’m assuming this would cause a re-build of this index.

 

But I also have  a Tag_Count index, which groups tags from my Audio documents, this index was also rebuilt today, even though nothing should have changed in this index, should this of happened?

 

There was no data changed at any point.

[Oren Eini (Ayende Rahien)]

Okay, strange.

Are you running on a tenant DB, by any chance?

[Paul Hinett]

Yes

[Oren Eini (Ayende Rahien)]

Okay, tenant databases are shut down after about 10 minutes of inactivity. If you are rebuilding an index and have a large amount of data, that might explains that. 

Change the value of Raven/Tenants/MaxIdleTimeForTenantDatabase to be a high number 86,400 or something like that.

See what happens then.

What is probably happening is that because we shut down the database because it is inactive, you are only seeing indexing when you are actually looking at the database.

[Paul Hinett]

I have done that now, although I will need to do some testing to see if it makes any difference.

 

However I think I just noticed that restarting my web app (recompiling and browsing my site) caused all my indexes to be stale again.

The index creation code on startup, will this delete and re-create indexes, it appears so.

 

This is my index creation code:

var catalog = new CompositionContainer(new AssemblyCatalog(Assembly.GetAssembly(typeof(Tag_Count))));

var catalog2 = new CompositionContainer(new AssemblyCatalog(Assembly.GetAssembly(typeof(Audio_Index))));

 

IndexCreation.CreateIndexes(catalog, DocumentStore.DatabaseCommands.ForDatabase(databaseName), DocumentStore.Conventions);

IndexCreation.CreateIndexes(catalog2, DocumentStore.DatabaseCommands.ForDatabase(databaseName), DocumentStore.Conventions);

 

Ignore that fact that I have 2 catalogues, I need to refactor this.

 

Paul

[Oren Eini (Ayende Rahien)]

That is strange, this should be a NOOP, because you are creating the same thing.

Can you check whatever you have something in the index that is changing for each compilation?

[Paul Hinett]

It’s definitely when I call CreateIndexes, If I add a breakpoint on that line then step through, it causes just 2 indexes to go stale out of 29 every time.

 

1 Index is just a Map index called Audio_Index.

 

The other is a Map/Reduce index called Tag_Count.

 

I can re-produce this every time I run my web app without making any modifications and no data changes.

 

Index code for both is here, don’t think there is anything unusual about them:

https://gist.github.com/2007710

[Paul Hinett]

Ok getting somewhere, If I remove ‘Tags’ from my Audio_Index it no longer puts them into a stale state.

 

Is this by design or a bug?

[Oren Eini (Ayende Rahien)]

This is a bug, yes.

What do you mean, remove tags?

Can you try creating reproducable test case?

[Paul Hinett]

Well after a little more testing, I have no idea what was going on earlier, it seems to be ok now, maybe the config option I added earlier done something, I’m going to keep an eye on it though.

 

However, my Tag_Count index seems to take forever to index all my tags compared to all my other indexes.

 

I currently have around 150,00 Audio documemts, each having an array of strings called Tags, on average there will be around3-5 tags per Audio. There are 25,000+ unique tags in total.

 

My Tag_Count index aggregates all these tags together by name and gives a count…am I giving my index too much work to do here?

 

I suppose in production it won’t be so bad after the initial indexing is complete, but while testing and altering indexes etc it’s taking about 20 minutes to re-index and seems to block other indexes being indexed until it’s complete.

 

Sorry for flurry of questions!

[Paul Hinett]

After playing around with some more testing, I have managed to create a failing test where by my index should not be stale.

 

I think it’s related to specifying my index to use an Analyzer, like so:

 

Analyzers.Add(x => x.Name, "SimpleAnalyzer");

 

A self –contained test project can be found here:

http://www.house-mixes.com/test/HM.Tests.Individual.rar

 

I had to use a RavenDb server rather than embedded, embedded always returned a stale result for some reason.

 

The first test run will create the dummy data and will always fail the test because it has to index the data, but once the data is indexed if you run the test again it fails again with stale data, removing the analyzer (and running the test twice giving it chance to re-index the data), or not calling IndexCreation.CreateIndexes passes the test.

 

Hope this helps to find problem.

 

Paul

[Matt Warren]

However, my Tag_Count index seems to take forever to index all my tags compared to all my other indexes.

This is expected, your Tag_Count index is a Map/Reduce index, whereas your other indexes are just Map indexes. There's more work to be done in a Map/Reduce index. First it has to run the Map statement over all 150,000 docs, producing an intermediate results for each tag, something like { "tagName", 1 }. Then it has to group batches of these results together to end up with something like { "tagName", 1525 }

I currently have around 150,00 Audio documemts, each having an array of strings called Tags, on average there will be around3-5 tags per Audio. There are 25,000+ unique tags in total.

My Tag_Count index aggregates all these tags together by name and gives a count…am I giving my index too much work to do here?

If you want just Count of individual tags, you can use Faceted search. See http://ravendb.net/docs/client-api/advanced/faceted-search and there's an code sample here https://gist.github.com/1214297 that should do exactly what you want.

I suppose in production it won’t be so bad after the initial indexing is complete, but while testing and altering indexes etc it’s taking about 20 minutes to re-index and seems to block other indexes being indexed until it’s complete

Yeah, adding a few new docs to a Map/Reduce index isn't such an expensive operation, RavenDB is intelligent about how much new work it has to do.

 

[Paul Hinett]

Didn't think to use faceted search, will give this a go later.

In the meantime did anyone notice the fairly big problem when using an analyser in your index, causing your index to rebuild on every call to IndexCreator.CreateIndexes?

I posted a failing test to reproduce it.

That was what was causing my indexing to be dog slow. I'm much happier now I have removed the analyser, however i need the analyser for searching to work.

Thank you,

Paul

Sent from my iPad

[Oren Eini (Ayende Rahien)]

Thanks, I reproduced the issue, you should be able to generate fix that by specifying the full namespace for the analyzer.

Fixed in the next build.