How should stale indexes be handled during testing?

204 views
Skip to first unread message

Arnold Zokas

unread,
Jan 29, 2012, 10:12:58 PM1/29/12
to rav...@googlegroups.com
I am using RavenDB in In-Memory mode for unit testing. My queries are backed by static indexes. I am not using `WaitForNonStaleResults()` API (nor do I want to).

Typical workflow for a test is:
  1. Initialise RavenDB in In-Memory mode
  2. Integrate indexes using `IndexCreation.CreateIndexes(Assembly, IDocumentStore)`
  3. Insert test data (for verifying  query behaviour)
  4. Run query
  5. Verify query output
I have noticed steps 1-3 happen so quickly, static indexes have time to get updated before step 4 - indexes are stale.
I have created a quick work-around for this. After step 3, I execute:

    while (documentStore.DocumentDatabase.Statistics.StaleIndexes.Length != 0)
        Thread.Sleep(10);

This feels cumbersome. What I would like to know is:
  • Is it normal for indexes to be stale when running RavenDB in In-Memory mode?
  • Is there a better way to avoid stale indexes during testing?

Javier Fernandez-Ivern

unread,
Jan 29, 2012, 10:21:21 PM1/29/12
to rav...@googlegroups.com
If the writes and reads happen in the same session, you could pass in Conventions = { DefaultQueryingConsistency = ConsistencyOptions.QueryYourWrites } to the document store initializer in your tests.  That'll guarantee that any queries you run get results consistent with previous writes done *in that same session*.
--
Javier Fernandez-Ivern

arcbees.com

Arnold Zokas

unread,
Jan 29, 2012, 10:36:51 PM1/29/12
to rav...@googlegroups.com
That worked. Thanks!

Very useful tip - this simplifies some of test infrastructure code.

Justin A

unread,
Jan 29, 2012, 11:11:46 PM1/29/12
to rav...@googlegroups.com
Arnold, also check out my RavenOverflow sample product. In the Tests project, I handle stale indexs and stuff .. similar to what you've got. Plus i make sure all queries require indexes to be not-stale.


i should be finishing my youtube video for this, hopefully this week *fingers crossed that kids to hassle me*

Arnold Zokas

unread,
Jan 30, 2012, 7:05:41 AM1/30/12
to rav...@googlegroups.com
Thanks Justin. I had a look at your code, specifically NoStaleQueriesListener. I didn't know you could do that with listeners.

Justin A

unread,
Jan 31, 2012, 7:20:10 AM1/31/12
to rav...@googlegroups.com
:) :) Isn't RavenDb awesome? :)

Chris Marisic

unread,
Jan 31, 2012, 10:21:36 AM1/31/12
to rav...@googlegroups.com
Just be careful you never ship that to production.

Justin A

unread,
Apr 1, 2012, 7:42:07 AM4/1/12
to rav...@googlegroups.com
@Arnold Zokas,

I know this is a wee bit late, but (i'm pretty sure) the NoStaleQueriesListener doesn't work for indexes.

Why is this important?

With your unit tests, each Fact/Test should be independent of each other .. and therefore the embedded ravendb should be seeding some fake data .. for each test :)

Now, the test/fact which is being asserted -might- require a static index. now, it's possible that the index required might be stale .. because the test is running so fast, that the index hasn't had a chance to complete.
As such, I do your 'Sleep' thing.

So, whenever i SEED some data, i then sleep while the indexes are stale.

This also has a SECOND bonus reason. For my actual website, i -love- using fake data for my development and when i need to get some hard manual testing, i switch the db over to a copy of a real db snapshot. Now, in dev mode, cause i'm using fake data .. the web site might fire off the indexing, but the main web page might show so quickly, that the index (which the home page needed) was still stale .. as therefore nothing showed on the homepage.

for example : RavenOverlflow demo app. The homepage uses 2 indexex. When i first ran the website on my development localhost box, it seeded data, starting indexing .. but the homepage was always -empty- or blank. I hit refresh and viola! data now showed. This meant that my computer was running so fast, the indexes didn't get a chance to complete before the normal ASP.NET lifecycle completed :P Or more to the point, the ravendb session which queried the indexes in the ActionMethod was getting called so quickly .. the index (or indexes) was/were still stale!

So i've updated my RavenOverflow demo project with code that shows how my unit tests take leverage of this.

Check out the global.asax and RavenDbFactBase.cs files to see how i leverage both the Listener and the Sleeping technique when seeding.

Arnold Zokas

unread,
Apr 1, 2012, 10:26:01 AM4/1/12
to rav...@googlegroups.com
Hi Justin,

Never too late :-)

Comments in red.

On Sunday, 1 April 2012 12:42:07 UTC+1, Justin A wrote:
@Arnold Zokas,

I know this is a wee bit late, but (i'm pretty sure) the NoStaleQueriesListener doesn't work for indexes.
   NoStaleQueriesListener does work. I use ot it on my project with no problems.

Why is this important?

With your unit tests, each Fact/Test should be independent of each other .. and therefore the embedded ravendb should be seeding some fake data .. for each test :)
   I am using proper test isolation (spin up new embedded db per test, seed data per test, etc.). That was never the issue.

Now, the test/fact which is being asserted -might- require a static index. now, it's possible that the index required might be stale .. because the test is running so fast, that the index hasn't had a chance to complete.
As such, I do your 'Sleep' thing.

So, whenever i SEED some data, i then sleep while the indexes are stale.
   I started out by using sleep, but that approach resulted in slower test execution (at least in my solution, this may be different in yours).

This also has a SECOND bonus reason. For my actual website, i -love- using fake data for my development and when i need to get some hard manual testing, i switch the db over to a copy of a real db snapshot. Now, in dev mode, cause i'm using fake data .. the web site might fire off the indexing, but the main web page might show so quickly, that the index (which the home page needed) was still stale .. as therefore nothing showed on the homepage.

for example : RavenOverlflow demo app. The homepage uses 2 indexex. When i first ran the website on my development localhost box, it seeded data, starting indexing .. but the homepage was always -empty- or blank. I hit refresh and viola! data now showed. This meant that my computer was running so fast, the indexes didn't get a chance to complete before the normal ASP.NET lifecycle completed :P Or more to the point, the ravendb session which queried the indexes in the ActionMethod was getting called so quickly .. the index (or indexes) was/were still stale!
   This is normal behaviour as far as indexes are concerned. I was mainly interested in getting my unit tests to work reliably. I do not use NoStaleQueriesListener outside unit tests.

Justin A

unread,
Apr 1, 2012, 11:36:26 PM4/1/12
to rav...@googlegroups.com
I never use  NoStaleQueriesListener  outside of my (embedded ravendb) Test project. Totally agreed there!! (that would be a disaster!)

But I swear I thought the listener didn't work for indexes. I'll have to double check. Maybe my problem was that the indexes (in each test/fact) were stale, still .. when my test executed the 'Act' part of the Arrange/Act/Assert ? But if they were stale .. then the listener would force the query to wait .. hmm ... Time to do some more testing.

Oh .. another thing with indexes .. i've updated my Facts/Tests to only specify which indexes to execute. Like why execute all the indexes, when a particular test might only use 1? or even none? :) (code is in my public github repo).

Arnold Zokas

unread,
Apr 2, 2012, 8:21:02 AM4/2/12
to rav...@googlegroups.com
Yes, I noticed that. Executing specific indexes is a good way to speed up tests :-)
Reply all
Reply to author
Forward
0 new messages