Importing backup will create indexing errors and stale indexes

[Teemu Qvick]

When I create a backup from studio (admin/settings/backup) and then later restore it (admin/settings/restore) I see that all indexes are stale and it takes some time before everything is indexed. When backup is created there are no stale indexes. From the documentation I would assume that after the restore all indexes would be ready to use immediately. Is it expected behavior that all indexes are build again when restoring the backup or is there some configuration / setting to look for?

Server Build #35181

Client Build #35181

[Oren Eini (Ayende Rahien)]

Can you start the db after restore with indexing disabled and then send us the stats? This shouldn't be the case.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Teemu Qvick]

What do you mean by starting db after restore? The db is created during the restore (at least when done using studio) and I cannot figure out a way to restore to existing db for which I would disable the indexing.  

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

[Oren Eini (Ayende Rahien)]

After restore, but before anyone has accessed the database, use:

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Tommi Hirvonen]

Answering on behalf of my colleague. Apparently this only happens if not explicitly defining index location during restore (We have databases and indexes configured to separate locations, i.e. not using the default).

Anyway, reproed with the following steps:

1) Create a new database with Voron

2) Create sample data to the db and wait for indexing to complete

3) Backup the database

4) Restore to another db, enter db name and backup location only

5) Disable indexing

6) Open the restored db, see indexes: All are stale with 0 entries.

Attached are two debug packages, the first one taken right after opening the db and the second one taken after enabling indexing again. Was this the "stats" you meant, or do you require some other information?

[Oren Eini (Ayende Rahien)]

Okay, that is probably something that we should provide better errors on.

The assumption is that if you restore a db with split indexes you'll also provide it then.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--

[Tommi Hirvonen]

Wouldn't it make sense to use the configured path (Raven/IndexStoragePath) for indexes in restore unless specified otherwise? Now Raven seems to create the indexes there eventually anyway (once you enable indexing on the restored DB), but the restore operation apparently creates the indexes to the database directory where they are never used again.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

[Oren Eini (Ayende Rahien)]

This is probably correct, I've added an issue on that.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Oren Eini (Ayende Rahien)]

http://issues.hibernatingrhinos.com/issue/RavenDB-5800

[Tal Weiss]

Hi Teemu,

What is actually happening is that we create a persistent task for updating the backup status and we include it in the backup.

The indexes aren't really stale when you do the restore but we have to process that backup task before we can say that for sure.

The indexes should become non-stale immediately after the database is loaded and had the chance to run the task (this should take milliseconds after the database is loaded) 

So to sum it up your indexes didn't lose any of their data during the backup process and they should be ready for querying soon after you load the database.

[Tommi Hirvonen]

The "non-stale immediately" happens, if the indexes have been restored to a correct path, as expected. But if the index path is configured explicitly to another path (as described in my earlier reply), and the path is not provided during restore, the indexes are actually re-indexed completely, and this of course takes a quite measurable amount of time.

Apparently the bug description didn't include the index path configuration. I added a comment to the issue too.