Handling Schema Upgrades

[Matteo Grolla]

Can you suggest a good strategy to handle schema upgrades with pouch db?

The app I'm writing has and will require changes on the documen format

I upgrade them server side in couchdb and let them replicate to clients but clients could have modified docs locally so I have conflicts

When I detect a conflict generated by a schema upgrade I make the local version win and run schema upgrade on the client for that document.

How do you handle this problem?

[Johannes J. Schmidt]

Hi Matteo,

Since CouchDB supports transactions only on the document level, an appropriate migration concept has to deal with partial migrations. This can be achieved with one database per schema version. Or by having multiple documents per schema version:

* make the version part of _id, eg `contact:62e7d76b491407b289cb7e0b3bf592c0:v:2`
* use views to restrict client access to the version(s) it understands
* reference documents by using only the `id` part of the documents, eg "contactId": "62e7d76b491407b289cb7e0b3bf592c0"
* run (continuous) server side migration processes
* store a stamp of the migration process which created/updated the document and a reference to the source

This concept has some benefits:

* support for multiple clients with different schema versions at the same time
* per document type schema versions
* continuous up and down migrations
* server side migration is more bandwidth efficient than client side since most users have better download than upload rates
* migration can be run pre-release and replicated to the clients in advance for enhanced upgrade experience
* client side migration and conflict resolution is complicated, hard to debug and can lead to loops

Johannes

--
You received this message because you are subscribed to the Google Groups "PouchDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pouchdb+u...@googlegroups.com.
To post to this group, send email to pou...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pouchdb/4d6b387f-cb02-431e-9736-f36c45250058%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matteo Grolla]

Hi Johannes,

     your suggestion is very interesting, and I'm wondering if it's applicable to my case.
This is a scenario that concerns me:

Hp: The application supports only one schema version

1) Me: release new application version that supports new schema

2) Me: upgrade schema server side

3) User: keeps editing data with old schema

4) User: upgrades application

5) User: sees stale data with new schema because they have been modified at 3)

If I upgrade schema for data modified at 3) this is what could happen

6) User: syncs to server

7) Me: upgrade schema for data modified at 3)

8) User: modifies stale data with the new schema

8) User: syncs to server and now I have conflicts on data modified at 3)

I see it as a problem that a user after an upgrade sees an old version for data he entered locally.
Am I missing something?

[Johannes J. Schmidt]

Hi Matteo,

we restrict client side access to documents via a view which only emits documents with the schema version the client understands. So newly edited documents does not show up in the clients until they have been pushed to the server, migrated and pulled back again (we do not change documents during migration, we create new documents for new schema versions - except for update migrations).

Of course this does not eliminate all possible conflicts but reduces them.

Note that we do server side migrations *before* releasing new applications, so that the apps can have the new documents already synced before they upgrade.

Johannes

To view this discussion on the web visit https://groups.google.com/d/msgid/pouchdb/55373424-04a6-4d58-abff-0b05b9f7b3aa%40googlegroups.com.