Exporting and Importing CBLDocuments

[Brendan Duddridge]

Hi,

I would like to make an export routine to store a whole bunch of CBLDocuments in a single file. I would like to later import that file into the same or different database. Imagine a scenario where a user has a bunch of data they send to another person, that other person makes changes to the data, then sends it back. I know this can easily be accomplished by syncing, but there are other reasons for wanting to share data like this. Another example might be that someone maintains a list of movie titles and reviews and wants to share that data on my website forum so that anyone else can download that same data into their database.

I'm thinking that I will either need to use the properties or userProperties property of the CBLDocument class. I know that properties contains the CBL metadata information who's properties start with the underscore character (e.g. "_id" and "_rev"). So I was just wondering if I should avoid using properties and just use userProperties because of that?

But then again, I have relationships in my model that I want to maintain, so I think maybe using properties is the correct thing to do in this case. That way all my relationships will be intact after the import.

Are there any issues I should watch out for by doing this? 

Thanks,

Brendan

[Mark]

Brendan,

Interesting, I was just thinking about this for one of my projects after remembering Flying Meat uses SQL as a “file format” (http://shapeof.com/archives/2013/4/we_need_a_standard_layered_image_format.html). One thing I was considering is just using a local CBL database to CBL database sync for import and export (I’m not sure if that’s what you are talking about?). Let CBL deal with the merge/diff issues. I *think* that will work, but I haven’t tested it yet.

Good luck!

> --
> You received this message because you are subscribed to the Google Groups "Couchbase Mobile" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to mobile-couchba...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/3575ddcc-3856-44d9-a488-4d6809084f6f%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Brendan Duddridge]

Hi Mark,

That's not a bad idea actually. The benefit of a .json file though is that you can exchange data with external systems. But creating a CBL database of just the content you want to import and export and setting up a simple pull replication between that file and the main database is a really good idea! I suppose even exporting the data can be just a filtered CBLReplication to an empty CBL database file.

Thanks for the input!

Brendan

[Jens Alfke]

On Mar 19, 2016, at 3:35 PM, Brendan Duddridge <bren...@gmail.com> wrote:

I would like to make an export routine to store a whole bunch of CBLDocuments in a single file. I would like to later import that file into the same or different database. Imagine a scenario where a user has a bunch of data they send to another person, that other person makes changes to the data, then sends it back. I know this can easily be accomplished by syncing, but there are other reasons for wanting to share data like this.

The simplest way would be to create a JSON document consisting of an array of objects, one per document. You can write this really easily without scalability problems by just writing a “[“, then writing the docs one at a time with a “,” after each one but the last, then ending with a “]”.

It’s not much harder to write and read the docs the way the replicator does — this would give you a sort of asynchronous replication where the recipient of the blob can import the docs into its db and the right thing happens if it already has earlier versions or the same versions of docs. On the master branch (i.e. 1.3) there’s a variant of -[CLBDocument put:] that inserts an existing revision keeping its revID and history; you need that for the receiving side. For the sending side you have to add a property — I think it’s _revisions — that contains the rev history of the document.

—Jens