Is replication synchronous (on Android)?

[Denis Howlett]

Hi,

We're doing filtered replications to and from Cloudant and I'm trying to understand how the replications work. We've particularly got problems when there's poor internet connection.

Here's what happens:

1. We POST a _replicate request to CBL with Cloudant as the source, CBL as the target and a filter to use. 

2. With a poor connection, this replication sometimes takes more than 5 minutes to return --> I was expecting it to return immediately with a session_id and ok: true indicating that the replication had been kicked off.

3. When it does eventually return, it returns a session_id and ok:true.

4. If I now query _active_tasks, there isn't anything with this session id (presumably because it only returned when the replication was complete?)

As a result, I've no way of knowing what happened during the replication and whether there were any changes.

Sometimes it seems that the replication finishes quicker than expected but documents which I know are on the source db don't appear in CBL. We've had a case just now where we seem to have 6 documents of the 7 we were expecting. What happens if the connection is dropped during a replication? Does it just add/update the documents it's got so far or is it "transactional" (i.e. all or nothing)? How does it work if you then re-sync? Is it able to work out that the extra document should be synced? We're finding that we don't necessarily get these documents on the next replicate.

So one of the "optimisations" I'm now doing is to query the Cloudant _changes feed to see if there are any new changes and not calling the replication if there are no new changes. I'd expect the replication protocol to do this already, but I'm guessing that since it's a filtered replication, this gets more complicated? I'm happy in my case to see first if there are any changes at all. If none, then no replication needed. If there are any, then I'll kick off the replication and let it decide if any of these changes apply to me.

We're using Cordova and the Couchbase-Lite-PhoneGap-Plugin. Our app needs to work in difficult conditions where the internet connection could be poor and could drop in and out.

Thanks very much for your help.

Denis

[Jens Alfke]

On Jan 21, 2016, at 4:17 AM, Denis Howlett <denis....@isocra.com> wrote:

1. We POST a _replicate request to CBL with Cloudant as the source, CBL as the target and a filter to use. 

2. With a poor connection, this replication sometimes takes more than 5 minutes to return --> I was expecting it to return immediately with a session_id and ok: true indicating that the replication had been kicked off.

Continuous replications send a response immediately; one-shot replications don’t send a response until complete. (That’s not our idea, it’s how CouchDB implemented it and we’re going for compatibility.)

We've had a case just now where we seem to have 6 documents of the 7 we were expecting.

That shouldn’t happen unless the connection is lost.

What happens if the connection is dropped during a replication? Does it just add/update the documents it's got so far or is it "transactional" (i.e. all or nothing)?

The puller adds docs as it goes. There is no notion of a transaction.

How does it work if you then re-sync? Is it able to work out that the extra document should be synced?

Yes, it keeps track of how far it got last time, and will pick up from there on the next replication.

—Jens

[Denis Howlett]

Thanks Jens,

That's useful to know. However, if _replicate isn't responding until the replication is complete, shouldn't it return info like the following? This is what we get back from Cloudant and Apache CouchDB.

1. history: Array[4]
2. no_changes: true
3. ok: true
4. replication_id_version: 3
5. session_id: "e433737319339f83d02906a6010a09df"
6. source_last_seq: 51

Also... what happens if we put a timeout on our call to _replicate? Does CBL continue with the replication?

Thanks again

Denis