Re: Asynchronous replication

[Jens Alfke]

On Feb 10, 2014, at 3:26 AM, endika....@gmail.com wrote:

So this is the code I have having as CBLReplictaion pullReplication and pushReplication:
http://pastie.org/private/a35qxzxa8yba8rsuoeznq

Looks basically OK, but this test is incorrect:

if (object == self.pullReplication && [self.pullReplication changesCount] == 0) {

        NSLog(@"Info : CouchDBProvider Replication Done");

}

The right way to check if the replication is done is to compare its status property to kCBLReplicationStopped (or kCBLReplicationIdle for a continuous replication.) The sample apps including Grocery Sync demonstrate this.

The problem with checking changesCount is that it'll also be zero at the start of replication before the replicator figures out what documents it needs to download.

The point is that the replication is done correctly, but I have observed that it's done quickly untill I get the following warning:
2014-02-10 12:17:39.530 xxxx[9100:501b] -[CBL_FMDatabase executeUpdate:error:withArgumentsInArray:orVAList:]:757 Database busy (/Users/xxxxx/Library/Application Support/xxxxxx/CouchbaseLite/uid_52f206f367842.cblite)

This means that a thread is keeping the SQLite database busy for a long time, blocking another thread from accessing it. It isn't an error, but it indicates a performance problem.

When this happens, the replication is stopped for a time. It can take the replication to resume maybe 1 minute or 2. Can I do something in order to wait less time.
For more information, I am moving moreless 35.000 documents with a total size of 25MB. 

My guess is that your main CBL-using thread is querying a view when this is going on. The initial query of a view has to run the map function on every document in the database. It shouldn't take as long as you're saying (a minute), but perhaps either (a) your map function is inefficient, or (b) CBL's view updating code is behaving slowly in this case.

The way to tell is to pause the app in the debugger while this is going on, and look at what your threads are doing. If your database thread is inside a call to -[CBLQuery run:], then this is what's going on. In that case could you post a backtrace of that thread? (Select the thread and then enter "bt" in the debugger console.)

—Jens

[Endika Montejo]

Thanks a lot Jens!

Then I change that method to:

- (void) observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object

                         change:(NSDictionary *)change context:(void *)context

{

    if (object == self.pullReplication && [self.pushReplication status] == kCBLReplicationStopped) {

        NSLog(@"CouchDBProvider Replication Done");

    }   

    if (object == self.pushReplication && [self.pushReplication status] == kCBLReplicationStopped ) {

    }

}

And I also add the following method to synchronously make sure the replication is completed:

-(void) completeSyncReplication{

    while (([self.pullReplication status] > kCBLReplicationStopped ) && ([self.pushReplication status] > kCBLReplicationStopped )){

    }    

}

Thanks a lot!

[Jens Alfke]

On Feb 10, 2014, at 8:40 AM, Endika Montejo <endika....@smoobility.com> wrote:

> -(void) completeSyncReplication{
> while (([self.pullReplication status] > kCBLReplicationStopped ) && ([self.pushReplication status] > kCBLReplicationStopped )){
> }
> }

Yikes! Never write a loop like that, in any context. It's going to run the CPU at 100% power while it waits, draining battery like crazy.

It's not a good idea to waiting block for replication to complete. Instead, trigger the action you're waiting to perform in response to getting a notification.

If you do have to wait, at least add something like [NSThread sleep: 0.01] in the body of the loop so it's not continually busy.

—Jens

[Endika Montejo]

Thanks, yes I will work with observes... really much better

[Endika Montejo]

Hi Jens,

I would like to ask you one question.

I am working in my Mac app (I build in here the couchbase framework) and even though the replication is done I find several unexpected behaviours:

- When I create the replication I don't start the pusher and the puller, but the replication works. (From what I have read the replication must be started)

- Even when I stop asynchronously the replication, it keeps working.

- In the observer I log the completedChangesCount and changesCount of both pusher and puller and I always get a 0 in both properties.

Here you have my code (with the start method commented):

http://pastie.org/private/sajsntzy8ii7qlpmajdng

Notice in the observer I should append into the if's "&& self.xxxreplication completedChangesCount >0, but as I have said it always log me a 0.

Thanks a lot!

[Jens Alfke]

On Feb 13, 2014, at 3:41 AM, Endika Montejo <endika....@smoobility.com> wrote:

- When I create the replication I don't start the pusher and the puller, but the replication works. (From what I have read the replication must be started)

New replications must be started. You made the replications persistent, so on the next launch of the app the replications already exist and start automatically without your intervention.

I recommend not making the replications persistent. Post-beta-2 we've actually removed the concept of persistent replications. It was a feature that came from CouchDB, but it's not really useful in an embedded database and it makes the code and the API more complex.

- Even when I stop asynchronously the replication, it keeps working.

You may have multiple replications running. You used "exclusively:NO" so when you create the replications other pre-existing replications to different URLs won't be removed. If you've ever changed the remote URL, you may still have persistent replications left over that are still replicating with the old URL.

To make sure old persistent replications aren't left behind, I recommend deleting your app from the device/simulator, then running the app again. (After taking out the line that makes the replications persistent.)

- In the observer I log the completedChangesCount and changesCount of both pusher and puller and I always get a 0 in both properties.

You're observing the key path "completed" when the property name is "completedChangesCount", so you're not getting called when the property changes.

—Jens