Indicating Sync Progress in cordova apps

[Vinay Bhinde]

I have successfully established sync across android and ios devices running cordova apps with sync gateway. I am now facing few issues updating the UI when new change is pulled down by the app from SG.

I read about querying the _active_tasks api to get status of the replication in this post but i am not sure how do i pass the continuous parameter to _active_changes API. What i want is someway to fire a event or do something whenever my cblite receives something new from SG so that i can call my code which updates my UI only when it receives new documents from SG. I tried the below code but it does not work.

 coax([config.server, "_active_tasks"], { continous: true }, function(err, tasks)    {
       var me;
       for (var i = tasks.length - 1; i >= 0; i--)
       {
          if (tasks[i].task == id)
          {
             me = tasks[i];
          }
       }
       console.log(me);
    });

The logic i have with me should be run only when there is a new document inserted in cblite from SG and not when changes are made to local cblite.

Any ideas ?

[J. Chris Anderson]

Yes. Use the changes API, not _active_tasks, to get notified of data changes. Active tasks is about things like replication status or view building, the changes API can tell you exactly which documents changed.

Chris

[Vinay Bhinde]

_changes api will fire even if i make a change to my cblite locally (i.e nothing new is pulled from SG). My ui update code already is called when locally i add a document so if i use _changes api my code will be called twice (i.e once when locally doc is added on save button or something like that and second when _changes api is called).

 Also my UI update code changes from screen to screen so i cannot just have some specific code in _changes api response.

Any other way around to just get my function called whenever a pull operation is performed and for ex a new doc is inserted in cblite from SG ?

[Jens Alfke]

On Jan 1, 2015, at 5:46 AM, Vinay Bhinde <vin...@gmail.com> wrote:

_changes api will fire even if i make a change to my cblite locally (i.e nothing new is pulled from SG). My ui update code already is called when locally i add a document so if i use _changes api my code will be called twice (i.e once when locally doc is added on save button or something like that and second when _changes api is called).

It's often better to decouple the UI update from the input handling. So instead of redrawing the UI after the Save button is pressed, leave it for the _changes handler to do. That way your UI updates correctly in response to all database changes no matter where they come from.

 Also my UI update code changes from screen to screen so i cannot just have some specific code in _changes api response.

Sounds like you need an abstraction layer where the _changes listener can post events that are handled differently on different screens.

Any other way around to just get my function called whenever a pull operation is performed and for ex a new doc is inserted in cblite from SG ?

Not in the REST API. It derives from the CouchDB API, which doesn't draw a distinction: the replicator is just another client of the database that's making changes to it.

(In the native API there is a 'source' property on a DatabaseChanged notification that tells you if it came from a remote db.)

—Jens

[Vinay Bhinde]

OK I understand what you suggest. Thank you for the tip.

Also is there any way to know if a particular replication has been completed ? ie Suppose if i have 10 docs in SG and i use my app with on a new device with continuous replication on then my _changes will fire 10 times initially and it will call my UI update function 10 times which causes issues.

Initially it would be great if i could check if the pull replication of 10 docs has been finished..if yes then call my function...and not call the function on every document received.

Also i checked the native api docs and I found you provide status property to check if the replication has been completed. It says continuous replication never stops. So how do I wait for a continuous replication to complete before calling my function in cordova app ?

Do I need to query _active_tasks continuously for the status ? If yes then how ?

[Jens Alfke]

On Jan 3, 2015, at 9:43 PM, Vinay Bhinde <vin...@gmail.com> wrote:

Also is there any way to know if a particular replication has been completed ?

For a non-continuous replication, the POST to _replicate won't return until the replication completes. (This behavior changed; it didn't use to be this way in 1.0.)

For a continuous replication, GET _active_tasks and see if its "status" property is "Idle".

ie Suppose if i have 10 docs in SG and i use my app with on a new device with continuous replication on then my _changes will fire 10 times initially and it will call my UI update function 10 times which causes issues.

You can buffer up the changes, i.e. don't redraw immediately but wait like ¼ sec in case there are more notifications.

In general it's best to let your UI updates be driven by _changes, as I've said before.

Also i checked the native api docs and I found you provide status property to check if the replication has been completed. It says continuous replication never stops. So how do I wait for a continuous replication to complete before calling my function in cordova app ?

See above.

Do I need to query _active_tasks continuously for the status ? If yes then how ?

Um, start a timer and GET _active_tasks once a second? There's also a "?continuous=true" parameter you can add to it that acts like the continuous _changes feed.

(Hey, people-who-know-more-about-PhoneGap-than-I-do: Don't we have a sample app that shows this kind of stuff?)

—Jens

[Federico Hernandez]

There're two applications using phonegap and couchbase, but the synchronization process is not clear enough. It'd be great to know how the sync flow should be.

[Dominique Legault]

Federico, in the changes REST API there is a since parameter to get changes from a specific update sequence. when you get the db from the sync gateway it returns the current update_seq document like so :

http -auth username:password https://myurl.com:4984/testdb/
{
    "committed_update_seq": 3745,
    "compact_running": false,
    "db_name": "testdb",
    "disk_format_version": 0,
    "instance_start_time": 1420650337223095,
    "purge_seq": 0,
    "update_seq": 3745
}

then pass the update_seq number to the changes API as the since parameter and you will only get new changes.

I have made significant changes to the example phonegap app if your looking for another example of how things are done you can check out my app:

https://github.com/deefactorial/openmoney-mobile/

you can download the app from the google play store by joining my google group.
https://groups.google.com/forum/#!forum/openmoney-development

[Vinay Bhinde]

Um, start a timer and GET _active_tasks once a second? There's also a "?continuous=true" parameter you can add to it that acts like the continuous _changes feed.

—Jens

The API reference here does says that _active_tasks does not support any query parameters. So how i make it act like continous _changes feed.

[Jens Alfke]

The API reference is wrong. That parameter is nonstandard and didn't get documented. We need to fix that.

—Jens 

--
You received this message because you are subscribed to the Google Groups "Couchbase Mobile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mobile-couchba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/9d6f9855-7ec4-4b4f-9c9f-97253852e693%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Federico Hernandez]

Thanks for the example Dominique.

I've implemented the sync process based on your example. However, I still have some issues. I created two document and a user. The app sent the changes to the sync manager and then to the couchbase server. Now, I removed the app, installed it again and stopped the sync gateway. The app started up and I could see in the logs the changes coming from the couchbase server to the sync gateway, but the problem came up in this step. If I try to see the changes through the admin interface (SG) I can see them well. However, if I try to get them from the mobile app, I only get one of them, the one related to the user document.

Here I copy the logs from SG:

[I think the following is from the sync between couchbase and sync gateway]

15:11:28.786582 HTTP:  #012: GET /adb/_changes?feed=longpoll&heartbeat=1000&style=all_docs&since=0&limit=100&include_docs=true  (ADMIN)

2015/01/08 15:11:29 go-couchbase: call to ViewCustom("sync_gateway", "channels") in github.com/couchbaselabs/sync_gateway/base.couchbaseBucket.ViewCustom took 243.772833ms

15:11:29.030749 changes_view: Query took 243.893437ms to return 2 rows, options = db.Body{"limit":100, "stale":false, "startkey":[]interface {}{"*", 0x1}, "endkey":[]interface {}{"*", 0x4}}

15:11:29.035647 WARNING: Changes feed: error getting doc "_user/fede": 400 Invalid doc ID -- db.(*Database).addDocToChangeEntry() at changes.go:57

15:11:29.035713 HTTP+: #012:     --> 200 OK  (0.0 ms)

[The following shows the changes retrieved through the admin interface]

15:44:22.054865 HTTP:  #038: GET /vdb/_changes?feed=longpoll&heartbeat=1000&style=all_docs&since=0&limit=100&include_docs=true  (ADMIN)

15:44:22.138401 WARNING: Changes feed: error getting doc "_user/fede": 400 Invalid doc ID -- db.(*Database).addDocToChangeEntry() at changes.go:57

15:44:22.138459 HTTP+: #038:     --> 200 OK  (0.0 ms)

{

results: [

  {

    seq: 2,

    id: "B14D6B41-DB44-404E-A355-43DDB252A81E",

    doc: {

      _id: "B14D6B41-DB44-404E-A355-43DDB252A81E",

      _rev: "1-0c0ff608417c7a4748215a822b61d2ae",

      active: true,

      createdAt: "2015-01-08T15:19:24.629Z",

      id: "record-158",

      modifiedAt: "2015-01-08T15:19:24.629Z",

      name: "Thu Jan 08 2015 12:19:20",

 type: "event"

    },

    changes: [

    {

    rev: "1-0c0ff608417c7a4748215a822b61d2ae"

    }

    ]

  },

  {

    seq: 3,

    id: "C4315EBA-1C17-4136-AECE-0D1AF33C10BE",

    doc: {

      _id: "C4315EBA-1C17-4136-AECE-0D1AF33C10BE",

      _rev: "1-f2fc940cb0aab6289228e1ef9884ab3f",

      active: true,

      createdAt: "2015-01-08T15:19:42.406Z",

      id: "record-161",

      modifiedAt: "2015-01-08T15:19:42.406Z",

      name: "Thu Jan 08 2015 12:19:40",

      type: "event"

    },

    changes: [

      {

     rev: "1-f2fc940cb0aab6289228e1ef9884ab3f"

      }

    ]

  },

  {

    seq: 4,

    id: "_user/fede",

    changes: [

      {

        rev: ""

      }

    ]

  }

  ],

  last_seq: "4"

}

[The following shows the changes retrieved from the my app]

15:18:52.892483 HTTP:  #020: GET /adb/_changes?feed=longpoll&include_docs=true&conflicts=true&style=all_docs&since=1&limit=100

15:18:52.892575 HTTP: #020:     --> 401 Login required  (2.3 ms)

15:18:53.495125 HTTP:  #021: GET /adb/_changes?feed=longpoll&include_docs=true&conflicts=true&style=all_docs&since=1&limit=100  (as fede)

15:18:53.495422 WARNING: Changes feed: error getting doc "_user/fede": 400 Invalid doc ID -- db.(*Database).addDocToChangeEntry() at changes.go:57

15:18:53.495822 HTTP+: #021:     --> 200 OK  (0.0 ms)

{"results":[{"seq":4,"id":"_user/fede","changes":[]}],"last_seq":"4"}

As you can see, the changes attribute is an empty array here. Could you tell me why is that and how could I get the same changes that I get through the admin interface?

Thanks in advance,

[Federico Hernandez]

On Thursday, January 8, 2015 3:58:46 PM UTC-3, Federico Hernandez wrote:

On Wednesday, January 7, 2015 3:36:05 PM UTC-3, Dominique Legault wrote:

Federico, in the changes REST API there is a since parameter to get changes from a specific update sequence. when you get the db from the sync gateway it returns the current update_seq document like so :

[The following shows the changes retrieved from the my app]

15:18:52.892483 HTTP:  #020: GET /adb/_changes?feed=longpoll&include_docs=true&conflicts=true&style=all_docs&since=1&limit=100

15:18:52.892575 HTTP: #020:     --> 401 Login required  (2.3 ms)

15:18:53.495125 HTTP:  #021: GET /adb/_changes?feed=longpoll&include_docs=true&conflicts=true&style=all_docs&since=1&limit=100  (as fede)

15:18:53.495422 WARNING: Changes feed: error getting doc "_user/fede": 400 Invalid doc ID -- db.(*Database).addDocToChangeEntry() at changes.go:57

15:18:53.495822 HTTP+: #021:     --> 200 OK  (0.0 ms)

{"results":[{"seq":4,"id":"_user/fede","changes":[]}],"last_seq":"4"}

As you can see, the changes attribute is an empty array here. Could you tell me why is that and how could I get the same changes that I get through the admin interface?

Another question would be why are there two HTTP requests to get the changes? In fact, the app makes only one request to get the changes. If you see above, the first request fails with a 401 Login Required, but the second one works fine. Can the first one be related to the pullRep ?

Thanks in advance,

[Dominique]

what's your sync function look like, are sure that the user fede is subscribed to the channel the document is in ?

I also notice that you have since=1 on your changes feed for the user, that will exclude the first change.

--
You received this message because you are subscribed to a topic in the Google Groups "Couchbase Mobile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mobile-couchbase/GGHPkISNrGw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mobile-couchba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/bc4bf46f-bd2f-47fe-950d-f0dc570bde79%40googlegroups.com.

[Dominique]

There is also a warning about your user document that the changes feed is having a difficulty getting that document because : "Invalid doc ID" , how are you creating your user ?

[Federico Hernandez]

On Thursday, January 8, 2015 4:39:49 PM UTC-3, Dominique Legault wrote:

There is also a warning about your user document that the changes feed is having a difficulty getting that document because : "Invalid doc ID" , how are you creating your user ?

curl -X PUT http://localhost:4985/vdb/_user/fede --data \ '{"name": "fede", "email":"fe...@m.com", "channels":["*"], "role":"user", "password": "********"}'

[Federico Hernandez]

On Thursday, January 8, 2015 4:34:34 PM UTC-3, Dominique Legault wrote:

what's your sync function look like, are sure that the user fede is subscribed to the channel the document is in ?

I've used the one that comes as default:

function (doc) {

    channel(doc.channels);

}

 

I also notice that you have since=1 on your changes feed for the user, that will exclude the first change.

Yes, I wanted to exclude the 1st one.

[Dominique]

then in your document you have to have a channels property that assigns it to a channel.

for example.

doc: {

      active: true,

      createdAt: "2015-01-08T15:19:42.406Z",

      id: "record-161",

      modifiedAt: "2015-01-08T15:19:42.406Z",

      name: "Thu Jan 08 2015 12:19:40",

      type: "event",

      channels: "event"

    }

currently none of your docs are in any channels so you wouldn't get any.

To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/681501cb-4f98-48c6-9028-d7f1d141af7f%40googlegroups.com.

[Dominique]

On Thu, Jan 8, 2015 at 11:48 AM, Federico Hernandez <fede.he...@gmail.com> wrote:

On Thursday, January 8, 2015 4:39:49 PM UTC-3, Dominique Legault wrote:

There is also a warning about your user document that the changes feed is having a difficulty getting that document because : "Invalid doc ID" , how are you creating your user ?

curl -X PUT http://localhost:4985/vdb/_user/fede --data \ '{"name": "fede", "email":"fe...@m.com", "channels":["*"], "role":"user", "password": "********"}'

 

I have not seen the use of the channels property in that way before.

I don't see it on the documentation either
http://developer.couchbase.com/mobile/develop/guides/sync-gateway/administering-sync-gateway/authorizing-users/index.html#accounts

I explicitly set the channels a user has access to in the sync function.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/7c7a30cc-f6e3-4fdd-a8c3-800057a3a6ef%40googlegroups.com.

[Federico Hernandez]

On Thursday, January 8, 2015 at 5:01:13 PM UTC-3, Dominique Legault wrote:

On Thu, Jan 8, 2015 at 11:48 AM, Federico Hernandez <fede.he...@gmail.com> wrote:

On Thursday, January 8, 2015 4:39:49 PM UTC-3, Dominique Legault wrote:

There is also a warning about your user document that the changes feed is having a difficulty getting that document because : "Invalid doc ID" , how are you creating your user ?

curl -X PUT http://localhost:4985/vdb/_user/fede --data \ '{"name": "fede", "email":"fe...@m.com", "channels":["*"], "role":"user", "password": "********"}'

 

I have not seen the use of the channels property in that way before.

I don't see it on the documentation either
http://developer.couchbase.com/mobile/develop/guides/sync-gateway/administering-sync-gateway/authorizing-users/index.html#accounts

I explicitly set the channels a user has access to in the sync function.

 

Thanks for the answers Dominique. I had an issue with a wrong attribute name.

[Vinay Bhinde]

I am trying to query it in the following way but it doesn't work. Any help how do i query _active_tasks in cblite ?

coax([serverUrl, "_active_tasks",{continous:true} ], function(err, tasks) {

});

[Jens Alfke]

On Jan 20, 2015, at 4:37 AM, Vinay Bhinde <vin...@gmail.com> wrote:

coax([serverUrl, "_active_tasks",{continous:true} ], function(err, tasks) {

I don't know that API, but assuming that "{continuous:true}" specifies the URL query parameters, that's wrong. It should be "{feed:"continuous"}".

—Jens