Number of documents replicated less than those reported by Sync Gateway

[Adam Wilson]

Following on from https://forums.couchbase.com/t/sync-replication-progress/14987

I have a view on Sync Gateway that counts the number of documents in a view. This view is basically the same as a channel I have defined. I have also defined the same view in Couchbase Lite (iOS, v1.4). I am monitoring progress of very large pull replications by first calling the SG view, and then calling the same view on CBL whenever there is a pull event, to update a progress meter. 

I am seeing slight differences in the amount of documents reported by SG and those reported when the replication completes. e.g. 36711 from SG and 36691 from CBL, another example: 36700 reported by SG, 36697 by CBL. It seems to be slightly different each time. 

I'm wondering if this is an issue with the replicator not getting all the documents? Or maybe the SG view counting deleted documents? 

The view:

    "wordlist_count": {
      "map": "function(doc) { if (doc.wordlist)  emit(doc.wordlist, null) }",
      "reduce" : "_count"
    }

[Traun Leyden]

It's hard to speculate without knowing more about your setup.  For example, depending on your sync function, certain docs might not replicate down to the device, but still be available via the SG view.

Does it appear sporadic or is it the same behavior each time?

Also, you might try getting the doc id's, sorting them, and then diffing them to find the "missing" docs, which might provide some clues.

[Adam Wilson]

Thanks, thats a good idea to diff for the missing ones. I just wondered if its behaviour thats been seen before. 

The sync function:

function(doc) {
  if (doc.category == "free") channel("free")
  channel(doc.wordlist);  // This is the part that I expect to replicate the same docs as the SG view.
}

To answer your other question, the number of missing docs *seems* to be different each time. More investigation needed..