Random documents being removed from our users change feeds [1.0.4] CBL + SG

[Alexander Selling]

Hi!

This happens all day long for us but so far I've been able to mitigate it by routinely updating revs of the documents when needed.

But lately it's been getting worse and it's getting harder to predict which documents will get randomly removed from the change feed.. any tips on where I should get started in order to re-produce this so that you guys can solve it?

I've uploaded a text file(A change feed for one of my users) where you can see that a lot of documents get "removed" and what they all have in common(the removed docs) is that the sequence number is divided up into two parts: "295132:207890"

https://slack-files.com/T03G71GP5-F046J6S3E-dac6597a43

Our logs are filled with these warnings:

08:39:38.824109 WARNING: changeCache: Giving up, accepting #5697162 even though #5697161 is missing -- db.(*changeCache)._addPendingLogs() at change_cache.go:320
08:40:08.824279 WARNING: changeCache: Giving up, accepting #5697171 even though #5697170 is missing -- db.(*changeCache)._addPendingLogs() at change_cache.go:320

Thanks in advance!

[Andrew Reslan]

Which versions of Couchbase Lite and Sync Gateway do you have deployed

Andy

[Alexander Selling]

Like I stated in the subject line I'm using 1.0.4 for both CBL and SG

[ajres]

For the second part of the issue you are seeing with the log warnings:

08:39:38.824109 WARNING: changeCache: Giving up, accepting #5697162 even though #5697161 is missing -- db.(*changeCache)._addPendingLogs() at change_cache.go:320

This has been resolved on the Sync Gateway git repo master branch via issue #525

https://github.com/couchbase/sync_gateway

The issue with the deleted revisions is not clear.

From the logs, these appear to be valid removals.

The first few log entries are associated with channel:

"removed":["137bdf30cfe2c88af1bcb7976c82667c_project"]

Their removal is related to the change that took place at sequence : 295132

That does not appear later in the changes feed so I can't tell what it is.

Are you able to manually get the changes feed from the ADMIN port starting at sequence 295132

The following curl should work:

curl -k -X GET https://localhost:4985/kitchen-sync/_changes?since=295131

Please then paste the output for sequence 295132 here.

Andy