Multiple deletes of the same document possible?

[Brendan Duddridge]

Hi,

A customer of mine told me that after a sync, one of their CBL documents disappeared. Wondering how that could happen, they sent me their database and I noticed that there appeared to be multiple current and deleted revisions to the missing CBL document. For each one marked as current, it was also marked as deleted.

I've attached an image showing the revisions sorted by the "current" flag.

These are all with the same doc_id 319. You can see the revID values with the discontiguous numbers before the dash. It seems to me that it was deleted, modified, then deleted and modified a bunch more times until it was deleted for the last time in revision 234.

I'm wondering if there's another explanation for how this can happen other than the customer explicitly deleting the document?

The weird thing is the document still existed in the database on their Mac, but not on the iOS device.

It was no longer in the maps_1 table (the CBLView for this data), but it still exists in the revs and the docs tables in the SQLite database. 

Thanks,

Brendan

[Jens Alfke]

This looks like the result of a subtle-but-known bug in revision pruning that hasn't been properly fixed yet in 1.x. A resolved conflict can get its history pruned away over time, which leads to a false conflict appearing, which leads to the app creating another revision to resolve it, and then that can later end up as an other false conflict.

I'm not involved in 1.x development anymore, so I don't know what the prognosis is for fixing this bug. There is a fix in .NET, where it was originally reported,  but it hasn't been ported to the other platforms yet (here's the corresponding open iOS/Mac issue.) I suggest commenting there if getting this fixed is important to you.

—Jens

[Brendan Duddridge]

Hi Jens,

Thanks for the quick reply.

On Tuesday, August 22, 2017 at 10:01:17 PM UTC-6, Jens Alfke wrote:

This looks like the result of a subtle-but-known bug in revision pruning that hasn't been properly fixed yet in 1.x. A resolved conflict can get its history pruned away over time, which leads to a false conflict appearing, which leads to the app creating another revision to resolve it, and then that can later end up as an other false conflict.

I'm not involved in 1.x development anymore, so I don't know what the prognosis is for fixing this bug. There is a fix in .NET, where it was originally reported,  but it hasn't been ported to the other platforms yet (here's the corresponding open iOS/Mac issue.) I suggest commenting there if getting this fixed is important to you.

Is there a workaround for this bug until whomever can get a fix in for iOS? Should I increase the maxRevTreeDepth? I just have it at the default.

Thanks,

Brendan

[Brendan Duddridge]

Hi Jens,

I just looked at the fix for the .Net implementation and it appears to just be a slight modification to the SQL for pruning.

Would it be appropriate then to modify the CBL_SQLiteStorage.m file around line 2261 and change the SQL to the following?

// There are no branches, so just delete everything below minGenToKeep:

if (![_fmdb executeUpdate: @"DELETE FROM revs WHERE doc_id=? AND revid < ? AND current=0 AND sequence NOT IN (SELECT parent FROM revs WHERE doc_id=? AND current=1)", @(docNumericID), $sprintf(@"%u-", minGenToKeep), @(docNumericID)]) {

The new bit seemed to be this part:

 AND sequence NOT IN (SELECT parent FROM revs WHERE doc_id=? AND current=1)

would that hurt anything to make that change?

Thanks,

Brendan

[Jens Alfke]

> On Aug 22, 2017, at 10:51 PM, Brendan Duddridge <bren...@gmail.com> wrote:
>
> Hi Jens,
>
> I just looked at the fix for the .Net implementation and it appears to just be a slight modification to the SQL for pruning.
> Would it be appropriate then to modify the CBL_SQLiteStorage.m file around line 2261 and change the SQL to the following?

Pasin and I just looked at the Obj-C pruning code, and it looks like it works correctly already. I haven't studied the C# code, but I think it's based on an earlier version of the pruning algorithm, and Jim's fix there there doesn't apply to the iOS/Mac codebase.*

(What that code you're looking at is doing: If the document history is linear, it just deletes all revisions below a minimum generation #. Otherwise it fetches the topology of the tree (every revisions sequence and parent sequence), finds the revisions that aren't near enough to a leaf, and deletes them.)

Sorry to send you on a wild goose chase!
And now I don't understand what's going on with that database. :(

—Jens

* This is a perfect example of why we're moving to common core code in 2.0!

[Brendan Duddridge]

Sorry to send you on a wild goose chase!

No worries. It's good to explore all avenues.

 

And now I don't understand what's going on with that database. :(

Me neither. Maybe because I use putExistingRevision heavily? And perhaps the revision histories are getting messed up causing this problem? I thought about just creating new revisions when syncing instead of using putExistingRevision, but I didn't know how to avoid a sync loop. And in this case, by syncing, I'm referring to my CloudKit sync implementation, not the native CBL syncing. I use putExistingRevision to facilitate it.

Fortunately my customer said his form re-appeared after syncing from another device that still had it.

 

* This is a perfect example of why we're moving to common core code in 2.0!

And unfortunately I'm scared I'm going to be left behind until someone provides a DBAAS like Cloudant that I can redirect my customers to.