Question about Todo sample app and user profile synchronization

[Christoph Berlin]

I am not sure how many people are familiar with the great todo sample app however due to the minimalistic documentation approach it serves as a bible to me right now understanding the joint work of Mobile and Sync Gateway.

However I would have a question about the flow and whether I just don't get it or there is a logic issue within the app.

When looking at the sample I noticed that a profile document gets created as part of the loginWithFacebook function. That works well when the user truly doesn’t exist in the database but what happens when the user logs back in and already has a profile on the remote database? For example I use the sample app on my iPhone, sign up (profile document gets created and synced) -> I also install the app on a a different device and log back in. It is my current understanding that the profile documents gets created again, is that correct? But it doesn’t seem right or how are the two documents with the same document name handled? Do they update incrementally?

Asked a bit differently, our solution requires a profile document however when the user logs in from a device, we want to check whether a profile document already exists, sync first and then use it or create one if it doesn’t exist already? 

Can anyone shed some light on that? Am I missing something?

- (void)loginWithFacebookUserInfo:(NSDictionary *)info accessTokenData:(FBAccessTokenData *)tokenData { NSAssert(tokenData, @"Facebook Access Token Data is nil"); NSString *userId = [info objectForKey:@"email"]; NSString *name = [info objectForKey:@"name"]; [self setCurrentUserId:userId]; CBLDatabase *database = [self databaseForUser:userId]; [self setCurrentDatabase:database]; [self setGuestLoggedIn:NO]; Profile *profile = [Profile profileInDatabase:database forUserID:userId]; if (!profile) { profile = [[Profile alloc] initProfileInDatabase:self.database withName:name andUserID:userId]; NSError *error; if ([profile save:&error]) { [self migrateGuestDataToUser:profile]; } else { NSLog(@"Cannot create a new user profile : %@", [error description]); [self showMessage:@"Cannot create a new user profile" withTitle:@"Error"]; } } if (profile) { [self startReplicationWithFacebookAccessToken:tokenData.accessToken]; } }

[Traun Leyden]

Hey Christoph,

 

When looking at the sample I noticed that a profile document gets created as part of the loginWithFacebook function. That works well when the user truly doesn’t exist in the database but what happens when the user logs back in and already has a profile on the remote database?

Yes, the system can create a conflict here.  However -- and this is a key point in the Couchbase Lite design -- the system will deterministically pick a winner from those conflicting revisions, and continue on using the winning revision.  

One key point here is the fact that the winning revision of a conflict is chosen deterministically.  When I first started wrapping my head around Couchbase Lite, this was a huge "aha moment" for me.  What it means is that there is an algorithm that specifies how the the winning revision of a conflict will be chosen, and all nodes in the system apply the same algorithm, and will therefore get the same result.

So to make it more concrete ..

Suppose you have two devices: 

iphone

ipad

and suppose on the iphone, a revision is created:

doc id: christop...@foo.com

rev id: 1-324

and on the ipad:

doc id: christop...@foo.com

rev id: 1-989

after these devices sync, you end up with this on both the iphone and ipad:

 

It's key to realize that this will be the state of affairs on both devices after sync -- due to the fact it's a deterministic algorithm, we know that both devices will have picked the same winning revision.

What this means is:

* If you call getDocument("christop...@foo.com").getCurrentRevisionID() on either  device, you will get rev 1-929.

* If you add a new revision to the doc "christop...@foo.com" on the ipad, say 2-455, that will happen on top of 1-929, so your rev history will be [2-455, 1-929]

So revision 1-324 will still be there, but will essentially be a dead branch on the revision tree and ignored.

At this point, you might be thinking .. don't I have to resolve this conflict?  Don't I have to clean up this mess?

Actually, you can, and you should -- but you don't have to.  The system will happily continue with a conflict in its revision tree, and it won't mess anything up.  However, you should clean them up if possible, since it will waste some space if you don't.

You also might be wondering, but what if rev 1-324 had the user's website address in the profile and 1-929 did not, is that information going to be lost?  Nope, it won't be lost, but it will remain hidden indefinitely, so your user will effectively never see it.  To make sure that information makes it into your app, you'll need to write some code that looks for conflicts, and does some application specific merging to merge the information from the deleted revision.

We have some examples of that I can dig up if you need it.

Asked a bit differently, our solution requires a profile document however when the user logs in from a device, we want to check whether a profile document already exists, sync first and then use it or create one if it doesn’t exist already? 

Well, if you take the approach used in TodoLite and just allow a potential conflict, you can worry less about having to make sure the profile already exists somewhere else or not.

Depending on your business requirements, you may need to actively handle those conflicts and have business specific merging rules.

[Christoph Berlin]

Traun,

Wow - thanks much! As always you provided incredible insight and it helps tremendously.

Christoph

[Christoph Berlin]

Traun,

I actually would have one more question if you don't mind. Looking at your explanation it makes total sense but what is the difference compared to trying to create a new document with the same document ID?

Based on your example I understand it flows like this:

1) iPad creates user_profile document offline at login and then syncs to database

2) iPhone creates user_profile document offline at login and then syncs to database as well.

If I understand you correctly both documents get created but the version with the higher revision wins, correct?

However lets assume a slightly different scenario:

1) iPad creates user_profile document offline at login and then syncs to database

2) iPhone is online and synced. When I try to create a NEW document with the ID (same step as creating a user profile offline) I get an error that the document ID already exists, correct?

My question would be whether it is save to assume that I could technically create two documents with the same ID on two different offline devices and Sync Gateway will try to clean up the mess? If so how do I protect my data logic from running into that problem? Meaning there is a reason that I usually cannot create the same document ID twice but in this scenario it bypasses that check.

Thanks Christoph

[Jens Alfke]

> On Oct 23, 2014, at 8:26 AM, Christoph Berlin <appm...@gmail.com> wrote:
>
> I also install the app on a a different device and log back in. It is my current understanding that the profile documents gets created again, is that correct? But it doesn’t seem right or how are the two documents with the same document name handled? Do they update incrementally?

It's considered a conflict, just a special case where there is no common ancestor.

> Asked a bit differently, our solution requires a profile document however when the user logs in from a device, we want to check whether a profile document already exists, sync first and then use it or create one if it doesn’t exist already?

If the profile doc will have the exact same contents no matter where it's created, then you can just create it immediately; there won't be a conflict because the revision IDs will be identical. But probably a lot of the time that won't be the case, e.g. you might store something like a creation date that won't be the same on every device.

In the general case, you'll need to run a pull replication first before trying to create the profile document, to determine whether it already exists. Or instead, you could write custom code to send a GET to the server to look for the document.

—Jens

[Christoph Berlin]

Jens,

perfect, thanks much!

Christoph

[Jens Alfke]

On Oct 23, 2014, at 2:12 PM, Christoph Berlin <appm...@gmail.com> wrote:

My question would be whether it is save to assume that I could technically create two documents with the same ID on two different offline devices and Sync Gateway will try to clean up the mess?

Yes. Actually it's not the Gateway doing anything special. It's just that, in any database, an attempt to get the current contents of a conflicted document follows the same algorithm where one revision is picked as the winner. Once both devices are in sync, they have the same revision history of that document so they'll pick the same revision as the current one.

If so how do I protect my data logic from running into that problem? Meaning there is a reason that I usually cannot create the same document ID twice but in this scenario it bypasses that check.

The client API tries to prevent conflicts when it can, by returning a 409 error if your change would create a conflict with another revision in the local database.

But in the general case, conflicts can't be prevented because this is a distributed system. (In terms of the CAP theorem, we've sacrificed Consistency.)

So there are two different manifestations of conflict handling — one where you get an error before the conflict would happen, and another where the system tries to deal with a conflict that already exists.

—Jens

[Traun Leyden]

On Thu, Oct 23, 2014 at 2:12 PM, Christoph Berlin <appm...@gmail.com> wrote:

If I understand you correctly both documents get created but the version with the higher revision wins, correct?

Yes, as Jens mentioned it's a special case of a conflict where there is no common parent.  Two branches, each having it's own root.

The rule for determining the winning conflict, in the case that there are no branches with delete revisions and the branches are of the same length, is a lexicographic comparison between the digest parts, and the lower one wins.

Eg if your conflicting revs were:

1-a342

1-z343

Since "a342" has lower sort order than "z343", then 1-a342 would be the chosen winning revision.

There are some more complicated rules if you have deletion revisions on any of the branches, or the branches are of different length.

 

However lets assume a slightly different scenario:

1) iPad creates user_profile document offline at login and then syncs to database

2) iPhone is online and synced. When I try to create a NEW document with the ID (same step as creating a user profile offline) I get an error that the document ID already exists, correct?

Yup, it will reject it with a 409 error.

 

My question would be whether it is save to assume that I could technically create two documents with the same ID on two different offline devices and Sync Gateway will try to clean up the mess?

Yeah as Jens mentioned, it's not really Sync Gateway cleaning up any mess, it's more that the distributed algorithm takes this scenario into account, and deals with it gracefully on all parts of the system (each Couchbase Lite node and the Sync Gateway node).  

So if you create two docs with the same ID on two different offline devices, once they go online and reconcile, one of those docs will be considered the winning revision and the other will be considered the losing revision, and all future edits will happen on the winning revision.  The behavior will be consistent across all nodes in the system.  (eg, each node will pick the same winning revision)

 

If so how do I protect my data logic from running into that problem? Meaning there is a reason that I usually cannot create the same document ID twice but in this scenario it bypasses that check.

I would argue that for some business requirements, its not actually a problem.  In TodoLite, you can create two profiles with the same email on two different devices while they are offline, and it doesn't create any problems.  Sure, there might be conflicting revisions within that profile doc once the devices sync, but in and of itself doesn't create a "problem".  I mean, it doesn't break anything.  As I said before, if you don't clean up the conflict, it will just use a bit more storage than it otherwise would.

But, if your business requirements consider it a problem, then you could do what Jens mentioned and first issue a GET to the Sync Gateway and see if that doc already exists.  Of course, that puts the network back into the critical path for your app for that operation, so you might get some extra user attrition for impatient users on slow networks.  So there's a tradeoff.

[Christoph Berlin]

Thanks guys, much appreciated it!

On Thursday, October 23, 2014 8:26:09 AM UTC-7, Christoph Berlin wrote: