Announcing Couchbase Mobile 2.0 Beta 2 (DB 23)

[Priya Rajagopal]

Happy to announce the release of Couchbase Mobile 2.0 Beta 2 which includes Couchbase Lite 2.0 Beta 2 and Sync Gateway 2.0 Beta 2. NOTE that Couchbase Lite 2.0 Beta 2 clients require Sync Gateway 2.0 Beta 2. Please refer to the [Compatibility matrix ] for details. 

You can download the latest Beta Build from our Downloads page under “Pre-Release” versions : https://www.couchbase.com/downloads

The second beta release includes following major changes - 

- Updates to behavior of SaveDocument API and automatic conflict resolution policy. (Related [blog] ).

- Encryption is moved out of 2.0 release

- Replication bug fixes and optimizations

- General performance improvements

We have two versions of the build now - Community Edition (CE) and an Enterprise Edition (EE). Both editions are free to use for development purposes. Enterprise edition has some stipulations when you go production. 

- This is the [CE License]

- This is the [EE License]

The Couchbase Lite 2.0 Beta  release is available for iOS(Swift, ObjC), .NET (UWP, Xamarin) and Android platforms . Xamarin will a CE only release in 2.0. 

**Links to platform specific release notes -**

- [Sync Gateway 2.0]

- [Swift/ObjC]

- [.Net]

- [Android]

**Related  Blogs :**

- [Introducing Queries in Couchbase Lite 2.0]

- [Introducing Full Text Search in Couchbase Lite 2.0]

- [Querying Array Collections in Lite]

- [Replication in Couchbase Mobile 2.0]

- [Automatic Conflict Resolution in Couchbase Mobile 2.0]

- [Join Queries in Couchbase Lite 2.0]

- [Certificate Pinning in Android with Couchbase Mobile 2.0]

**Sample Apps :**

- ToDo App : https://github.com/couchbaselabs/mobile-training-todo/tree/feature/2.0

- Travel Sample App : https://github.com/couchbaselabs/mobile-travel-sample

**About Couchbase Mobile 2.0:**

Couchbase Mobile  2.0 is a groundbreaking new release for Couchbase Mobile. Key enhancements include a cross-platform common core, new and improved Query API with Full-Text search capabilities, automatic conflict resolution and a new web sockets based replication protocol 

**About Beta Builds:**

Beta release is a way for you to test the latest functionality of a release before it is generally available. These will eventually become production releases with full support when they are stable and features are complete.

[Ben Kennedy]

I am extremely concerned about the changes to conflict resolution in 2.0, as elucidated in the linked blog post.

In short, my understanding is that there is no longer any way for the client to deal with conflicts arising from a pull replication (where a new local revision conflicts with one delivered by Sync Gateway). This seems like it will promise ongoing non-deterministic and silent data loss when operating offline (or, indeed, when racing against other mobile clients speaking to Sync Gateway). Really?

This issue is serious enough I wanted to raise it for clarification here (despite having also posted a reply on the blog post). 

-ben

[Traun Leyden]

Hey Ben,

You are correct in your assumptions that all conflict resolution in 2.0 will be wallclock-based "last write wins", since there is no longer an API to plugin custom conflict resolvers.  (previous 2.0 beta versions did have this feature)

 

I can give you some insight into the thinking of the product management at Couchbase on this issue.  Basically there appears to be a very small minority of users that will actually need to implement custom conflict resolvers based on their use case, and including this into the API raises the complexity quite a bit.  So the thinking was to rip this out in 2.0, but if enough users complain, re-add it in 2.1.   

I'm not in product management at Couchbase, but I'll make sure they see your feedback.  If you have a channel with Couchbase sales or support, I'd definitely use that channel as well to try to influence the product direction. 

Also, if you are able to post details on your particular use case that highlights the need for custom conflict resolvers, that will help make a case for re-adding it.

Thanks again for posting!

[Ben Kennedy]

> On Mar 21, 2018, at 10:55 AM, Traun Leyden <traun....@gmail.com> wrote:
>
> You are correct in your assumptions that all conflict resolution in 2.0 will be wallclock-based "last write wins", since there is no longer an API to plugin custom conflict resolvers. (previous 2.0 beta versions did have this feature)

What prompted its removal?

> Basically there appears to be a very small minority of users that will actually need to implement custom conflict resolvers based on their use case, and including this into the API raises the complexity quite a bit.

What sort of complexity?

We're still building with DB021 at the moment (due to the need to target Sync Gateway 1.5 since we're not yet in a position to deploy SG beta to our server environment), but there it's been possible to register a conflict resolver with the replicator. I presume that if/when a conflicting rev comes down in a pull, the resolver would be called, and I could return an appropriate rev (be it A, B, or a new merged C).

Granted, our development on this app is still in early enough stages that we haven't gotten to the point of implementing or exercising conflict resolution logic yet, but the principle seemed straightforward. I wonder what I'm missing.

> Also, if you are able to post details on your particular use case that highlights the need for custom conflict resolvers, that will help make a case for re-adding it.

Priya wrote me directly asking for a similar case, so I'll reiterate here the explanation I gave her, for the benefit of you and others:

Our application (iOS, Android, and web) services financial accounting for small-business customers. The main area under development is mobile capture and classification of financial documents (e.g. expense receipts, tax forms) both in the field and at a computer.

Typically, one or more users might be making different edits to a particular document at any given time. For example, a person might purchase some supplies at a shop and file the expense immediately from her phone. An hour later while at a coffee shop she might supplement the record with some more details (e.g. notes on items purchased). Meanwhile, in the interim or afterwards, a data-entry clerk might transcribe some fields from the attached image into well-formed data fields (e.g. price, date, invoice number). An accountant back at the office might start to act on the expense and enter it into the general ledger. A project manager might also concurrently assign some of the bought items to a project.

All of these actions would involve making changes (additions, edits, deletions) to a variety of fields in a particular Couchbase document that embodies the purchase.

In several of our document types, for auditing purposes, changes and additions made by various users are added to an "annotations" array. At any given time the app's business logic examines the array in order to create a current picture of the represented data. This array is expected to be appended to at random.

As you should now intuit, it is crucial that additions to the "annotations" array be coalesced from sibling revisions in the face of a conflict, rather than one particular set chosen at random.

Couchbase Lite is extremely well-suited to our application both for its use of non-structured documents and its offline performance. However, a lack of control over conflict resolution makes offline use dangerous and unreliable; even online use would become unpredictable if more than a single client expects to manipulate the same document (via SG).

cheers,

-ben

[Traun Leyden]

On Wed, Mar 21, 2018 at 12:02 PM, Ben Kennedy <ben.k...@kashoo.com> wrote:

> On Mar 21, 2018, at 10:55 AM, Traun Leyden <traun....@gmail.com> wrote:
>
> You are correct in your assumptions that all conflict resolution in 2.0 will be wallclock-based "last write wins", since there is no longer an API to plugin custom conflict resolvers.  (previous 2.0 beta versions did have this feature)

What prompted its removal?

Sorry but I don't know the exact reason.  It may have been related to assessing the testing/documentation effort.  

> Basically there appears to be a very small minority of users that will actually need to implement custom conflict resolvers based on their use case, and including this into the API raises the complexity quite a bit.

What sort of complexity?

Maybe "surface area" is a better way to put it.  Any feature has to be thoroughly tested and documented.  Rather than delay the release, sometimes features need to be removed and postponed until (potential) future releases.

 

We're still building with DB021 at the moment (due to the need to target Sync Gateway 1.5 since we're not yet in a position to deploy SG beta to our server environment), but there it's been possible to register a conflict resolver with the replicator. I presume that if/when a conflicting rev comes down in a pull, the resolver would be called, and I could return an appropriate rev (be it A, B, or a new merged C).

Granted, our development on this app is still in early enough stages that we haven't gotten to the point of implementing or exercising conflict resolution logic yet, but the principle seemed straightforward. I wonder what I'm missing.

I don't think you're missing anything, and yes the principle is straightforward.  

 

> Also, if you are able to post details on your particular use case that highlights the need for custom conflict resolvers, that will help make a case for re-adding it.

Priya wrote me directly asking for a similar case, so I'll reiterate here the explanation I gave her, for the benefit of you and others:

Our application (iOS, Android, and web) services financial accounting for small-business customers. The main area under development is mobile capture and classification of financial documents (e.g. expense receipts, tax forms) both in the field and at a computer.

Typically, one or more users might be making different edits to a particular document at any given time. For example, a person might purchase some supplies at a shop and file the expense immediately from her phone. An hour later while at a coffee shop she might supplement the record with some more details (e.g. notes on items purchased). Meanwhile, in the interim or afterwards, a data-entry clerk might transcribe some fields from the attached image into well-formed data fields (e.g. price, date, invoice number). An accountant back at the office might start to act on the expense and enter it into the general ledger. A project manager might also concurrently assign some of the bought items to a project.

All of these actions would involve making changes (additions, edits, deletions) to a variety of fields in a particular Couchbase document that embodies the purchase.

In several of our document types, for auditing purposes, changes and additions made by various users are added to an "annotations" array. At any given time the app's business logic examines the array in order to create a current picture of the represented data. This array is expected to be appended to at random.

As you should now intuit, it is crucial that additions to the "annotations" array be coalesced from sibling revisions in the face of a conflict, rather than one particular set chosen at random.

Couchbase Lite is extremely well-suited to our application both for its use of non-structured documents and its offline performance. However, a lack of control over conflict resolution makes offline use dangerous and unreliable; even online use would become unpredictable if more than a single client expects to manipulate the same document (via SG).

Ok thanks. If you have a hard requirement to keep the annotations together in a single document, this seems like a perfectly reasonable case where you'd need to merge the annotations w/ some custom business logic.

 

cheers,

-ben

[Ben Kennedy]

Congratulations to the CBL team on the GA release of 2.0!

Unless I have missed it, nothing further has been said on the subject of this replication design flaw. Given Couchbase [Lite]'s reputation for data integrity, and considering the expectations of those planning to migrate from 1.x, I am surprised that there isn't any allusion or mention of this in the release notes or getting-started docs. (The latter even includes a list of “bugs” and “known issues” at the bottom; wouldn't this type of thing warrant an entry?)

-b

[Priya Rajagopal]

Hi Ben

The automatic conflict resolution process in 2.0 was designed as such based on feedback we received from a number of our customers on 1.x. Hence, it is not listed as a bug or as a known issue because it is "working as designed" !

That said, we understand that the automatic conflict resolution may not be applicable in some instances. To that end, we are are consolidating use-case feedback from users such as yourself to determine the optimal way to design the system. So this would be a future enhancement - it wouldn’t make the 2.1 release (in couple months) but stay tuned .

Again, appreciate your feedback and details on your use case.

-Priya