ANN: Support for document encryption (iOS)

[Jens Alfke]

Alan McKean and I just implemented some hooks for transforming documents during replication. The intended use for this feature is to let apps implement server-side encryption of documents. An app would implement these hooks to transform a document into encrypted form during a push replication, and decrypt an encrypted document during a pull.

This commit does not implement encryption for you, just the minimal infrastructure needed for implementing it yourself.

Below is the new API — you’ll find it in CBLReplication.h on the master branch. iOS-only for now.

—Jens

typedef NSDictionary *(^CBLPropertiesTransformationBlock)(NSDictionary *);

/** Optional callback for transforming document bodies during replication; can be used to encrypt documents stored on the remote server, for example.

    In a push replication, the block is called with document properties from the local database, and the transformed properties are what will be uploaded to the server.

    In a pull replication, the block is called with document properties downloaded from the server, and the transformed properties are what will be stored in the local database.

    The block takes an NSDictionary containing the document's properties (including the "_id" and "_rev" metadata), and returns a dictionary of transformed properties. It may return the input dictionary if it has no changes to make. The transformed dictionary MUST preserve the values of "_id" and "_rev".

    The block will be called on the background replicator thread, NOT on the CBLReplication's thread! It must be written in thread-safe fashion. */

@property (strong) CBLPropertiesTransformationBlock propertiesTransformationBlock;

[Nate]

This looks great.  A couple of questions:

1. I saw an older commit to TouchDB where you added support for the data protection APIs.  I still see some references to this in the current codebase, but I'd like to confirm that Couchbase Lite is still encrypting local data where available.

2. This seems like an easy way to process documents on sync even if they aren't being transformed.  As an example, I need to check documents for changes after a pull replication and update data outside of CBL if the documents have been modified.  Is there a better way to handle this with CBL (something like the changes feed)?

Thanks!

[Jens Alfke]

On Mar 2, 2014, at 3:18 PM, Nate <na...@pixelauthorityllc.com> wrote:

1. I saw an older commit to TouchDB where you added support for the data protection APIs.  I still see some references to this in the current codebase, but I'd like to confirm that Couchbase Lite is still encrypting local data where available.

All files on iOS are encrypted; there are just different levels of security of the keys. The Apple white paper I posted a link to a few days ago goes into a lot of detail; if you’re interested in file security it’s a must-read.

In CBLDatabase+Internal.m:

    int flags =  SQLITE_OPEN_FILEPROTECTION_COMPLETEUNLESSOPEN;

and CBLBlobStore has several places it applies the NSDataWritingFileProtectionCompleteUnlessOpen attribute to attachment files.

2. This seems like an easy way to process documents on sync even if they aren't being transformed.  As an example, I need to check documents for changes after a pull replication and update data outside of CBL if the documents have been modified.  

Hm, that’s really not the intention of this API, and I wouldn’t use it for any operation with external side effects because it’s called by the puller before the documents are added to the database, so there’s a chance they might not get added.

Is there a better way to handle this with CBL (something like the changes feed)?

There’s no native access to the changes feed, but you can create a view that emits the document’s sequence number as the key, and use that similarly. The sequence number is available to a map function as the “_local_seq” property of the document dictionary.

—Jens

[Jens Alfke]

On Mar 3, 2014, at 9:02 AM, Nate <na...@pixelauthorityllc.com> wrote:

I'm getting null for the _local_seq property in a very simple test view (running beta 2).

Oops, forgot to say this was added after beta 2! You’ll need to build & run the ‘master’ branch. (Or, you could check out the 1.0-beta2 tag and then cherry-pick the very small commit 78307ef92c2d.)

—Jens

[Nate]

Oops, forgot to say this was added after beta 2! You’ll need to build & run the ‘master’ branch. (Or, you could check out the 1.0-beta2 tag and then cherry-pick the very small commit 78307ef92c2d.)

Ah, thanks.  Guess this raises a new question: any plans for beta 3?

[Jens Alfke]

On Mar 4, 2014, at 1:42 PM, Nate <na...@pixelauthorityllc.com> wrote:

Ah, thanks.  Guess this raises a new question: any plans for beta 3?

Coincidentally I just brought this up yesterday with some other people on the team.  The 1.0 deadline has moved out a bit for various reasons, so there’s enough time to put out another beta. And a lot has changed since beta 2, so it would be good to give you more of a chance to use it and perhaps find a bug or two. More news soon.

—Jens