Xcode 8 compiled app no longer sees encryption with CBL on iOS (9 and 10)

[Brendan Duddridge]

Hi,

There was some discussion in the following GitHub issue about SQLCipher on macOS Sierra not being recognized, but that issue is closed now.

https://github.com/couchbase/couchbase-lite-ios/issues/1406

Might be something similar with iOS too.

With Xcode 8 GM seed, I get an error that SQLCipher is not implemented when I run my app and try to access an encrypted database.

Database: Opening <CBL_SQLiteStorage: 0x171268a00>
Database: Open /private/var/mobile/Containers/Data/Application/DBCF0DB2-7759-4247-AA67-664154FF570A/Documents/My New Database.tapforms/db-190076733d434f5bbdd11dc9eb33e2f6.cblite2/db.sqlite3 with SQLite (flags=6, encryption key given)
WARNING: CBL_SQLiteStorage: encryption not available (app not built with SQLCipher) {at -[CBL_SQLiteStorage decryptWithKey:error:]:267}
Error Domain=CBLHTTP Code=501 "unimplemented" UserInfo={NSLocalizedFailureReason=unimplemented, NSLocalizedDescription=unimplemented}

I made no changes to my project. It works fine if I run it from Xcode 7, but not when run from Xcode 8. I'm currently using macOS 10.11.6 for development and the problem happens on iOS 9 and iOS 10 GM. So it seems like it's more likely an Xcode 8 issue rather than an OS level issue.

Not sure why it's not finding SQLCipher with an Xcode 8 compiled app though. I have libsqlcipher.a included in the `Link Binary with Libraries` phase.

It seems like it may be the way that Xcode 8 is looking for static libraries perhaps during runtime. There are no compile errors.

The libsqlcipher.a static library was compiled from the Couchbase SQLCipher github source using the Xcode 7 command-line tools.

Has anyone else seen this issue or is there a different way that Xcode 8 handles library search paths? Or maybe the Runpath Search Paths needs to be different somehow? I currently have it set to: $(inherited) @executable_path/Frameworks

Thanks,

Brendan

[Jens Alfke]

This is weird! I don't think it's related to the Github issue you linked to (#1406) — that one's kind of the reverse, where the built-in SQLite library claims to support encryption but doesn't.

It definitely sounds like something changed in Xcode 8 such that your app ends up linking with the system libsqlite3.dylib instead of your static libsqlcipher.a. But I don't know how that could happen, assuming your target links with the latter and not with the former.

Try running "otool -L" on your application binary: that should list the dynamic libraries that the binary imports. If "libsqlite3.dylib" is in there, that would be the problem; then we have to figure out why Xcode ended up doing that.

—Jens

[Brendan Duddridge]

Well there you go! That's definitely the problem then.

brendan$ otool -L Tap\ Forms 

/Tap Forms.app/Tap Forms:

/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 307.4.0)

/usr/lib/libsqlite3.dylib (compatibility version 9.0.0, current version 252.0.0)

/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)

I know for a fact that I have libsqlcipher.a in my Link Binary with Libraries build phase. And I do NOT have libsqlite3.dylib linked anywhere.

A bit of a mystery, but this is definitely progress.

[Brendan Duddridge]

And I also noticed that the otool command didn't even list libsqlcipher.a as being linked anywhere.

[Brendan Duddridge]

More info. I am using the FMDB cocoapod. I think it may be pulling in sqlite3 because I see this in the build log:

-lsqlite3

That must be it then.

[Jens Alfke]

On Sep 8, 2016, at 11:48 AM, Brendan Duddridge <bren...@gmail.com> wrote:

More info. I am using the FMDB cocoapod. I think it may be pulling in sqlite3 because I see this in the build log:

CBL already includes its own customized copy of FMDB. Are you using FMDB for something else unrelated to CBL?

—Jens

[Brendan Duddridge]

Yes I am. I have a migration process which reads my old version (possibly) encrypted SQLite files and then uses CBL to create the new database file.

Is it possible to ditch the cocoa pod version of FMDB and just use the one from CBL?

[Jens Alfke]

On Sep 8, 2016, at 12:25 PM, Brendan Duddridge <bren...@gmail.com> wrote:

Is it possible to ditch the cocoa pod version of FMDB and just use the one from CBL?

I wouldn’t recommend it. Ours is (a) based on a very old version of FMDB, and (b) significantly modified, including removing or altering some of its API.

Do you build FMDB as a dynamic or static library? If it’s a dylib (or framework), it shouldn’t make any difference to your app whether it links to sqlite or sqlcipher, because your app is linked separately. If it’s a static library, it doesn’t have any link phase at all.

—Jens

[Brendan Duddridge]

FMDB is linked in via cocoa pods and the libPods-Tap Forms.a is a static library. Pretty sure all the libraries get included via that one static library.

I realized the CBL version of FMDB wouldn't work because it doesn't have FMDatabaseQueue either which I need.

So what I've discovered now is that if I simply modify the FMDB.podspec.json file on my local machine and remove the library reference to sqlite3 then it works for me now and I get encryption (and all the other goodies from the CBL SQLCipher static library).

But if they ever update FMDB I'm going to have to make that change again. Not an ideal solution, but it gets me moving ahead.

[Jens Alfke]

On Sep 8, 2016, at 12:43 PM, Brendan Duddridge <bren...@gmail.com> wrote:

So what I've discovered now is that if I simply modify the FMDB.podspec.json file on my local machine and remove the library reference to sqlite3 then it works for me now and I get encryption (and all the other goodies from the CBL SQLCipher static library).

Sounds like CocoaPods is ‘helpfully’ adding libsqlite3.dylib to your target’s linked libraries.

I can’t think of any better solution than your workaround (though I know very little about CocoaPods.) This is kind of a weird situation where SQLite and SQLCipher are different libraries with exactly the same API, so it’s not possible to link the same target with both of them.

—Jens

[Brendan Duddridge]

Sounds like CocoaPods is ‘helpfully’ adding libsqlite3.dylib to your target’s linked libraries.

I can’t think of any better solution than your workaround (though I know very little about CocoaPods.) This is kind of a weird situation where SQLite and SQLCipher are different libraries with exactly the same API, so it’s not possible to link the same target with both of them.

The strange thing is why it worked fine with Xcode 7 but not with Xcode 8. Must be some new checks to see if a linked library has the same API or something and to ignore it if it does.

[Jens Alfke]

On Sep 8, 2016, at 4:05 PM, Brendan Duddridge <bren...@gmail.com> wrote:

The strange thing is why it worked fine with Xcode 7 but not with Xcode 8. Must be some new checks to see if a linked library has the same API or something and to ignore it if it does.

You had a situation where the same symbols appeared both in a static library and in an imported dylib. It’s likely to be ‘undefined’ which instance of such a symbol the linker will choose to bind with, and that behavior may have changed for some reason in the Xcode 8 tool set. Or perhaps the linker always chooses the library that appears first in its command line, but the ordering of the parameters in the command line generated by Xcode 8 is different. Or maybe it was something else... O_o

—Jens