sqlcipher_random + iOS = BAD_ACCESS

[Johnnes]

Hi there,

we're actually quite happy with the state of sqlcipher, as it did run
without any problems for so long, but since a few weeks we are having
problems with bad access errors.

We're having some issues with sqlcipher_random.
The problem is RAND_bytes((unsigned char*)buffer, length), where the
buffer variable is not set at all, while length comes with (probably
valid) arguments.
The SQLString itself also is perfectly valid, and for now the app runs
without encryption just fine.

We are encountering this problem more often when there's heavy file
traffic on the system, in this case it's different files that are read
or written, not the same. The whole system is based on concurrent FMDB
Database Queues, wich comes down to Grand Central Dispatch. As the app
runs fine w/o encryption we figure it'll not be a concurrency issue?

While I'm trying to setup a demo app for that problem: Did someone
came across an issue like this and might be able to give a hint? We
would really appreciate that :(

http://dl.dropbox.com/u/13869889/sqlite_bad_acc.png

greets,
johannes

[Stephen Lombardo]

Hi Johannes,

We haven't had any reports of this sort of issue that I'm aware of, but it definitely sounds like a legitimate issue. I have a few questions:

1. In the debugger, can you see if there are any other threads active when this occurs that are using SQLCipher / OpenSSL (perhaps also calling RAND_bytes)?

2. How many databases are open by the application? 

2. What version of OpenSSL are you building with?

Cheers,

Stephen

[Steven Parkes]

Are you setting up openssl to handle threading (since GCD uses threads under the covers)?

OpenSSL needs some callbacks to work correctly in the presence of threads:
http://www.openssl.org/docs/crypto/threads.html

Here's what I use (it's C++ but can be trivially translated to C):

pthread_mutex_t* mutex_buffer;

void thread_id_function(CRYPTO_THREADID* id) {
CRYPTO_THREADID_set_pointer(id, pthread_self());
}

void locking_function(int mode, int id, const char *file, int line) {
if(mode & CRYPTO_LOCK) {
pthread_mutex_lock(&mutex_buffer[id]);
} else {
pthread_mutex_unlock(&mutex_buffer[id]);
}
}

void openssl_init() {
static pthread_mutex_t* mutex_buffer;
if (!mutex_buffer) {
mutex_buffer = new pthread_mutex_t[CRYPTO_num_locks()];
for(int i=0; i<CRYPTO_num_locks(); ++i) {
pthread_mutex_init(&mutex_buffer[i], 0);
}

CRYPTO_set_locking_callback(locking_function);
CRYPTO_THREADID_set_callback(thread_id_function);
}
}

Note I use openssl for sqlcipher, libcurl, and my own code. I use all these packages from multiple queues and I'm pretty sure I can trivially show things will go south if I didn't have the callbacks set up. That said, I'm not 100% sure it's the source of your issue ...

[Johnnes]

Hi all,

we were able to trace the problem to a custom third party library.
This library wasn't properly converted to ARC. After we finally
noticed, ported and fixed some surfacing memory leaks, everything
works fine.
Still we don't know exactly why this caused to backfire on sqlcipher.
We are still testing the app, but it seems to be stable now.

Sorry for the fuzz, but we really had no clue leading to this end of
the project :(

Thanks a lot for the fast response and hints.

Kind regards,
Johannes

[Quentin Adam]

I had the exact same issue with sqlitecipher on iOS. My app is using multiple encrypted sqlite database files (but ensuring of course only one thread accessing each file).

The project has been in production for a few months and had been working perfectly fine up to the moment when there has been a big workload of transactions to be executed on several database files at once.

The program was throwing EXC_BAD_ACCESS exception from time to time. The exception seemed to be located in the crypto lib (sqlcipher_random > RAND_bytes > sha1_block_data_order) (see attached screenshot for stack trace)

To me, it seems that somehow the openssl crypto lib that I had compiled was not including multi-threading support (however I did this recently around beginning of 2013 and used the version from https://github.com/sqlcipher/openssl-xcode as documented). I recompiled the openssl crypto lib (now using https://github.com/st3fan/ios-openssl), re-compiled sqlcipher and that seemed to have solved the problem.

I also created a small test routine if anybody is interested in pining down the issue. It basically opens 100 sqlite database handles referring to 100 different files and starts 100 dispatch_queues.

In each queue, 1000 batches of 100 inserts (wrapped into a transaction) are executed. With my old libcrypto.a, the code was never completing (always throws a EXC_BAD_ACCESS at some random point). However, including all database related transactions inside a dispatch_sync on a single common queue did solve the issue (see commented out portions). Using the newly compiled libcrypto.a seemed to definitely solve the issue.

[Stephen Lombardo]

Hi Adam,

Thanks for putting together a clear and reproducible test case for this, we'll look into it further early next week. 

I noticed that the attached test case isn't setting the openssl thread id or locking callbacks. I haven't looked closely at the ios-openssl code yet, but are you aware of whether it's setting those up under the hood?

Cheers,

Stephen

--
 
---
You received this message because you are subscribed to the Google Groups "SQLCipher Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlcipher+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Quentin Adam]

Hi Stephen,

Thanks for the quick reply.

To be completely fair, I have no idea. It seemed however that ios-openssl is not changing any callbacks or so because it does not modify the source code (it is basically a build.sh script that compiles the  any openssl tar.gz source code archive). Looking at their build.sh script it basically seems to do a standard build of openssl (replacing some values in the makefiles for compiling for armv7 and armv7s) and combining the output library files together with lipo.

One thing that I did though is add the "thread" option on the ./Configure command to make sure it is building openssl with multi-threaded support, however looking at the openssl docs, that seems to be the default option anyway.

[Stephen Lombardo]

Hi Adam,

We've committed a change to the prerelease branch that provides an external mutex around calls to RAND_bytes(). This is a simple workaround to prevent concurrent calls to RAND_bytes for applications that have not configured threading callbacks in OpenSSL (the callbacks would still be considered a more proper solution).

Could you give the latest code on prerelease a try with your test scenarios and let us know if it resolves the problem? 

https://github.com/sqlcipher/sqlcipher/tree/prerelease

Thanks!

Cheers,

Stephen