Replication and Concurrency Question

133 views
Skip to first unread message

Chris Fuentes

unread,
Nov 13, 2014, 11:48:21 AM11/13/14
to mobile-c...@googlegroups.com
According to the most recent docs, (using 1.0.3)

If your app uses Couchbase Lite on multiple threads, then on each thread (or dispatch queue) it must:

  • Create a new CBLManager instance. If you use multiple threads, do not use the sharedInstance.
  • Use only objects (Databases, Documents, ...) acquired from its Manager.
  • Not pass any Couchbase Lite objects to code running on any other thread/queue.
Does this mean that a set of replication objects must be created for each thread as well, even if they're all pointing to the same sync gateway instance? 

Chris Fuentes

unread,
Nov 13, 2014, 12:22:25 PM11/13/14
to mobile-c...@googlegroups.com
What I am primarily interested in doing is separating the replication thread from the threads that will be performing reads/writes, since the replication thread becomes very busy from time to time and effectively freezes interaction with the db. 

Jens Alfke

unread,
Nov 13, 2014, 3:29:34 PM11/13/14
to mobile-c...@googlegroups.com

On Nov 13, 2014, at 9:22 AM, Chris Fuentes <ch...@crowdcomfort.com> wrote:

What I am primarily interested in doing is separating the replication thread from the threads that will be performing reads/writes, since the replication thread becomes very busy from time to time and effectively freezes interaction with the db. 

Replication always runs on a background thread. The CBLReplication object is just a facade that configures the real replication task and reports what it's doing. So it doesn't really matter what thread you start a replication on; it's often best to do it on the UI thread so you can easily use the notifications to drive progress indicators.

—Jens

Chris Fuentes

unread,
Nov 13, 2014, 3:41:23 PM11/13/14
to mobile-c...@googlegroups.com
I notice a very strong correlation between UI freezing and my replication logging statements, which seem to be getting dispatched on the main thread. The CBL instance is tied to a background thread, so I am not sure what else it could be. 

Is the replication running on the same thread that the CBL manager instance is tied to? If that's the case, then perhaps the replication activity is simply blocking reads to the database, which are blocking UI actions? Normally I'd assume reads of a single doc by ID to be imperceptibly fast, but there are noticeable UI lags whenever replication seems to be occurring. 

Jens Alfke

unread,
Nov 13, 2014, 3:57:02 PM11/13/14
to mobile-c...@googlegroups.com
On Nov 13, 2014, at 12:41 PM, Chris Fuentes <ch...@crowdcomfort.com> wrote:

Is the replication running on the same thread that the CBL manager instance is tied to?

No. There's a single background thread, created by CBL, that's reserved for replication, view updates and async view queries.

If that's the case, then perhaps the replication activity is simply blocking reads to the database, which are blocking UI actions? Normally I'd assume reads of a single doc by ID to be imperceptibly fast, but there are noticeable UI lags whenever replication seems to be occurring. 

SQLite has some limitations on concurrency. While one connection (thread) has an active transaction, no other connection can read or write from the database. This is probably what you're seeing. (It's also a major reason we're switching to ForestDB, which doesn't have this limitation.)

If you can pause the app in the debugger while the UI thread is blocked, you should be able to see which other thread is using the database and what it's doing. That would really help diagnose this.

—Jens

Chris Fuentes

unread,
Nov 13, 2014, 6:16:21 PM11/13/14
to mobile-c...@googlegroups.com
According to the backtrace, kind of looks like it's some kind of query being executed on the main thread (which is odd, because the manager was instantiated on a background thread):

stop reason = signal SIGSTOP

    frame #0: 0x3a945324 libsystem_kernel.dylib`pread + 20

    frame #1: 0x3a6021a2 libsqlite3.dylib`___lldb_unnamed_function28$$libsqlite3.dylib + 86

    frame #2: 0x3a61a800 libsqlite3.dylib`___lldb_unnamed_function81$$libsqlite3.dylib + 440

    frame #3: 0x3a619724 libsqlite3.dylib`___lldb_unnamed_function75$$libsqlite3.dylib + 268

    frame #4: 0x3a634592 libsqlite3.dylib`___lldb_unnamed_function128$$libsqlite3.dylib + 122

    frame #5: 0x3a63aab4 libsqlite3.dylib`___lldb_unnamed_function170$$libsqlite3.dylib + 968

    frame #6: 0x3a634674 libsqlite3.dylib`___lldb_unnamed_function129$$libsqlite3.dylib + 44

    frame #7: 0x3a62a4ae libsqlite3.dylib`___lldb_unnamed_function124$$libsqlite3.dylib + 2942

    frame #8: 0x3a628eda libsqlite3.dylib`sqlite3_step + 418

    frame #9: 0x003a8f18 Crowd Comfort`-[CBL_FMResultSet step] + 40

    frame #10: 0x003a8c70 Crowd Comfort`-[CBL_FMResultSet initWithStatement:usingParentDatabase:] + 100

    frame #11: 0x003a7bd2 Crowd Comfort`-[CBL_FMDatabase executeQuery:withArgumentsInArray:orVAList:] + 1058

    frame #12: 0x003a7cc8 Crowd Comfort`-[CBL_FMDatabase executeQuery:] + 36

    frame #13: 0x00357a56 Crowd Comfort`-[CBLDatabase(Internal) documentCount] + 46

  * frame #14: 0x000b2394 Crowd Comfort`-[CBSync logProgress](self=0x16d84a60, _cmd=0x004ce39a) + 824 at CBSync.m:375

    frame #15: 0x000b24ea Crowd Comfort`-[CBSync replicationProgress:](self=0x16d84a60, _cmd=0x004ce321, n=0x16d041d0) + 206 at CBSync.m:385

    frame #16: 0x2cbaf4a0 CoreFoundation`__CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ + 12

    frame #17: 0x2cb0b93c CoreFoundation`_CFXNotificationPost + 1784

    frame #18: 0x2d83b9b8 Foundation`-[NSNotificationCenter postNotificationName:object:userInfo:] + 72

    frame #19: 0x2d84051e Foundation`-[NSNotificationCenter postNotificationName:object:] + 30

    frame #20: 0x0039970c Crowd Comfort`-[CBLReplication updateStatus:error:processed:ofTotal:] + 480

    frame #21: 0x0039a0d4 Crowd Comfort`__35-[CBLReplication bg_updateProgress]_block_invoke + 40

    frame #22: 0x00388eee Crowd Comfort`catchInBlock + 90

    frame #23: 0x00c2eaea libdispatch.dylib`_dispatch_call_block_and_release + 10

    frame #24: 0x00c2ead6 libdispatch.dylib`_dispatch_client_callout + 22

    frame #25: 0x00c324f6 libdispatch.dylib`_dispatch_main_queue_callback_4CF + 810

    frame #26: 0x2cbbcbe8 CoreFoundation`__CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 8

    frame #27: 0x2cbbb2e8 CoreFoundation`__CFRunLoopRun + 1512

    frame #28: 0x2cb09620 CoreFoundation`CFRunLoopRunSpecific + 476

    frame #29: 0x2cb09432 CoreFoundation`CFRunLoopRunInMode + 106

    frame #30: 0x33e720a8 GraphicsServices`GSEventRunModal + 136

    frame #31: 0x300f4358 UIKit`UIApplicationMain + 1440

    frame #32: 0x0001b8d0 Crowd Comfort`main(argc=1, argv=0x00c1db30) + 244 at main.m:19


  thread #2: tid = 0xf84bf, 0x3a9442a0 libsystem_kernel.dylib`kevent64 + 24, queue = 'com.apple.libdispatch-manager'

    frame #0: 0x3a9442a0 libsystem_kernel.dylib`kevent64 + 24

    frame #1: 0x00c3b678 libdispatch.dylib`_dispatch_mgr_invoke + 280

    frame #2: 0x00c305b2 libdispatch.dylib`_dispatch_mgr_thread + 38


  thread #6: tid = 0xf84eb, 0x3a9589cc libsystem_kernel.dylib`__workq_kernreturn + 8

    frame #0: 0x3a9589cc libsystem_kernel.dylib`__workq_kernreturn + 8

    frame #1: 0x3a9d2eac libsystem_pthread.dylib`_pthread_wqthread + 792

    frame #2: 0x3a9d2b84 libsystem_pthread.dylib`start_wqthread + 8


  thread #8: tid = 0xf84ef, 0x3a9589cc libsystem_kernel.dylib`__workq_kernreturn + 8

    frame #0: 0x3a9589cc libsystem_kernel.dylib`__workq_kernreturn + 8

    frame #1: 0x3a9d2eac libsystem_pthread.dylib`_pthread_wqthread + 792


  thread #9: tid = 0xf84f0, 0x3a957b38 libsystem_kernel.dylib`__psynch_cvwait + 24

    frame #0: 0x3a957b38 libsystem_kernel.dylib`__psynch_cvwait + 24

    frame #1: 0x3a9d43f8 libsystem_pthread.dylib`_pthread_cond_wait + 520

    frame #2: 0x3a9d52dc libsystem_pthread.dylib`pthread_cond_wait + 40

    frame #3: 0x2d88bf66 Foundation`-[NSCondition wait] + 194

    frame #4: 0x004097ae Crowd Comfort`-[PFCommandCache runLoop](self=<unavailable>, _cmd=<unavailable>) + 510 at PFCommandCache.m:501

    frame #5: 0x2d90638a Foundation`__NSThread__main__ + 1118

    frame #6: 0x3a9d4e92 libsystem_pthread.dylib`_pthread_body + 138

    frame #7: 0x3a9d4e06 libsystem_pthread.dylib`_pthread_start + 118

    frame #8: 0x3a9d2b90 libsystem_pthread.dylib`thread_start + 8


  thread #12: tid = 0xf84f6, 0x3a9444f0 libsystem_kernel.dylib`mach_msg_trap + 20

    frame #0: 0x3a9444f0 libsystem_kernel.dylib`mach_msg_trap + 20

    frame #1: 0x3a9442e8 libsystem_kernel.dylib`mach_msg + 40

    frame #2: 0x2cbbcb52 CoreFoundation`__CFRunLoopServiceMachPort + 146

    frame #3: 0x2cbbb0f8 CoreFoundation`__CFRunLoopRun + 1016

    frame #4: 0x2cb09620 CoreFoundation`CFRunLoopRunSpecific + 476

    frame #5: 0x2cb09432 CoreFoundation`CFRunLoopRunInMode + 106

    frame #6: 0x000d1084 Crowd Comfort`+[ASIHTTPRequest runRequests](self=0x0074e7a0, _cmd=0x004d0871) + 260 at ASIHTTPRequest.m:4798

    frame #7: 0x2d90638a Foundation`__NSThread__main__ + 1118

    frame #8: 0x3a9d4e92 libsystem_pthread.dylib`_pthread_body + 138

    frame #9: 0x3a9d4e06 libsystem_pthread.dylib`_pthread_start + 118


  thread #13: tid = 0xf84f7, 0x3a9444f0 libsystem_kernel.dylib`mach_msg_trap + 20, name = 'com.apple.NSURLConnectionLoader'

    frame #0: 0x3a9444f0 libsystem_kernel.dylib`mach_msg_trap + 20

    frame #1: 0x3a9442e8 libsystem_kernel.dylib`mach_msg + 40

    frame #2: 0x2cbbcb52 CoreFoundation`__CFRunLoopServiceMachPort + 146

    frame #3: 0x2cbbb0f8 CoreFoundation`__CFRunLoopRun + 1016

    frame #4: 0x2cb09620 CoreFoundation`CFRunLoopRunSpecific + 476

    frame #5: 0x2cb09432 CoreFoundation`CFRunLoopRunInMode + 106

    frame #6: 0x2c6c23de CFNetwork`+[NSURLConnection(Loader) _resourceLoadLoop:] + 486

    frame #7: 0x2d90638a Foundation`__NSThread__main__ + 1118

    frame #8: 0x3a9d4e92 libsystem_pthread.dylib`_pthread_body + 138

    frame #9: 0x3a9d4e06 libsystem_pthread.dylib`_pthread_start + 118


  thread #15: tid = 0xf8506, 0x3a95808c libsystem_kernel.dylib`__select + 20, name = 'com.apple.CFSocket.private'

    frame #0: 0x3a95808c libsystem_kernel.dylib`__select + 20

    frame #1: 0x2cbc131e CoreFoundation`__CFSocketManager + 498

    frame #2: 0x3a9d4e92 libsystem_pthread.dylib`_pthread_body + 138

    frame #3: 0x3a9d4e06 libsystem_pthread.dylib`_pthread_start + 118


  thread #17: tid = 0xf850b, 0x3a9444f0 libsystem_kernel.dylib`mach_msg_trap + 20, name = 'TestFlight Network Runloop Thread'

    frame #0: 0x3a9444f0 libsystem_kernel.dylib`mach_msg_trap + 20

    frame #1: 0x3a9442e8 libsystem_kernel.dylib`mach_msg + 40

    frame #2: 0x2cbbcb52 CoreFoundation`__CFRunLoopServiceMachPort + 146

    frame #3: 0x2cbbb0f8 CoreFoundation`__CFRunLoopRun + 1016

    frame #4: 0x2cb09620 CoreFoundation`CFRunLoopRunSpecific + 476

    frame #5: 0x2cb09432 CoreFoundation`CFRunLoopRunInMode + 106

    frame #6: 0x2d84142c Foundation`-[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 264

    frame #7: 0x2d88f8ec Foundation`-[NSRunLoop(NSRunLoop) run] + 80

    frame #8: 0x002d4dc4 Crowd Comfort`+[TFURLConnectionOperation _runNetworkThread:] + 212

    frame #9: 0x2d90638a Foundation`__NSThread__main__ + 1118

    frame #10: 0x3a9d4e92 libsystem_pthread.dylib`_pthread_body + 138

    frame #11: 0x3a9d4e06 libsystem_pthread.dylib`_pthread_start + 118


  thread #18: tid = 0xf8514, 0x3a9444f0 libsystem_kernel.dylib`mach_msg_trap + 20, name = 'CouchbaseLite'

    frame #0: 0x3a9444f0 libsystem_kernel.dylib`mach_msg_trap + 20

    frame #1: 0x3a9442e8 libsystem_kernel.dylib`mach_msg + 40

    frame #2: 0x2cbbcb52 CoreFoundation`__CFRunLoopServiceMachPort + 146

    frame #3: 0x2cbbb0f8 CoreFoundation`__CFRunLoopRun + 1016

    frame #4: 0x2cb09620 CoreFoundation`CFRunLoopRunSpecific + 476

    frame #5: 0x2cb09432 CoreFoundation`CFRunLoopRunInMode + 106

    frame #6: 0x2d84142c Foundation`-[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 264

    frame #7: 0x003640ce Crowd Comfort`-[CBL_Server runServerThread] + 310

    frame #8: 0x2d90638a Foundation`__NSThread__main__ + 1118

    frame #9: 0x3a9d4e92 libsystem_pthread.dylib`_pthread_body + 138

    frame #10: 0x3a9d4e06 libsystem_pthread.dylib`_pthread_start + 118


  thread #19: tid = 0xf8530, 0x3a9589cc libsystem_kernel.dylib`__workq_kernreturn + 8

    frame #0: 0x3a9589cc libsystem_kernel.dylib`__workq_kernreturn + 8

    frame #1: 0x3a9d2eac libsystem_pthread.dylib`_pthread_wqthread + 792


On Thursday, November 13, 2014 11:48:21 AM UTC-5, Chris Fuentes wrote:

Jens Alfke

unread,
Nov 14, 2014, 5:26:20 PM11/14/14
to mobile-c...@googlegroups.com

On Nov 13, 2014, at 3:16 PM, Chris Fuentes <ch...@crowdcomfort.com> wrote:

According to the backtrace, kind of looks like it's some kind of query being executed on the main thread (which is odd, because the manager was instantiated on a background thread):

The CBLReplication is posting a notification, which is causing your(?) CBSync to ask for the database's documentCount, which runs a SQLite query.

There are no bugs I know of with notifications being delivered on the wrong thread, so it looks as though the object in question (replication, database, manager) are bound to the main thread.

—Jens
Reply all
Reply to author
Forward
0 new messages