Accessing Couchbase through separate service process

[Hunter Barrington]

I'm trying to run index updates in a separate service process in Android. Ideally I'd run a sync and then tell my service (using IPC) to slowly update indexes in the service background.

My app is kind of CMS-y, so I'd like to do stale queries and update indexes in a separate service process. Right now loading views (especially after a sync or any process that writes/updates alot) is wicked slow.

We have a minimum of 1300 documents and most times have between 3K and 5K.

I've created an indexing service and am able to load the manager and the database by name but it seems to create some weirdness with the main app. I started by trying just one view - "accounts_by_key_with_name"  No problems in the service but the livequery (which I'll probably turn into a normal query) gives me about 20 of the following error:

11-18 00:33:11.959      439-998/com.-----.app W/View﹕ Failed to rebuild view accounts_by_key_with_name.  Result code: 500

11-18 00:33:11.959      439-998/com.-----.app E/Query﹕ Exception caught in runAsyncInternal

    java.lang.NullPointerException

            at com.couchbase.lite.View.updateIndex(View.java:547)

            at com.couchbase.lite.Database.queryViewNamed(Database.java:2046)

            at com.couchbase.lite.Query$1.run(Query.java:403)

            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:422)

            at java.util.concurrent.FutureTask.run(FutureTask.java:237)

            at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:152)

            at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:265)

            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)

            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)

            at java.lang.Thread.run(Thread.java:841)

When I stop querying there and shuffle things around a bit, that error goes away but I get some illegal state exceptions from my indexing service about not being able to load the database. Any thoughts? I can give more code if you'd like.

A top level question, which is probably a "your mileage will vary" type question, will being in a separate process save me that much GC/performance gains in comparison to the headache it saves? Documentation says that managers are threadsafe by default in Android so that could be a quick fix.

Dirty proof of concept code to index in a separate service process:

     try {

            Database database = mManager.getExistingDatabase(Application.DATABASE_NAME);

            if (database == null){

                mDatabase = mManager.getDatabase(Application.DATABASE_NAME);

                Log.d (TAG, "Database created");

            } else {

                mDatabase = mManager.getDatabase(Application.DATABASE_NAME);

                Log.d (TAG, "Database loaded");

            }

        } catch (CouchbaseLiteException e) {

            Log.e(TAG, "Couchbase error", e);

        }

        View view = mDatabase.getView("accounts_by_key_with_name");

        if(view.getMap() == null) {

            Mapper map = new Mapper() {

                @Override

                public void map(Map<String, Object> document, Emitter emitter) {

                    if ("account".equals(document.get("type"))) {

                        emitter.emit(document.get("_id"), document.get("name"));

                    }

                }

            };

            view.setMap(map, "1");

        }

        try{

            view.updateIndex();

            Log.e(TAG, "updated index");

        } catch (CouchbaseLiteException e){

            Log.e(TAG, "shit went down");

        }

[Traun Leyden]

Regarding the null pointer exception, it looks like the mapBlock is null:

https://github.com/couchbase/couchbase-lite-java-core/blob/e91098107e1c130ecb8bd8e997ef0c202d9360e0/src/main/java/com/couchbase/lite/View.java#L547

Can you paste the code where you're defining your map block for the view?

Regarding running couchbase lite from two separate processes, that's "uncharted waters".  It should work in theory, but you may run into problems that we haven't seen yet.  

I'd recommend keeping everything in the same process until you have a hard requirement to do otherwise.

--
You received this message because you are subscribed to the Google Groups "Couchbase Mobile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mobile-couchba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mobile-couchbase/7d7b3c65-9852-4dc6-9e23-0da6a5f0d7db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jens Alfke]

On Nov 18, 2014, at 9:54 AM, Hunter Barrington <hun...@ftwdev.com> wrote:

I'm trying to run index updates in a separate service process in Android. Ideally I'd run a sync and then tell my service (using IPC) to slowly update indexes in the service background.

I would not recommend this.

First: We don't support multiple processes accessing the same CBL database. It's kind of a gray area in that it should basically work (SQLite will manage concurrency using file locking), but there are things that won't happen, such as notifying your objects of changes made by the other process. I don't think we can provide support if you're going down this path.

Second: I don't think updating indexes in a separate process will buy you anything. It won't go any faster than doing it in the same process, and you won't get better concurrency. There is still a hard limit that when one SQLite connection/thread is updating a database no other connection can access it. The real fix for that is ForestDB, coming in version 1.1.

—Jens