Preview of the new TouchDB API

[Jens Alfke]

As I said a few days ago, I’ve been working on merging CouchCocoa into TouchDB. This makes its code simpler & faster because it can just call the internal TouchDB core directly instead of having to go through an internal REST API. While I was doing this I made other changes too such as:

• Changing the namespace prefix from “Couch” to “TD”

• Making nearly all the API calls synchronous (i.e. no more RESTOperation)

• Getting rid of the “design document” concept and exposing views directly

• Merging the two replication classes into one

This version will become TouchDB 1.1 or 2.0 or something (I haven’t decided on a version number yet!) but you can try it out now by checking out the “public-api” branch from Github. I’ve written a new document [on the wiki] that describes how to change your project and source code. There is a script included that helps with the ‘Couch’—>’TD’ renaming, but there’s more conversion that has to be done by hand. Hopefully it won’t be hard; I was able to convert Grocery Sync pretty quickly.

Converted apps appear to be about 300k smaller! I haven’t measured performance, but it should be better.

I’m interested to hear what people think of this. This is an opportunity to make incompatible changes to the API, so if there’s something that’s been bugging you about CouchCocoa, now’s the time to suggest a fix. Or if you think any of the changes go in the wrong direction, let me know why.

—Jens

[Eduardo Scoz]

Hi Jens,

I haven't run the new API yet, but from the comparison (https://github.com/couchbaselabs/TouchDB-iOS/compare/master...public-api) seems like a lot of work was indeed put into it. I'll try to run it later today.

One question: now that there's only one project, is it now easier to have the source code being built as part of the application, instead of loading the precompiled framework? Have you done it?

[Jens Alfke]

On Nov 10, 2012, at 8:25 AM, Eduardo Scoz <eduar...@gmail.com> wrote:

One question: now that there's only one project, is it now easier to have the source code being built as part of the application, instead of loading the precompiled framework? Have you done it?

I think that should work, though you’ll need to #include the TouchDB prefix header (.pch file) as part of your app’s prefix.

—Jens

[Eduardo Scoz]

fantastic, I'll give it a try.

[lenkawell]

My requests are:

(1) A very efficient interface to add multi-megabyte attachments (https://github.com/couchbaselabs/CouchCocoa/issues/19 ). 

(2) A simple interface to add an attachment to a new document (https://github.com/couchbaselabs/CouchCocoa/issues/42 ).

(3) An interface to completely clear replication history (both local and remote).

[Andrew Reslan]

Jens

Hi, will features that rely on sending HTTP header requests i.e Issue #127 still be supported. If not then I'd like to see a direct API call to get the local path of an attachment.

[Traun Leyden]

Very exciting news - can't wait to try it!

On Friday, November 9, 2012 2:44:42 PM UTC-8, Jens Alfke wrote:

[Jens Alfke]

On Nov 11, 2012, at 6:44 AM, Andrew Reslan <andrew...@mac.com> wrote:

> Hi, will features that rely on sending HTTP header requests i.e Issue #127 still be supported. If not then I'd like to see a direct API call to get the local path of an attachment.

Good point. I’m sure there are some holes like this in the new API — I hadn’t thought of the attachment-path header. I’ve just checked in a new property of TDAttachment to get the path. If you run into other missing bits, please file issues on them; they’re mostly going to be pretty trivial to add, I think.

—Jens

[Traun Leyden]

I had to copy the following headers manually to get past some compile errors:

TDCache.h

MYDynamicObject.h

TDUITableSource.h

Is my setup wrong or was this an oversight?

(I'm building TouchDB as a framework and copying into my application project)

[Traun Leyden]

Also how would this code get converted to the public-api branch?  

    CouchDatabase *database = ... 

    CouchResource *changes = [[[CouchResource alloc] initWithParent:database relativePath:@"_changes"] autorelease];

    RESTOperation *operation = [changes GET];

    [operation onCompletion: ^{

        if (operation.error) {

            DLog(@"Error getting changes: %@", [operation.error localizedDescription]);

        }

        else {

             RESTBody *body = [operation responseBody];

             NSError *error = nil;

             NSData *data = [[body asString] dataUsingEncoding:NSUTF8StringEncoding];

            NSDictionary *result = [NSJSONSerialization JSONObjectWithData:...]

            ....

        }

    }];

    [operation start];

[Jens Alfke]

On Nov 12, 2012, at 12:17 PM, Traun Leyden <traun....@gmail.com> wrote:

I had to copy the following headers manually to get past some compile errors:

TDCache.h

MYDynamicObject.h

TDUITableSource.h

Oops! Sorry, that’s a problem that doesn’t show up when building a sample app that’s in the same project folder as the framework, since it can see all the project headers.

Just checked in a fix.

—Jens

[Jens Alfke]

On Nov 12, 2012, at 12:24 PM, Traun Leyden <traun....@gmail.com> wrote:

Also how would this code get converted to the public-api branch?  

There isn’t any direct access to the _changes feed in the new API, mostly because I didn’t think anyone would need it. What are you using it for?

—Jens

[Traun Leyden]

Looks like this was just some unimportant debugging related thing.. so nevermind, not really a priority.

Another issue I ran into trying to get grocery-sync working and pointing to a remote couchdb url..

When I set the url or hardcode it, I'm getting this error.

2012-11-12 13:43:37.294 Grocery Sync[97689:c07] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[NSURL initWithString:relativeToURL:]: nil string parameter'

*** First throw call stack:

(0x1835012 0x15fde7e 0x1834deb 0xff3d6d 0x106aab4 0xa1ca0 0x9a0b6 0xa1877 0x990be 0x9924b 0x91a27 0x55af 0x3939 0x626817 0x626882 0x626b2a 0x63def5 0x63dfdb 0x63e286 0x63e381 0x63eeab 0x63efc9 0x63f055 0x7443ab 0x59592d 0x16116b0 0x2bd1fc0 0x2bc633c 0x2bc6150 0x2b440bc 0x2b45227 0x2be7b50 0x5439ff 0x5444e1 0x555315 0x55624b 0x547cf8 0x27ecdf9 0x27ecad0 0x17aabf5 0x17aa962 0x17dbbb6 0x17daf44 0x17dae1b 0x5437da 0x54565c 0x2bde 0x2af5 0x1)

libc++abi.dylib: terminate called throwing an exception

(lldb) bt

* thread #1: tid = 0x1c03, 0x98685a6a libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT

    frame #0: 0x98685a6a libsystem_kernel.dylib`__pthread_kill + 10

    frame #1: 0x92514acf libsystem_c.dylib`pthread_kill + 101

    frame #2: 0x01b1957b libsystem_sim_c.dylib`abort + 140

    frame #3: 0x01c3cf7b libc++abi.dylib`abort_message + 105

    frame #4: 0x01c3aa25 libc++abi.dylib`default_terminate() + 34

    frame #5: 0x015fe0c1 libobjc.A.dylib`_objc_terminate() + 94

    frame #6: 0x01c3aa65 libc++abi.dylib`safe_handler_caller(void (*)()) + 13

    frame #7: 0x01c3aacd libc++abi.dylib`std::terminate() + 23

    frame #8: 0x01c3bbc2 libc++abi.dylib`__cxa_throw + 110

    frame #9: 0x015fdf89 libobjc.A.dylib`objc_exception_throw + 311

    frame #10: 0x01834deb CoreFoundation`+[NSException raise:format:] + 139

    frame #11: 0x00ff3d6d Foundation`-[NSURL(NSURL) initWithString:relativeToURL:] + 95

    frame #12: 0x0106aab4 Foundation`-[NSURL(NSURL) initWithString:] + 48

    frame #13: 0x000a1ca0 Grocery Sync`-[TDReplication initWithDocument:] + 352 at TDReplication.m:84

    frame #14: 0x0009a0b6 Grocery Sync`-[TDModel initWithNewDocumentInDatabase:] + 310 at TDModel.m:49

    frame #15: 0x000a1877 Grocery Sync`-[TDReplication initWithDatabase:remote:pull:] + 567 at TDReplication.m:58

    frame #16: 0x000990be Grocery Sync`-[TDDatabaseManager replicationWithDatabase:remote:pull:create:] + 1038 at TDDatabaseManager.m:176

    frame #17: 0x0009924b Grocery Sync`-[TDDatabaseManager createReplicationsBetween:and:exclusively:] + 203 at TDDatabaseManager.m:188

    frame #18: 0x00091a27 Grocery Sync`-[TDDatabase replicateWithURL:exclusively:] + 119 at TDDatabase.m:223

    frame #19: 0x000055af Grocery Sync`-[RootViewController updateSyncURL] + 463 at RootViewController.m:279

    frame #20: 0x00003939 Grocery Sync`-[RootViewController viewDidLoad] + 1593 at RootViewController.m:79

    frame #21: 0x00626817 UIKit`-[UIViewController loadViewIfRequired] + 536

[Traun Leyden]

I worked around that NSInvalidArgumentException issue with this: https://github.com/signaturelabs/TouchDB-iOS/commit/e749778756369fdadaed695911f1250cee6b41a8

as well as another issue (see commit message): https://github.com/signaturelabs/TouchDB-iOS/commit/d51b359ac4816ae9255c0536e08191e31d3e7428

[Traun Leyden]

One more thing I noticed, it looks like the "continuous" property is not being set to true in the new grocery-sync.

[Matias Piipari]

Interesting, look forward to trying this out! Any particular reason you got rid of the asynchronous APIs? I understand I can recreate a similar API easily on top of a synchronous API, but I quite like the "asynchronous by default" way that the CouchCocoa RESTOperation API has made me design stuff (only wait or add a completion handler when necessary).

[Jens Alfke]

On Nov 16, 2012, at 3:35 PM, Matias Piipari <matias....@gmail.com> wrote:

Interesting, look forward to trying this out! Any particular reason you got rid of the asynchronous APIs?

Simpler implementation, primarily. Since the public API is now just calling native internal APIs rather than going through a REST/HTTP interface, it’s a lot easier if it can just call it directly. Otherwise I have to wrap dispatch_async around the insides of every public API call, allocate response/operation objects, etc. (This really adds up — an iOS app built with the new TouchDB appears to be ~300k smaller than one built with TouchDB 1.0 + CouchCocoa.)

Also, the CouchCocoa API wasn’t consistently asynchronous. Most of the higher level stuff ended up being synchronous, especially once you got to the model layer. So it’s a more consistent API design to give everything the same behavior.

—Jens

[Matias Piipari]

Ok, understood, sounds sensible. I notice from the wiki that the HTTP listener will still be available, and the use case of having a web view talk to the database directly over HTTP is mentioned. This is a central use case also for the Mac application I'm building. Just to check, is the intention with this changed implementation also to still to allow also replicating TouchDBs directly between each other without a CouchDB in between?

--
 
 

--
- - - - -
Matias Piipari, PhD

http://twitter.com/mz2

Papers - Your Personal Library of Science

Winner of the Apple Design Awards & Ars Design Award
Best Mac OS X Scientific Solution
http://mekentosj.com/papers

Papers for iPad - all your papers
available wherever you go:
http://mekentosj.com/papers/ipad

[Jens Alfke]

On Nov 16, 2012, at 4:23 PM, Matias Piipari <matias....@gmail.com> wrote:

Just to check, is the intention with this changed implementation also to still to allow also replicating TouchDBs directly between each other without a CouchDB in between?

Yup, provided you link in the listener of course. (Basically the TDRouter class, which implements the REST API, has moved into the listener framework since the core TouchDB doesn’t need it anymore.)

—Jens

[Traun Leyden]

Any ETA on when this will be merged into the stable release?

(I realize this is a tough question, but just looking for a realistic ballpark, eg, 1 mo, 3 mo, 6 mo..)

[thstart]

tried to compile but got:

Downloads/TouchDB-iOS-public-api/Source/TouchDBPrefix.h:19:9: 'CollectionUtils.h' file not found

On Friday, November 9, 2012 2:44:42 PM UTC-8, Jens Alfke wrote:

[AJ]

I'd be interested in this also, before I startup a big new project.  Even wildly inaccurate guestimates would help.

[Jens Alfke]

On Nov 20, 2012, at 9:05 AM, Traun Leyden <traun....@gmail.com> wrote:

Any ETA on when this will be merged into the stable release?

(I realize this is a tough question, but just looking for a realistic ballpark, eg, 1 mo, 3 mo, 6 mo..)

I’m thinking of the new API as being part of a “1.1” or “1.5” type of release, i.e. the next major release. If you’re asking when that would ship, I’d guess 3-4 months would be the right ballpark.

—Jens

[Jens Alfke]

On Nov 24, 2012, at 3:12 PM, thstart <brag...@bragbuddy.com> wrote:

> tried to compile but got:
>
> Downloads/TouchDB-iOS-public-api/Source/TouchDBPrefix.h:19:9: 'CollectionUtils.h' file not found

Sounds like you haven’t run “git submodule update”.

—Jens

[Eduardo Scoz]

Hey Jens, I just started looking into the new branch and so far it looks great.

Also the fact that new TouchDB can be run as a simple submodule / subproject is awesome, really makes things simpler to stay in sync with the repo.