iOS peer to peer replication

[sas]

Hi all,

I'm trying to set up peer to peer replication between iOS devices for
my app. Out of habit, I've been working with TouchDB but I just
realized that I won't be able to replicate between devices with
TouchDB, because it does not expose its db via http, right? The REST
operations in the framework are resolved through TDRouter and
"touchdb://" URLs if I'm not mistaken.

I don't suppose there's any way to expose a TouchDB on one device in
such a way that another could pull directly from it, is there? All the
examples I've seen (and implemented) always had a "real" couchdb at
the remote end, with the touchdb local. In order to propagate from
device to device one would have to go via a this central couchdb hub,
which is a setup I want to avoid.

So to replicate between devices directly, I'd have to go back to the
erlang iOS Couchbase [1], don't I?

Cheers,
Sven

[1] http://files.couchbase.com/developer-previews/mobile/ios/nightlybuild/ios-couchbase-2.0.1-3.zip

[J Chris Anderson]

On Apr 30, 2012, at 6:21 AM, sas wrote:

> Hi all,
>
> I'm trying to set up peer to peer replication between iOS devices for
> my app. Out of habit, I've been working with TouchDB but I just
> realized that I won't be able to replicate between devices with
> TouchDB, because it does not expose its db via http, right? The REST
> operations in the framework are resolved through TDRouter and
> "touchdb://" URLs if I'm not mistaken.
>
> I don't suppose there's any way to expose a TouchDB on one device in
> such a way that another could pull directly from it, is there? All the
> examples I've seen (and implemented) always had a "real" couchdb at
> the remote end, with the touchdb local. In order to propagate from
> device to device one would have to go via a this central couchdb hub,
> which is a setup I want to avoid.
>
> So to replicate between devices directly, I'd have to go back to the
> erlang iOS Couchbase [1], don't I?

Don't worry. TouchDB has you covered. I'm sorry I don't know the exact details but I know Jens recently fixed some bugs he found while testing p2p TouchDB replication.

If I wanted to figure out how to implement p2p in my app I'd probably start by looking at the classes and stuff referenced by TouchServ and maybe grep the rest of the project for them.

If you are game to publish something on how to set up p2p I think people will appreciate it.

Chris

[sas]

Hi Chris,

thanks for the fast reply. That sounds great, I'll start looking into
it right away and I'd be happy to publish what I find.

Cheers,
Sven

> > [1]http://files.couchbase.com/developer-previews/mobile/ios/nightlybuild...

[Jens Alfke]

On Apr 30, 2012, at 6:21 AM, sas wrote:

I don't suppose there's any way to expose a TouchDB on one device in
such a way that another could pull directly from it, is there?

There’s an optional TouchDBListener framework you can use to run a real HTTP interface to TouchDB. Build its target and add it to your app, then start it up like this:

    [server tellTDServer: ^(TDServer* tdServer) {

        TDListener* istener = [[TDListener alloc] initWithTDServer: tdServer port: 8888];

        [listener start];

    }];

where ‘8888’ is whatever TCP port you want to listen on.

Things to be aware of — yes, these are significant issues, but I haven’t gotten to them yet, as P2P sync isn’t officially on the 1.0 feature list.

• THERE IS NO SECURITY. In CouchDB terms, it runs in “admin party” mode. Anyone who knows your IP address and the server’s port number has full read-write access to your databases.

• The listener only runs while your app is active. (It might continue to run for a short while after the user switches apps, but that’s at the whim of the OS.)

• You will probably have to close and re-open the listener after your app is re-activated, because the OS may have closed its TCP socket.

That being said, try it out! I just wouldn’t recommend using it in an app you want to release to the general public.

—Jens

[sas]

Thanks for your comments, Jens. I had already gone for the TouchServ
route, which I'm setting up in my app like so:

- (void)listen
{
  if (_listener) {
    [_listener stop];
  }
  NSError* error;
  TDServer* server = [[TDServer alloc]
initWithDirectory:GetServerPath() error: &error];
  if (error) {
    NSLog(@"FATAL: Error initializing TouchDB: %@", error);
    return;
  }
  int kPortNumber = 59840;
  _listener = [[TDListener alloc] initWithTDServer: server
port:kPortNumber];
  [_listener start];
  NSLog(@"TouchServ %@ is listening on port %d ... relax!", [TDRouter
versionString], kPortNumber);
}

This appears to be working: I can connect to my device at http://10.0.1.77:59840/
and I get a reply:

{
  "version" : "0.69",
  "couchdb" : "Welcome",
  "TouchDB" : "Welcome"
}

However, this works only once. Reloading the url leads to a timeout.
I'm not deactivating the app or anything: the listener seems to stop
after the first connect. I haven't used CocoaHTTPServer yet, so I
should probably do some digging and set up a standalone test for this.
But maybe this problem rings a bell for you?

Cheers,
Sven

>  smime.p7s
> 6KViewDownload

[Nicolas Lapomarda]

Sven,

For what it's worth, I'd already setup a standalone test for P2P replication failing completely between TouchDB instances, but that was for the Mac version of the code.
I'm not sure if the issue you're having is the same as the one I had, but I can make the project available to you if you'd like. I shared it to Jens back in March and I believe he's started investigating it recently.

Cheers,
Nico

[Jens Alfke]

On May 1, 2012, at 12:30 PM, sas wrote:

However, this works only once. Reloading the url leads to a timeout.
I'm not deactivating the app or anything: the listener seems to stop
after the first connect. I haven't used CocoaHTTPServer yet, so I
should probably do some digging and set up a standalone test for this.
But maybe this problem rings a bell for you?

No, I haven’t seen that problem. If you turn on logging (-Log YES -LogTDListener YES -LogTDRouter YES) does anything get logged when you send the second request? Can you stop the app in the debugger and see if any threads look like they’re deadlocked?

—Jens

[Jens Alfke]

On May 1, 2012, at 12:33 PM, Nicolas Lapomarda wrote:

For what it's worth, I'd already setup a standalone test for P2P replication failing completely between TouchDB instances, but that was for the Mac version of the code.
I'm not sure if the issue you're having is the same as the one I had, but I can make the project available to you if you'd like. I shared it to Jens back in March and I believe he's started investigating it recently.

Actually I lost the attachment along with the email it was in, somehow :( … I mailed you last week asking for another copy but didn’t get a reply. I can take a look at it today if you send it again.

—Jens

[Jens Alfke]

I’ve modified the TouchDB iOS demo app to run the listener, and I couldn’t reproduce the problem. I can send lots of requests and it keeps answering them.

I wonder if your object that owns the listener is getting closed or dealloced somehow, and stopping the listener? Try setting breakpoints on the lines where you start and stop the listener.

—Jens

[Jens Alfke]

On May 1, 2012, at 2:45 PM, Jens Alfke wrote:

I’ve modified the TouchDB iOS demo app to run the listener, and I couldn’t reproduce the problem. I can send lots of requests and it keeps answering them.

OK, now I can reproduce it. It only seems to get stuck if TouchDB is otherwise is busy — I’m getting this if I send requests during a big replication.

I think there’s some kind of race condition bug in the listener code. Hopefully I can track it down without too much trouble. I’ve filed this as https://github.com/couchbaselabs/TouchDB-iOS/issues/73 .

—Jens

[Nicolas Lapomarda]

Jens,

I replied to that email with the attachment as soon as I got it! I think your mail client has grown to ignore my emails ;)

I'll resend it now though, but it's great that you've managed to replicate it on your own. It does seem like a weird bug.

Cheers,

Nico

[sas]

Ah, good to know that you can reproduce it :) Some more info:

When breaking in TDHTTPResponse's "onDataAvailable:finished:" is see
that my requests are being handled and that "data" contains the
response. "finished" is YES and "onFinished" processes to the end.
Going further, I ended up in [TDRouter finished]. There I noticed that
the "_onFinished" block is nil, which may be normal. TDRouter's
"_response.body.asJSON" looks fine:

(lldb) po [[NSString alloc] initWithData:_response.body.asJSON
encoding:1]
(id) $89 = 0x08598890 {"doc_count":6,"db_name":"my_test","disk_size":
94208,"update_seq":6,"db_uuid":"48D0B315-C99E-4593-910C-C31F687E4EEB"}

Most of the times I'm stepping through I actually get a response from
curl now, perhaps supporting your speculation of a race condition.

My TouchDB should not be otherwise busy, by the way, at least as far
as I can tell. There's no sync set up and no writing or reading going
on.

[Nicolas Lapomarda]

Jens,

I'm happy to report that your fix for issue #73 seems to have fixed my problem from March. P2P databases now seem to replicate OK. I'd be interested to know if this fixes Sven's issue as well?

Cheers,
Nico

[Jens Alfke]

On May 1, 2012, at 10:23 PM, Nicolas Lapomarda wrote:

I'm happy to report that your fix for issue #73 seems to have fixed my problem from March. P2P databases now seem to replicate OK. I'd be interested to know if this fixes Sven's issue as well?

Yay! Yes, it was another race condition that wasn’t fixed by the patch that fixed the previous race condition :-p The internal API of the CocoaHTTPServer library I’m using is kind of complex, and the fact that it and TouchDB run on different threads makes all sorts of interleaved calls possible.

—Jens

[sas]

Jens, that fixed my initial problem and I can now query the http
reliably. Thanks for the quick fix!

I can't yet replicate, I'm getting a 502 error from TDChangeTracker,
but I need to check if the urls and everything are in order after I've
fiddled around with my set up quite a bit in the meantime.

Cheers,
Sven

[sas]

This may be the cause: I see initial replication of some documents.
Then I'm getting a

11:28:34.734| WARNING*** : Couldn't parse line from _changes: HTTP/
1.1 000 Continue

coming from:

TDSocketChangeTracker:133
- (BOOL) failUnparseable: (NSString*)line {

or upstream:

TDSocketChangeTracker:185
// Read the HTTP response status line:
if (![line hasPrefix: @"HTTP/1.1 200 "])
[self failUnparseable: line];

I'm getting this even when I start out from empty databases. Http
status code of 000 looks odd, doesn't it?

[Jens Alfke]

On May 2, 2012, at 2:44 AM, sas wrote:

11:28:34.734| WARNING*** : Couldn't parse line from _changes: HTTP/
1.1 000 Continue

There is an actual HTTP status code “100 Continue”, though I’ve never seen it in real life.

What upstream CouchDB server are you talking to?

Is there an HTTP proxy in between?

—Jens

[sas]

Jens, that's the iPad Simulator talking to iPad, both running the same
code: TouchDB with CocoaHTTPServer driven TDListener. They're both on
the same WiFi and I initiate a single direction pull: One device
chooses the other and starts a pull replication. Note it says
"000" (triple zero), not "100".

I didn't have time today to investigate further but I'll report back
once I find out more, hopefully tomorrow.

Nico, you haven't seen anything like this in your peer to peer
replication, have you?

BTW I'm running

CouchCocoa 9a3ada3a409dcc8020a725bc632b5033f71abc7b (Wed Apr 11)
TouchDB 69e399e71d54a24c0b9eabbdc3914f1657e4783a (Tue May 1)

Cheers,
Sven

[Nicolas Lapomarda]

Sven,

I haven't seen this issue (yet) so unfortunately I can't really help you here. Will keep an eye on this thread and post if I come across the same issue in the near future.

Nico

[sas]

Ok, I've found the source of the problem and a work-around. The status
code 0 originates in HTTPConnection.m:1174:

// Default status code: 200 - OK
NSInteger status = 200;

if ([httpResponse respondsToSelector:@selector(status)])
{
status = [httpResponse status];
}

Debugger output at this point:

(lldb) po httpResponse
2012-05-04 11:37:04.147 App[9404:707] >>>>>> 1 docs being updated
(NSObject *) $5 = 0x1de81b20 Response[GET /app_test/_local/
bfca65ed062d9d28d05dd2e37eceda78c31e7914]
(lldb) po [httpResponse status]
(id) $6 = 0x00000000 <nil>

What's interesting is that httpResponse is of NSObject, not
TDResponse. If I read this correctly, it's a response that
CocoaHTTPServer generates, not one that comes from your override in
[TDHTTPConnection httpResponseForMethod:URI:].

When I assume that the "true" default of status should be 200 and
therefore change HTTPConnection accordingly, like so:

if ([httpResponse respondsToSelector:@selector(status)] &&
[httpResponse status] != 0)
{
status = [httpResponse status];
}

I find that replication works for me :)

I'm still puzzled why it works for Nico but not me. My CocoaHTTPServer
version is 37a78143ac3401e1f2da6973bbdf8e79ca9db742 (Wed Apr 25).

I'll ask Robbie Hanson if this could be a bug in CocoaHTTPServer.

Cheers,
Sven

[sas]

For reference: https://github.com/robbiehanson/CocoaHTTPServer/issues/22

On May 3, 7:03 am, Nicolas Lapomarda <n...@conceited.net> wrote:

[Jens Alfke]

On May 4, 2012, at 3:13 AM, sas wrote:

(lldb) po httpResponse
2012-05-04 11:37:04.147 App[9404:707] >>>>>> 1 docs being updated
(NSObject *) $5 = 0x1de81b20 Response[GET /app_test/_local/
bfca65ed062d9d28d05dd2e37eceda78c31e7914]
(lldb) po [httpResponse status]
(id) $6 = 0x00000000 <nil>

What's interesting is that httpResponse is of NSObject, not TDResponse.

I think the “NSObject” in the lldb output is the static type, not the actual class of the object. You can get the actual class by entering

po [httpResponse class]

I’m pretty sure it’s a TDHTTPResponse (which is a class of mine that’s basically glue between TDResponse and HTTPResponse.) And that class has a -status method that just returns the status from the TDResponse.

TDResponse does return 0 for status until the router has a response ready. So I think this is (another) race condition in the listener, especially because the HTTPConnection calls -status on its own thread, which is not the same thread the router is running on. Fuck, I hate threads. :(

I’ll try to look at this today. (I’ve accumulated a big pile of pending patches in my Couchbase Server work that haven’t been reviewed yet, so that gives me time to focus on mobile stuff for a bit!)

—Jens

[sas]

Hm, I thought po would report the proper type and I could swear I had
a breakpoint in [TDResponse status] that wasn't hit. Or maybe I set
that later... It was a bit confusing ;) I'll retry tomorrow.

In the meantime Robbie replied in the ticket I raised and what he said
would support a race condition, I think:

"httpResponse.status shouldn't be returning zero. I'd have to take a
look at TDResponse to see what may be happening.

If TDResponse needs some async task to complete before returning
headers, there are some methods to do this. Have you filed a bug with
TouchDB?"

Cheers,
Sven

>  smime.p7s
> 6KViewDownload

[sas]

Hi Jens,

I think I found the problem. In THTTPResponse you've got

- (BOOL) delayResponeHeaders { // [sic]

when it should be

- (BOOL) delayResponseHeaders { // [sic]

(note the missing 's'). Your protocol method is never called,
therefore your thread doesn't get a chance to compute its status in
time.

When I fix this, it's working even without the patch to
CocoaHTTPServer :)

Cheers,
Sven

[sas]

I see what happened now. You were targeting CocoaHTTPServer from a
while back when it still had the typo in the protocol definition,
which is why you appended the [sic] comment. A month ago there was a
fix for this in CocoaHTTPServer:

https://github.com/robbiehanson/CocoaHTTPServer/commit/ea31279b33543fdb9ce6803ee5ff1657119c7859

which broke your class. Maybe it would be best to keep around

- (BOOL) delayResponeHeaders { // [sic]

return [self delayResponseHeaders];
}

for a while so people using older CocoaHTTPServer versions aren't
affected by fixing the typo.

Cheers,
Sven

On May 4, 11:17 pm, sas <s...@abstracture.de> wrote:

[Jens Alfke]

On May 5, 2012, at 1:22 AM, sas wrote:

I see what happened now. You were targeting CocoaHTTPServer from a
while back when it still had the typo in the protocol definition,
which is why you appended the [sic] comment. A month ago there was a
fix for this in CocoaHTTPServer:

A git submodule points to a specific commit in the target repo, so TouchDB is always using a specific version of CocoaHTTPServer regardless of what changes are made in the trunk of that repo. [For reasons such as this!] And I just checked, and the version used still has the typo in that method name.

I’m not sure how you could have gotten a newer version of CocoaHTTPServer in your TouchDB source tree. Did you change the submodule settings?

—Jens

[sas]

D'oh, of course! And now it all makes sense: I integrated TDListener
sources directly into my project, because they're not part of the
TouchDB framework. Only now I discovered the Listener framework
target, which I should have used. On top of that I made the mistake of
adding CocoaHTTPServer directly to my project (for whatever reason),
instead of at least choosing the one from my TouchDB submodule. Pile
one mistake on another for maximum effect...

I even posted the revision ids of TouchDB and CocoaHTTPServer above
but it was completely out of my mind that there could be a mismatch :(

I hope I didn't waste too much of your time with this, sorry.

Cheers,
Sven

>  smime.p7s
> 6KViewDownload

[sas]

> If you are game to publish something on how to set up p2p I think people will appreciate it.

Chris, I've written a blog post about how to set this up. Of course,
if you do things right from the start it's quite simple thanks to the
existing frameworks ;) So to make this worthwhile, I've added a
description on how to use this with bonjour advertising and discovery.
I hope someone will find that helpful!

http://feinstruktur.com/blog/2012/5/14/peer-to-peer-synching-with-touchdb.html

Cheers,
Sven

On May 5, 10:22 am, sas <s...@abstracture.de> wrote:
> I see what happened now. You were targeting CocoaHTTPServer from a
> while back when it still had the typo in the protocol definition,
> which is why you appended the [sic] comment. A month ago there was a
> fix for this in CocoaHTTPServer:
>

> https://github.com/robbiehanson/CocoaHTTPServer/commit/ea31279b33543f...