Re: TouchDB: continuous replication not working

[Robin De Mol]

Snooping around a bit I think this bug has already been reported (and is believed to be due to the libraries of apple). Is there any chance there will be a fix or workaround for this soon? I've read about the commit adding a "."-suffix to the _changes feed listener, but won't this give problems with any certificates when using ssl?

I suppose that last problem could be fixed by using a self signed certificate. Is it possible to add certificates to a white list for TouchDB-iOS?

Thanks,

Robin

[Jens Alfke]

On Jul 26, 2012, at 5:09 AM, Robin De Mol <arche...@gmail.com> wrote:

Snooping around a bit I think this bug has already been reported (and is believed to be due to the libraries of apple). Is there any chance there will be a fix or workaround for this soon?

It should be fixed already. Which version of TouchDB are you using?

I've read about the commit adding a "."-suffix to the _changes feed listener, but won't this give problems with any certificates when using ssl?

It's working with SSL, but I'm seeing some auth calls to my delegate objects that make me suspect Cocoa is warning me about an SSL hostname mismatch (but allowing the connection anyway). I need to look into this, since real hostname mismatches should be treated as errors for best security.

—Jens

[Robin De Mol]

Jens

I am using the latest version of the github branch SyncPoint. In my application I am still receiving timeouts for several GET requests. I assume they are stuck behind the _changes feed listener. Meanwhile I chose for a temporary "workaround" by implementing repeating timers that trigger non-continuous replication but this solution doesn't please me very much.

For the SSL bit: in case the hostname mismatch would be treated as an error in the future, what would you advise to keep everything working in the future? Or will the connection still be allowed, regardless of the error?

Thanks,

Robin

[Jens Alfke]

On Jul 26, 2012, at 10:23 AM, Robin De Mol <arche...@gmail.com> wrote:

I am using the latest version of the github branch SyncPoint. In my application I am still receiving timeouts for several GET requests. I assume they are stuck behind the _changes feed listener.

Dang. How long do these timeouts take after the request is posted? (You can find out by enabling 'RemoteRequest' logging; see the wiki for details.)

If the request is getting stuck, I would expect a timeout after 60 seconds. There are other cases where I've seen timeouts after longer intervals like 4 minutes; in those cases the request must be getting sent to the server, but for some reason the data stops coming back after a while; I was tentatively blaming this on the remote server but no one has gotten to the bottom of the problem yet.

For the SSL bit: in case the hostname mismatch would be treated as an error in the future, what would you advise to keep everything working in the future? Or will the connection still be allowed, regardless of the error?

Don't worry about it; since TDConnectionChangeTracker is the one modifying the hostname behind the scenes, it will also handle the hostname mismatch automatically. What I need to fix is to detect other, genuine hostname mismatches (as from man-in-the-middle attacks) and treat those as errors.

—Jens

[Robin De Mol]

Jens

I'm afraid I can't tell you the exact timeout duration at the moment as my development machine is at the office and right now I just got home, but I will be sure to post it here tomorrow. However I had all kinds of logging on and I can tell you 100% sure that the timeouts were a lot faster than 4 minutes. Judging from my memory I would say anywhere from 30 seconds to a minute sounds correct.

That's good to hear about the SSL! One last question on the topic: is it possible to add a white list of trusted certificates somehow or does TouchDB not support this?

Thanks,
Robin

[Jens Alfke]

On Jul 26, 2012, at 11:05 AM, Robin De Mol <arche...@gmail.com> wrote:

That's good to hear about the SSL! One last question on the topic: is it possible to add a white list of trusted certificates somehow or does TouchDB not support this?

It's not specific to TouchDB. Use the Keychain APIs to do this.

—Jens

[Robin De Mol]

Jens

I'm back at the office now. The timeouts happen in less than one minute time.

I've found out what the problem is though. During development, I was connecting to a machine in the local network running CouchDB 1.2.0 and I was using the IP address to connect to it. Hence the fix appending a dot didn't work for me. I set up a DNS rule and changed my code to use the newly created hostname. I then noticed the replication no longer timed out.

Overjoyed I started testing my replicators right away but noticed that continuous replication still wasn't working as I had though when using https with a certificate. After changing an attachment on the server, I logged my active tasks on the device and saw the status was still on idle: my suspicion is that the _changes feed isn't being read properly (due to the dot and the combination with https). For now I'm going back to my temporary workaround using timers firing single replicators which seems to work just fine.

Thanks,

Robin

[Jens Alfke]

On Jul 27, 2012, at 1:20 AM, Robin De Mol <arche...@gmail.com> wrote:

I was using the IP address to connect to it. Hence the fix appending a dot didn't work for me.

Oh crap. Another unexpected side effect of this kludgy workaround. I wonder if there's an alternate way I can mangle a numeric hostname; maybe by prepending a "0" to one of the numbers. (Filed as #124.)

Overjoyed I started testing my replicators right away but noticed that continuous replication still wasn't working as I had though when using https with a certificate. After changing an attachment on the server, I logged my active tasks on the device and saw the status was still on idle: my suspicion is that the _changes feed isn't being read properly (due to the dot and the combination with https).

I would really like to see what happens in the logs when you perform this experiment, with logging enabled for SyncVerbose, ChangeTracker, RemoteRequest. (Include the time at which you made the change on the server.)

—Jens

[Robin De Mol]

Jens

(Oops, I accidently a verb: noticed that continuous replication still wasn't working as I had expected though when using https)

I noticed the issue report you filed. Thanks for looking into it. I will try to deliver the logs as soon as I can though it might not be before monday as the entire configuration for development is set up on a local network at my office. I'll get back to this asap.

Thanks,
Robin

[Robin De Mol]

Jens

After extensive testing, I wasn't able to reproduce the errors I had with continuous replication; everything seems to work perfectly fine now. I suspect the problem I encountered before had to do with an unclean project or caches somewhere, or any other type of artefact data that was hampering the workflow.

Robin