Sync Gateway and sockets

[Deep Eddie]

Hello there,

First of all,  a big thanks for bringing out a product like syncgateway which will simplify and eliminate the need for custom webservices I have been writing past couple of years to achieve the same!

I have been playing around quite a bit on the various use cases and have a question based on a note I saw in the documentation regarding SyncGateway.

See note below.

Managing TCP Connections¶

Very large-scale deployments might run into challenges managing large numbers of simultaneous open TCP connections. The replication protocol uses a “hanging-GET” technique to enable the server to push change notifications. This means that an active client running a continuous pull replication always has an open TCP connection on the server. This is similar to other applications that use server-push, also known as “Comet” techniques, as well as protocols like XMPP and IMAP.

These sockets remain idle most of the time (unless documents are being modified at a very high rate), so the actual data traffic is low—the issue is just managing that many sockets. This is commonly known as the “C10k Problem” and it’s been pretty well analyzed in the last few years. Because Go uses asynchronous I/O, it’s capable of listening on large numbers of sockets provided that you make sure the OS is tuned accordingly and you’ve got enough network interfaces to provide a sufficiently large namespace of TCP port numbers per node.

----------------------------------------------------------------------------------------

Now lets say I have a cluster of sync gateways and have a F5 loadbalancer up in front of the sync gateways.

The way I understand is that each of my iOS mobile app clients will have an open socket connection to my F5 listening to change notifications.

A few questions on this - 

• Does this socket get established as soon as i call pull() from the iOS client?
  
• And the socket stays open foreever from that time onwards?  
  
• What happens to the socket when the app is minimized or in the background?
  
• What kind of factors do i need to consider at the F5 level to be able to scale appropriately?
  
• And finally, is there a way to indicate at the client side to not keep the connection open permanently?  Any drawbacks of this?  I have a shared loadbalancer and dont want to hog on all available sockets!
  

Thanks a lot in advance and let me know if you need more background on my questions.

-D

[Jens Alfke]

> On Aug 7, 2014, at 5:33 PM, Deep Eddie <deep...@gmail.com> wrote:
>
> • Does this socket get established as soon as i call pull() from the iOS client?

It's opened as soon as the pull replication goes online, yes.

> • And the socket stays open foreever from that time onwards?

A continuous pull replication keeps the socket open. A non-continuous replication stops as soon as all the current changes from the server have been downloaded.

> • What happens to the socket when the app is minimized or in the background?

Replications pause when the app is backgrounded. The socket will be closed, and re-opened when the app next activates.

> • What kind of factors do i need to consider at the F5 level to be able to scale appropriately?

I can't personally answer that, because I'm not a server expert and don't know much about load balancers.

> • And finally, is there a way to indicate at the client side to not keep the connection open permanently? Any drawbacks of this? I have a shared loadbalancer and dont want to hog on all available sockets!

Set a polling interval on the replication object. The drawback is increased latency of getting updates from the server.

—Jens