continuous replication limitation

[frank...@gmail.com]

Hi,

I'm using couchbase lite in a mobile app. Just wondering:

1. is there any limitation if I create continuous replication to sync data between mobile device and server?
   
2. How many concurrent continuous replication can be handled by one couchDB or coubchbase server?
   
3. If reach the limit, what will happen on mobile side and server side?
   

Thanks! 

Frank

[Jens Alfke]

On Nov 2, 2015, at 1:08 PM, frank...@gmail.com wrote:

I'm using couchbase lite in a mobile app. Just wondering:

• is there any limitation if I create continuous replication to sync data between mobile device and server?

• How many concurrent continuous replication can be handled by one couchDB or coubchbase server?

There’s not an easy answer to that; it depends on a lot of variables like the server hardware, how busy the database is, etc.

The hardest limit is the number of open TCP sockets the server can handle. This is a general problem that connection-oriented protocols run into (IMAP, XMPP, Comet, etc.) — it’s often called the “C10K Problem”. The good news is that Erlang and Go (the languages CouchDB and Sync Gateway are written in) are both good at handling massive I/O concurrency. But you do need lots of Ethernet adapters to support zillions of sockets on one machine!

But generally, long before you hit limits like that, you’ll run into CPU, RAM and I/O pressure on the server. Sync Gateway and Couchbase Server support horizontal scaling — you can plug in any number of servers and rig up a load-balancer to distribute requests among them. (CouchDB can do something similar but you have to rig up all of the servers to replicate their databases with each other, and I have doubts about the scalability of that. There’s also a variant called BigCouch which is designed to scale higher.)

• If reach the limit, what will happen on mobile side and server side?

Most likely the clients will see longer and longer latency before receiving updated documents, then eventually timeouts, then eventually socket errors like connection-refused or connection-dropped.

—Jens

[frank...@gmail.com]

Thanks Jens! Your answer is pretty what I expected. Looks like I have to figure out the logic when pull and push data by myself instead of lazzzzzzzzzy just using continuous replication. : )

[frank...@gmail.com]

Hi Jens, 

Is there any benchmark related to continuous replication? I just want to get a brief idea how many concurrent clients a couchbase server can handle (for example a simple chat room app). Let's say, one aws EC2 m3.medium instance for couchbase enterprise edition. https://aws.amazon.com/marketplace/pp/B013XDD6RE/ref=dtl_recsim_B013XDO1B4_B013XDD6RE_2

Thanks,

Frank

[Todd Freese]

Jens might be able to provide more information. But the docs/wiki used to give the VERY approx number of 5000 clients per sync gateway node. But there are a ton of things that can affect that number a lot. So there is no real benchmark as every app will be different.

T

[Jens Alfke]

On Nov 3, 2015, at 10:48 AM, Todd Freese <to...@filmworkers.com> wrote:

Jens might be able to provide more information. But the docs/wiki used to give the VERY approx number of 5000 clients per sync gateway node. But there are a ton of things that can affect that number a lot. So there is no real benchmark as every app will be different.

Yes, that was a figure we came up with as a guideline, using a test app that generates an artificial client workload, and some fairly typical server hardware.

The Sync Gateway team are adding some major architectural improvements in version 1.2 that will greatly increase scalability. That should be released by the end of the year, I think.

—Jens