etcd scalability

[Max Levine]

Hi All,

I am evaluating etcd to run in a production environment of about 4000 servers.

Has etcd been tested with this many nodes?

I was reading about the standby mode, so I understand that this is the way to scale a large cluster is to run the majority of nodes in standby mode. What is a good number of peers to setup, so as not to put too much stress on the leader?

 

 what is the recommended cluster discovery method in such a large cluster? discovery URL or harcode a list of -peers to all followers?

 

what are the limitations on the amount of data that can be stored?

 any other scalability issues that I need to think about?

[Blake Mizerany]

These should help:

https://coreos.com/docs/cluster-management/scaling/etcd-optimal-cluster-size/

https://coreos.com/docs/cluster-management/debugging/etcd-tuning/

https://coreos.com/docs/cluster-management/setup/etcd-cluster-discovery/

--
You received this message because you are subscribed to the Google Groups "etcd-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to etcd-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Pradeep Kiruvale]

Hi Max,

 I have similar use case as yours. I wanna use etcd as the distributed key value store. Did you do some kind of
a benchmarking?  What is your opinion on this?

Regards,
Pradeep

[Max Levine]

Hi Pradeep, 

I didn't do any bench marking yet - working on another project now, but I would like to see any bench-marking results that you may run. with 9 nodes as the maximum suggested cluster size i have some concerns about scaling the number of clients

--

[Pradeep Kiruvale]

Hi Max,

Thanks for the reply. Sure, I will let you know if I do any progress on that work.

Regards,
Pradeep