etcd- configuration as a service

[hyginous neto]

Hi Team,

We are planning to use ETCD as a datastore for storing application configuration for the "Configuration as a Service" usecase.

Before moving ahead with ETCD, we want to make sure the overall data size it can support.

1. Whats the maximum overall datasize a ETCD cluster can support?

2. Whats the maximum datasize a ETCD cluster can support for the "Value"?

3. Whats the max memory size limit for the ETCD cluster?

We have a requirement to store 1 GB of data in the cluster. Please confirm.

[Gyu-Ho Lee]

On Monday, December 5, 2016 at 4:57:02 PM UTC-8, hyginous neto wrote:

Hi Team,

We are planning to use ETCD as a datastore for storing application configuration for the "Configuration as a Service" usecase.

Before moving ahead with ETCD, we want to make sure the overall data size it can support.

1. Whats the maximum overall datasize a ETCD cluster can support?

Default storage size limit is 2GB, configurable up to 8GB with --quota-backend-bytes flag.

 

2. Whats the maximum datasize a ETCD cluster can support for the "Value"?

etcd limits the size of request in 1.5MB

https://github.com/coreos/etcd/blob/master/etcdserver/v3_server.go#L40-L44

3. Whats the max memory size limit for the ETCD cluster?

 

In our benchmark that writes 2 million keys with 1,000 clients (data size 1GB when it finishes), the memory usage was 500 ~ 600MB max.

We will share all these data and details soon.

[Xiang Li]

be sure to use etcd3 API. 1gb data is fine.

--
You received this message because you are subscribed to the Google Groups "etcd-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to etcd-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[hyginous neto]

HI xiang.li,

Thanks for reply.

Gyu-Ho Lee mentioned i can use upto 8 GB. is the same applies to earlier version of etcd? 

I cannot find any documention mentioning the size restrictions 

To unsubscribe from this group and stop receiving emails from it, send an email to etcd-dev+u...@googlegroups.com.

[Xiang Li]

On Wed, Dec 7, 2016 at 1:41 PM, hyginous neto <hygi...@gmail.com> wrote:

HI xiang.li,

Thanks for reply.

Gyu-Ho Lee mentioned i can use upto 8 GB. is the same applies to earlier version of etcd? 

No. etcd3 only.

 

To unsubscribe from this group and stop receiving emails from it, send an email to etcd-dev+unsubscribe@googlegroups.com.

[hyginous neto]

Thanks Gyu, what do you think about ETCD vs other NoSql solutions like MongoDB (which falls under CP under CAP theorem),which should be used as datasource for Configuration as a Service.

[Gyu-Ho Lee]

etcd is also CP, with high Availability.

I am not a MongoDB expert, and it depends on the use case.

What's your use case? Or expected data size and workload?

--

[hyginous neto]

Hi Gyu,

Our usecase is "Configuration as a Service" for Enterprise and we need store around 3-4 GB of (Applications configuration data) data.

[hyginous neto]

Thanks Xiang. 

Wnated to check with you too. We have a usecase "Configuration as a Service" for enterprise where we need to store around 3-4 GB of data. Please could you let us know your thoughts if ETCD is good for it?

[Gyu-Ho Lee]

I would simulate the 3-4GB workloads first, and see how etcd performs in your use-case.

You can try our benchmark CLI https://github.com/coreos/etcd/tree/master/tools/benchmark for stress-testing the etcd cluster.