Scalability in a cluster ?

[Dorian Hoxha]

So::

1. max number of nodes in a cluster (ex: rethinkdb supports up to ideally 20)
1.1. Does the scalability increase by 1 with each node (assuming even usage on all ranges) or something <1 like in postgresql-xl ?
2. max storage per node (some kind of limit on the lower storage layer, ex like pg has)
3. max rows/kv per node/range/table
4. max number of columns in a table (ex: we store column-id as 1-byte so 256 max or something similar)
5. max number of tables (ex: more than x tables it's an antipattern since each table has overhead in ram)
6. max number of ranges (each range has overhead, does each node know the position of each range, or only range.ranges?)

These I think should also be in the faq/docs ?

[jesse]

Hi Dorian,

In general, CockroachDB is designed to support thousands of nodes, and there are no hard limits in terms of max nodes, max storage per node, max columns in a table, etc. There may be limits in practice, though. For example, all table metadata is currently held in memory, so you don’t want to have too many tables. As we get closer to production readiness, we'll be doing larger scale tests that will let us discover the practical limits of the system, and we'll be sure to document those. 

Best,
Jesse

[Peter Mattis]

Hi Dorian,

On Sun, Jun 12, 2016 at 11:17 AM, Dorian Hoxha <dorian...@gmail.com> wrote:

So::

1. max number of nodes in a cluster (ex: rethinkdb supports up to ideally 20)

CockroachDB is designed to scale to hundreds and even thousands of nodes. In practice, 10s of nodes is more realistic at this time. We're working hard at making larger clusters work smoothly.

 

1.1. Does the scalability increase by 1 with each node (assuming even usage on all ranges) or something <1 like in postgresql-xl ?

The scalability depends on your workload, but the short answer is that you're going to see <1 performance increase for each additional node if you have any sort of realistic schema with multiple tables and indexes. I don't have a more concrete answer for you at this time, but agree it would be good to provide guidance here.

 

2. max storage per node (some kind of limit on the lower storage layer, ex like pg has)

Postgres has a limit on the disk storage it will use? There is no hardcoded limit in CockroachDB.

 

3. max rows/kv per node/range/table

There is no limit on the number of rows/kvs per node or table. Ranges hold up to 64MB of row/kv data, but will split automatically. The 64MB size is configurable, but you shouldn't set it larger right now as larger ranges have some problems which we're working to resolve (i.e. certain operations use memory proportional to the range size). This also implies that row/kv data should be significantly smaller than a range. 

 

4. max number of columns in a table (ex: we store column-id as 1-byte so 256 max or something similar)

There is no hardcoded limit. Column IDs are 32-bit variable length encoded integers. 

 

5. max number of tables (ex: more than x tables it's an antipattern since each table has overhead in ram)

There is no hardcoded limit, though each table will consume a small amount of memory (a few KB) on every node.

 

6. max number of ranges (each range has overhead, does each node know the position of each range, or only range.ranges?)

There is no hardcoded limit. The range indexing information is cached on demand. The design here is similar to BigTable/HBase and is intended to scale to very large numbers of ranges. 

 

These I think should also be in the faq/docs ?

 Agreed that the above should be in faq/docs. 

--
You received this message because you are subscribed to the Google Groups "CockroachDB user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cockroachdb-us...@googlegroups.com.
To post to this group, send email to cockroac...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cockroachdb-users/182bc3b3-a4b0-4af6-9329-2483067e8b71%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Spencer Kimball]

+jesse for adding these answers to the FAQ.

To view this discussion on the web visit https://groups.google.com/d/msgid/cockroachdb-users/CANKgOKh-3YFnR9-SCvzV9uK%3DEctocEhOPq09bfRUnU53SGy7-w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Spencer Kimball | Co-founder & CEO

Cockroach Labs

Help us Make Data Easy!

[Jesse Seldess]

Thanks, Spencer. Created https://github.com/cockroachdb/docs/issues/380. I'll work these into the FAQ soon.

[Dorian Hoxha]

@ Peter, pg limits:
Limit

	Value
Maximum Database Size	Unlimited
Maximum Table Size	32 TB
Maximum Row Size	1.6 TB
Maximum Field Size	1 GB
Maximum Rows per Table	Unlimited
Maximum Columns per Table	250 - 1600 depending on column types
Maximum Indexes per Table	Unlimited

[Peter Mattis]

Thanks, Dorian.

I wonder where the Postgres table size limitation comes from given that tables can contain an unlimited number of rows.

--
You received this message because you are subscribed to the Google Groups "CockroachDB user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cockroachdb-us...@googlegroups.com.
To post to this group, send email to cockroac...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cockroachdb-users/e98ca90f-ecdf-40ba-8745-77d66433e1de%40googlegroups.com.

[Dorian Hoxha]

Pg rows are stored in 8kb(default) pages and there can be up to 32bit signed integer pages. Pages can be up to 64kb in size

[Peter Mattis]

Got it. Thanks for the details.