Maximum number of tables per cluster

[David Lemieux]

Hi,

We are running a multi-tenant platform and we prefer to create separate tables per client.

We are having problems creating lots of tables on Cassandra which breaks down in the low hundreds.

The over simplification is this: adding a new table requires a consensus on the schema and getting a consensus is taking longer and longer as you add tables.

Eventually we can't get a consensus anymore so we can't add new tables.

Does anybody have experience with that kind of problem on ScyllaDB?

What is the maximum number of tables that a cluster can support before breaking down?

Thanks

/D

[Glauber Costa]

Hi David,

I have never seen this happening in our production deployments. I can think of at least two Enterprise customers of Scylla that I closely monitor

(unfortunately I can't disclose who they are) that have 100 - 200 tables and everything works well.

The trick here is that schema agreement is influenced by the number of boxes you have - since all nodes have to agree on the changes. And Scylla

makes it possible to extract performance from very large machines - making it economical to reduce the machine count by a lot. Both those customers I mentioned are using machines with at least 32 cores - so you can have dozens of those instead of hundreds. There are probably other Open Source users that I don't know about with similar use cases.

To be clear: there are various kinds of implications of using that many tables, and between using few of them and many, few will definitely yield more performance, less problems, etc. But it ultimately works - speaking from production experience - so if that is a requirement then go for it.

What I'd say is, use as few nodes as you can (by using as large machines as you can), and try it out.

--
You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-users+unsubscribe@googlegroups.com.
To post to this group, send email to scylladb-users@googlegroups.com.
Visit this group at https://groups.google.com/group/scylladb-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/b08322c1-e360-408c-9e0b-34754954df2b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Lemieux]

Thanks for the info and the quick reply.

/D

On Sunday, April 15, 2018 at 11:42:23 PM UTC-4, Glauber Costa wrote:

Hi David,

I have never seen this happening in our production deployments. I can think of at least two Enterprise customers of Scylla that I closely monitor

(unfortunately I can't disclose who they are) that have 100 - 200 tables and everything works well.

The trick here is that schema agreement is influenced by the number of boxes you have - since all nodes have to agree on the changes. And Scylla

makes it possible to extract performance from very large machines - making it economical to reduce the machine count by a lot. Both those customers I mentioned are using machines with at least 32 cores - so you can have dozens of those instead of hundreds. There are probably other Open Source users that I don't know about with similar use cases.

To be clear: there are various kinds of implications of using that many tables, and between using few of them and many, few will definitely yield more performance, less problems, etc. But it ultimately works - speaking from production experience - so if that is a requirement then go for it.

What I'd say is, use as few nodes as you can (by using as large machines as you can), and try it out.

On Sun, Apr 15, 2018 at 10:09 PM, David Lemieux <lem...@gmail.com> wrote:

Hi,

We are running a multi-tenant platform and we prefer to create separate tables per client.

We are having problems creating lots of tables on Cassandra which breaks down in the low hundreds.

The over simplification is this: adding a new table requires a consensus on the schema and getting a consensus is taking longer and longer as you add tables.

Eventually we can't get a consensus anymore so we can't add new tables.

Does anybody have experience with that kind of problem on ScyllaDB?

What is the maximum number of tables that a cluster can support before breaking down?

Thanks

/D

--
You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.
To post to this group, send email to scyllad...@googlegroups.com.

[Dor Laor]

On Mon, Apr 16, 2018 at 1:17 PM, David Lemieux <lem...@gmail.com> wrote:

Thanks for the info and the quick reply.

Just give it a shot, if there is a problem, we'll fix it!

 

To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-users+unsubscribe@googlegroups.com.
To post to this group, send email to scylladb-users@googlegroups.com.

Visit this group at https://groups.google.com/group/scylladb-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/417778e2-adce-4132-ad3a-b9f1b636aef3%40googlegroups.com.