Questions about Limitations

[Trevor Huey]

My company is pushing the limits of MySQL right now, and we're considering using Vitess to help us scale to the next level. After reading over the documentation, the following limitations make me nervous:

1. Vitess does not support other types of DDL statements, such as those that affect stored procedures or grants.
2. ALTER statements are only allowed if the table on the shard's master tablet has 100,000 rows or less.
3. For all other statements, the table on the shard's master tablet must have 2 million rows or less.

I have the following questions:

1. Are there plans to add better support for these? 

2. For 2 and 3, are these hard-coded limits, or are they configurable? 

3. What's the reasoning behind these limits?

4. We're considering a multi-tenant approach. We can either create a shard per customer, or we could create a keyspace/database per customer. Would Vitess work well for the latter of these approaches? Is there a recommended approach? If we did have a separate keyspace/database per customer, we could effectively get around the limitations that are listed above.

[Sugu Sougoumarane]

On Thu, Jan 28, 2016 at 9:49 AM, Trevor Huey <thue...@gmail.com> wrote:

My company is pushing the limits of MySQL right now, and we're considering using Vitess to help us scale to the next level. After reading over the documentation, the following limitations make me nervous:

1. Vitess does not support other types of DDL statements, such as those that affect stored procedures or grants.
2. ALTER statements are only allowed if the table on the shard's master tablet has 100,000 rows or less.
3. For all other statements, the table on the shard's master tablet must have 2 million rows or less.

I have the following questions:

1. Are there plans to add better support for these?

We could look at allowing some constructs as pass-through. But I don't think we'll support them natively in vitess, like understanding what a stored procedure does and reacting accordingly.

For vitess sharding to work, it has to control where data goes. Stored procedures are capably of circumventing this control, and could therefore break vitess's assumptions. This is why we don't allow these.

 

 

 

2. For 2 and 3, are these hard-coded limits, or are they configurable? 

I believe so. Even if they're not, we can trivially make them configurable.

 

3. What's the reasoning behind these limits?

These are just sanity checks to prevent accidental outage. If a DDL affects too many rows, it will lock the table and cause everything else to fail.

 

4. We're considering a multi-tenant approach. We can either create a shard per customer, or we could create a keyspace/database per customer. Would Vitess work well for the latter of these approaches? Is there a recommended approach? If we did have a separate keyspace/database per customer, we could effectively get around the limitations that are listed above.

The downside of a multi-tenant approach is that you need a look-aside table that tells you which shard each user is. So, it's an extra round-trip. Also, if a single user gets too big, you may have to shard that user's database. Then you'll need the user id and another key to find where the data is.

You many be able to overcome some of these limitations through caching, but you could find yourself cornered if the system grows in ways you don't expect.

Having said that, the Vitess architecture is somewhat agnostic to sharding scheme. Under the covers, we are range-base sharded. But you're allowed to define how the data maps to this range. Such mapping can be backed by a lookup table, or even custom code. You can see documentation here: https://github.com/youtube/vitess/blob/master/doc/VTGateV3Features.md.

 

--
You received this message because you are subscribed to the Google Groups "vitess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vitess+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.