Maximum number of rows per partition

[Piyush Katariya]

Thank you for giving us native Cassandra alternative. 

I have few queries about partition size and number of rows in it.

How many number of rows per partition are allowed beyond which ScyllaDB might not be performant ?
Is it same as DSE Cassandra distribution ?
Also when single row in partition data get updated (INSERT, UPDATE, DELETE), does it replicate whole partition or just row diff  changes ?

Ref:- https://docs.datastax.com/en/dse-planning/doc/planning/planningPartitionSize.html

[Avi Kivity]

On 10/30/2017 05:45 AM, Piyush Katariya wrote:

Thank you for giving us native Cassandra alternative. 

I have few queries about partition size and number of rows in it.

How many number of rows per partition are allowed beyond which ScyllaDB might not be performant ?

It really depends on your workload and queries, there is no single answer.

Is it same as DSE Cassandra distribution ?

It's not the same, but the general sentiment applies.

Also when single row in partition data get updated (INSERT, UPDATE, DELETE), does it replicate whole partition or just row diff  changes ?

No, only the updated cells are replicated.

Ref:- https://docs.datastax.com/en/dse-planning/doc/planning/planningPartitionSize.html

--
You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.
To post to this group, send email to scyllad...@googlegroups.com.
Visit this group at https://groups.google.com/group/scylladb-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/4340a7ad-ea5f-4af6-94e0-f7e5412f8118%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Piyush Katariya]

Thanks Avi.

About the number of rows per partition, suppose you have a EVENT table with fields ( day, sensor, map of sensor_attributes, map of sensor_status, event_time) 

This event table is used to store readings from various sensors for given day. Day being a primary key and event_time and sensor being secondary keys.

So that makes a Day ( for example 13/10/2009) a partition key. And there will be at least 5 million records for given Day against all IoT sensors we have.

Is it viable approach ?

Also I wanted to know that Cassandra and ScyllaDB are good at random hash partitioning , is there any way to use ScyllaDB for range partitioning due to its performance characteristics?

On 31-Oct-2017 01:45, "Avi Kivity" <a...@scylladb.com> wrote:

On 10/30/2017 05:45 AM, Piyush Katariya wrote:

Thank you for giving us native Cassandra alternative. 

I have few queries about partition size and number of rows in it.

How many number of rows per partition are allowed beyond which ScyllaDB might not be performant ?

It really depends on your workload and queries, there is no single answer.

Is it same as DSE Cassandra distribution ?

It's not the same, but the general sentiment applies.

Also when single row in partition data get updated (INSERT, UPDATE, DELETE), does it replicate whole partition or just row diff  changes ?

No, only the updated cells are replicated.

Ref:- https://docs.datastax.com/en/dse-planning/doc/planning/planningPartitionSize.html

--
You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-users+unsubscribe@googlegroups.com.

[Avi Kivity]

I recommend using a partition key of (day, sensor) and a clustering key of event_time, instead. This way your partitions will be bounded even if you have many sensors, and writes will be distributed across the cluster.

e.g. PRIMARY KEY ((day, sensor), event_time).

[ddorian43]

Also, no plan/issue for range sharding (cockroachdb, hbase style).

[Piyush Katariya]

Thanks.

I am aware that Range sharing is not supported natively. But i am wondering instead of adopting whole other database like Hbase or MaprDB to support range sharding, can I do it in ScyllaDB on clustering key with say at least few million records in given partition ? Would it be a overkill ?

...