Benchmark

[Buğra Çakır]

Hi All,

Do we have any performance comparison between

SWC-DB and HBase/OpenTSDB/KairosDB/InfluxDB for time series

data ingestion and query ?

Br,

Bugra

[Alex Kashirin]

Hello,

I have made a small search of performance benchmarking cases and found couple:

1. https://scylladb.medium.com/kairosdb-and-scylla-a-time-series-solution-for-performance-and-scalability-f351a5cfcd98
2. https://tdengine.com/performance-comparison-of-tdengine-and-influxdb/

There are several benchmarking options available of SWC-DB but these ain't specific for a time-series benchmarking.

The available workload tools are:

• SWC-DB load generator
• SWC-DB regression test example

-- both can represent a time-series workload but again not an explicit case for a time-series benchmarking

I'll be glad to know the kind dataset of metrics you would like to benchmark,

as by benchmark-case(1), its dataset is defined as such https://github.com/scylladb/scylla-code-samples/blob/737432904a7a370544e01289758d4449b101a0d4/deploy_kairosdb/1_better_insert_data_25K.py#L23

My wonder is whether a SENSOR_ID, TIME, VALUIE and STATUS is enough.

The main questions or/and Arguments a time-series benchmarking tool, I think, should include:

• The definitions of a SENSOR:

1. 1. number of sites
   2. number of sub levels in a site
   3. number of Devices on a site
   4. number of metrics/sensors each Device has
   5. number of samples in a given time-interval
   6. whether some Sensors measure volume in a time interval and require aggregation

• The approaches of SWC-DB data modeling:

1. 1. preference of access by site, device, sensor(metric) or time, whereas sequential writes by a unique property has better predetermined distribution over blocks and ranges
   2. Single Column for all or a Column for each Site/Device
   3. a COUNTER or a SERIAL column-type: Both can do math OPs in a case of aggregation for a time-period is required, whereas the SERIAL lets to store number of metrics(sensor) for a device in a single cell but SERIAL requires an initial cell for the time base aggregation.

• An implementation dependent:
  • Proprietary C++ client
  • Over Thrift Service: Python, Java, Ruby and rest of available https://github.com/apache/thrift/blob/master/LANGUAGES.md, the programming-languages available at https://github.com/kashirin-alex/swc-db/tree/v0.5.10/src/thrift/swcdb/thriftgen-0.17.0 are ready to date 

It will be important to know your interests of benchmark took capability.

Thank You,

Alex Kashirin