Pre-Splitting TSDB Tables

325 views
Skip to first unread message

Jonathan Creasy

unread,
May 8, 2013, 6:56:04 PM5/8/13
to open...@googlegroups.com
I put this in a previous thread but I wanted to send a message
directly about it to make it easier to find later.

We pre-split our tables, usually into 256 regions. I have seen lots of
questions about pre-splitting so I wanted to share our method.

I should clean it up a little for public consumption but I have not so
here it is:

https://gist.github.com/johann8384/5544290

tsuna

unread,
May 8, 2013, 7:08:16 PM5/8/13
to jcr...@box.com, open...@googlegroups.com
Thanks for sharing! I was going to post the following on GitHub, but
their comment system is broken right now and is giving me a 404
whenever I attempt to submit a comment:

ah, I was wondering why you had a `disable` / `alter`, instead of
doing everything straight in the `create` command. I suppose the
`create_table.sh` script upstream could be enhanced to:
* Allow passing extra custom arguments (like your `REPLICATION_SCOPE`
or `TTL` above)
* Allow passing an arbitrary number of splits in argument and having
the script automatically figure out sensible region boundaries to
pre-split on during the `create` command.

--
Benoit "tsuna" Sigoure

Jonathan Creasy

unread,
May 8, 2013, 7:15:09 PM5/8/13
to tsuna, open...@googlegroups.com
Yes, it has always been my intention to improve the upstream and
submit some of this but I haven't had time.

I figured it was time to share what I had done as is. :)

-Jonathan

Bryan Zubrod

unread,
May 10, 2013, 10:18:23 AM5/10/13
to open...@googlegroups.com, jcr...@box.com, j...@box.com
Very helpful in practice and education.  Thank you!

- Bryan

Chi Hoang

unread,
May 13, 2013, 6:02:56 PM5/13/13
to open...@googlegroups.com, tsuna
Thanks for sharing this.  I used these scripts, and it worked nicely.
Reply all
Reply to author
Forward
0 new messages