Key format in SST files
26 views
Skip to first unread message
Ashwini Raina
Oct 17, 2019, 12:31:12 PM10/17/19
to MyRocks - RocksDB storage engine for MySQL
Hello,
I am using MyRocks with linkbench for research purposes. I have a question on the key value format in SST files. Based on my inspection of SST files (using ssd_dump tool), I have come across two key value formats.
Format-1 (mostly 28 byte key size and variable value size):
HEX 0000011600000000075BCD1500000000048AD97900000000048AD97A: 0106534B4F62325486DD04D36D01000000000000
Format-2 (variable key size and fixed value size):
HEX. 0000010700000000057F26E600000000075BCD15810000016D20AB697300000000057F2A71000000003D254E202020202002: 0200000003
I understand Format-1 as I can print the keys in rocksdb GetImpl() api and that matches with format-1. But Format-2 is not clear to me. Why is the key size variable in Format-2 and value size fixed? Could you point me to some code or documentation reference that clarifies this?
thanks,
ashwini
MARK CALLAGHAN
Oct 17, 2019, 12:39:51 PM10/17/19
to Ashwini Raina, MyRocks - RocksDB storage engine for MySQL
I can't answer this question, but have a lot of experience with
linkbench. Are using it from https://github.com/mdcallag/linkbench or
https://github.com/facebookarchive/linkbench? The former is active,
the latter is not.
> --
> You received this message because you are subscribed to the Google Groups "MyRocks - RocksDB storage engine for MySQL" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to myrocks-dev...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/myrocks-dev/25dfcbc3-5f81-42a2-8ecf-1250aa24e7c3%40googlegroups.com.
--
Mark Callaghan
mdca...@gmail.com
linkbench. Are using it from https://github.com/mdcallag/linkbench or
https://github.com/facebookarchive/linkbench? The former is active,
the latter is not.
> You received this message because you are subscribed to the Google Groups "MyRocks - RocksDB storage engine for MySQL" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to myrocks-dev...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/myrocks-dev/25dfcbc3-5f81-42a2-8ecf-1250aa24e7c3%40googlegroups.com.
--
Mark Callaghan
mdca...@gmail.com
Ashwini Raina
Oct 17, 2019, 12:54:37 PM10/17/19
to MyRocks - RocksDB storage engine for MySQL
Hi Mark,
Thank you for replying. Looks like I am using the inactive branch. I will switch to your repo.
After some googling I found an old facebook presentation that touches upon the key formats used in rocksdb. I am suspecting the secondary key format is the one that has variable key sizes and a fixed value field.
ashwini
On Thursday, October 17, 2019 at 12:39:51 PM UTC-4, MARK CALLAGHAN wrote:
I can't answer this question, but have a lot of experience with
linkbench. Are using it from https://github.com/mdcallag/linkbench or
https://github.com/facebookarchive/linkbench? The former is active,
the latter is not.
On Thu, Oct 17, 2019 at 9:31 AM Ashwini Raina <ashwin...@gmail.com> wrote:
>
> Hello,
>
> I am using MyRocks with linkbench for research purposes. I have a question on the key value format in SST files. Based on my inspection of SST files (using ssd_dump tool), I have come across two key value formats.
>
> Format-1 (mostly 28 byte key size and variable value size):
> HEX 0000011600000000075BCD1500000000048AD97900000000048AD97A: 0106534B4F62325486DD04D36D01000000000000
>
> Format-2 (variable key size and fixed value size):
> HEX. 0000010700000000057F26E600000000075BCD15810000016D20AB697300000000057F2A71000000003D254E202020202002: 0200000003
>
> I understand Format-1 as I can print the keys in rocksdb GetImpl() api and that matches with format-1. But Format-2 is not clear to me. Why is the key size variable in Format-2 and value size fixed? Could you point me to some code or documentation reference that clarifies this?
>
> thanks,
> ashwini
>
> --
> You received this message because you are subscribed to the Google Groups "MyRocks - RocksDB storage engine for MySQL" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to myroc...@googlegroups.com.
Yoshinori Matsunobu
Oct 17, 2019, 12:56:38 PM10/17/19
to Ashwini Raina, MyRocks - RocksDB storage engine for MySQL
Yes, the format-1 was from primary keys with three BIGINT columns, and format-2 was from secondary keys.
Here is a wiki explaining MyRocks row format.
- Yoshinori
On Oct 17, 2019, at 9:54 AM, Ashwini Raina <ashwin...@gmail.com> wrote:
Hi Mark,
Thank you for replying. Looks like I am using the inactive branch. I will switch to your repo.
After some googling I found an old facebook presentation that touches upon the key formats used in rocksdb. I am suspecting the secondary key format is the one that has variable key sizes and a fixed value field.
To unsubscribe from this group and stop receiving emails from it, send an email to myrocks-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/myrocks-dev/630fa871-9042-46a9-a1ca-7a33b0b6ec64%40googlegroups.com.
<format.png>
Ashwini Raina
Oct 17, 2019, 1:10:06 PM10/17/19
to MyRocks - RocksDB storage engine for MySQL
To view this discussion on the web visit https://groups.google.com/d/msgid/myrocks-dev/630fa871-9042-46a9-a1ca-7a33b0b6ec64%40googlegroups.com.
<format.png>
Reply all
Reply to author
Forward
0 new messages