sequence number vs timestmp

[Hao Zhang]

In RocksDB, what's difference in purpose between sequence number and timestamp? Are they both recorded in sst file (since I can only use sst_dump to show the seq number)?

[Yanqin Jin]

Sequence number is 56-bit integer assigned internally by each RocksDB instance, while timestamp is a byte array that applications can specify.

They share some common purposes, e.g. providing multi-versioned access to data. Sequence number is not optional, and will always be part of InternalKey and present in SST files. Timestamp will not be used unless enabled by the user. If not used, SST file will contain no timestamp. If timestamp is used, it will be stored as follows in each internal key.

|<user key>|[timestamp]|<value type>|<sequence>|

                                              | 8 bits           |  56 bits        |

 

It’s currently not supported to dump timestamp using ldb or sst_dump.

--
You received this message because you are subscribed to the Google Groups "rocksdb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rocksdb+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rocksdb/4854c828-309a-4f65-a00a-aeb951dae5cao%40googlegroups.com.

[Yanqin Jin]

Forgot to mention that timestamp feature is not complete and still under development.

 

From: <roc...@googlegroups.com> on behalf of Hao Zhang <kenn...@gmail.com>
Date: Sunday, August 16, 2020 at 6:06 PM
To: rocksdb <roc...@googlegroups.com>
Subject: sequence number vs timestmp

 

In RocksDB, what's difference in purpose between sequence number and timestamp? Are they both recorded in sst file (since I can only use sst_dump to show the seq number)?

Like

Comment

Share

--

[Hao Zhang]

Thanks Yanqin

I have follow up questions:

1) I observed that when I use regular put, the sequence number in SST are all 0. Only if I use transaction or batch do I see sequence number being increased. Why so?

2) Any online document on exactly how sequence number is used in transaction or batch?

Thanks

On Sunday, August 16, 2020 at 9:44:23 PM UTC-7, Yanqin Jin wrote:

Forgot to mention that timestamp feature is not complete and still under development.

 

From: <roc...@googlegroups.com> on behalf of Hao Zhang <kenn...@gmail.com>
Date: Sunday, August 16, 2020 at 6:06 PM
To: rocksdb <roc...@googlegroups.com>
Subject: sequence number vs timestmp

 

In RocksDB, what's difference in purpose between sequence number and timestamp? Are they both recorded in sst file (since I can only use sst_dump to show the seq number)?

Like

Comment

Share

--
You received this message because you are subscribed to the Google Groups "rocksdb" group.

To unsubscribe from this group and stop receiving emails from it, send an email to roc...@googlegroups.com.

[Yanqin Jin]

Maybe it’s because that sequence number will be zero’ed when data is compacted to the bottommost level? You can write a simple test to verify. Just write a couple of keys, flush the sst and dump its content. I expect the sequence numbers to be non-zero in this case.

To unsubscribe from this group and stop receiving emails from it, send an email to rocksdb+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rocksdb/9f0567ce-fb82-4a27-87c2-44669d9eb5e7o%40googlegroups.com.