[Feature Request] CompactionFilter supports a new decision type of kKeepAndSkipUntil

6 views
Skip to first unread message

Duo Li

unread,
Jan 6, 2026, 9:24:18 AM (5 days ago) Jan 6
to rocksdb
Hi all,

Our legacy system uses rocksdb v3.11 until now because CompactionFilterV2 is needed.

Here is our key schema

| user_key | user_table | timestamp |

For the same user_key and user_table, there are many different timestamps. The goal of GC is to keep only one key of which the timestamp is less than a specific timestamp.

For example, there are 4 keys, the relationship of timestamp1 to timestamp4 is ts1 < ts2 < ts3 < ts4 and keys are compared by lexicographical order

key1 | user_key1 | user_table1 | timestamp1| <-drop key2 | user_key1 | user_table1 | timestamp2| <-drop key3 | user_key1 | user_table1 | timestamp3| <-keep <-timestampA key4 | user_key1 | user_table1 | timestamp4| <-keep

Now, a timestamp tsA is specified and the relationship is ts1 < ts2 < ts3 < tsA < ts4.
After GC, key1 and key2 is dropped because key3's timestamp is the biggest among ts1(key1), ts2(key2) and ts3(key3) and key4 is kept because ts4 is bigger than tsA.

CompactionFilterV2 is used for the demand above. Firstly, using prefix extractor same user_key and user_table are aggreated. Secondly, during once Filter invoke, keys can be picker properly.

Because of the deprecation of CompactionFilterV2, the legacy system can not upgrade the version of rocksdb compatibly. I came up with a solution and hope to discuss it and I would be very grateful for other ideas.

Same with kRemoveAndSkipUntil, add a new decision kKeepAndSkipUntil which keeps current key and skip range (key, skip_until). Using the same example above, for example, FilterV2 input key is key3, ts3 is less than tsA, so keep key3 and skip(|user_key1|user_table1|timestamp zero|, key3).


I have posted a issue on Sep 18, 2025 and on Dec 7, 2025 @jrahman also replied this issue with same requirements.


Thanks


Reply all
Reply to author
Forward
0 new messages