Query for RocksDB 6.29.5.x

[Christo Lolov]

Hello!

I am working on unblocking Apache Kafka's 3.4.0 release. Kafka has a dependency on RocksDB 6.29.4.1. We would like to move to a RocksDB version which has zlib 1.2.12 due to https://nvd.nist.gov/vuln/detail/cve-2018-25032. The first version which satisfies this is 7.1.2. However, that version has removed some APIs which Kafka exposes as public interfaces (additional information available at https://github.com/apache/kafka/pull/12809). The ideal solution would be for us to contribute to a new 6.29.x or 6.29.5.x version of RocksDB and upgrade zlib to 1.2.12. Is there any possibility for such a release or all development efforts have moved to 7.x and we should be looking for alternatives?

Best,

Christo

[Adam Retter]

Could you comment on which APIs you export from 6.x.x that have been removed in 7.x.x?

[Christo Lolov]

Yep,

https://javadoc.io/static/org.rocksdb/rocksdbjni/6.29.4.1/org/rocksdb/DBOptions.html#baseBackgroundCompactions()

https://javadoc.io/static/org.rocksdb/rocksdbjni/6.29.4.1/org/rocksdb/DBOptions.html#newTableReaderForCompactionInputs()

https://javadoc.io/static/org.rocksdb/rocksdbjni/6.29.4.1/org/rocksdb/DBOptions.html#preserveDeletes()

and their corresponding setters

None of the above appear in https://javadoc.io/static/org.rocksdb/rocksdbjni/7.1.2/org/rocksdb/DBOptions.html.

Best,

Christo

[Adam Retter]

Hi Christo,

So the Java API simply tried to reflect the C++ API.

Out of the 3 options that you mentioned you needed from version
6.29.4.1 that were subsequently removed, setting 2 of those options
even in version 6.29.4.1 actually had no effect on RocksDB. You can
see this in the documentation of options.h -
https://github.com/facebook/rocksdb/blob/v6.29.4/include/rocksdb/options.h.
Explicitly:

// NOT SUPPORTED ANYMORE: RocksDB automatically decides this based on the
// value of max_background_jobs. This option is ignored.
//
// Dynamically changeable through SetDBOptions() API.
int base_background_compactions = -1;

// This flag has no affect on the behavior of compaction and plan to delete
// in the future.
bool new_table_reader_for_compaction_inputs = false;

So if you were exposing those two options via your API previously, you
can continue to expose them via your API very easily without needing
any support from RocksDB as calling them is essentially a NO-OP.


The third option, `preserve_deletes` that you enquired about is
documented like this in 6.29.4.1:

// Deprecated, will be removed in a future release.
// Please try using user-defined timestamp instead.
// DEFAULT: false
bool preserve_deletes = false;

I think very simply in your API, you can continue to expose
preserveDeletes() and just always have it return false.

In this manner you should be able to preserve your API even after
upgrading to the latest RocksDB.

Kind regards. Adam.

> --
> You received this message because you are subscribed to the Google Groups "rocksdb" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rocksdb+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/rocksdb/780816f7-8836-4c56-ba5e-e7e7933f5e19n%40googlegroups.com.



--
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk

[Christo Lolov]

Hello Adam,

Thank you for the prompt response, for looking into the code and for the suggestions! I have since updated the pull request for Kafka 3.4.0 and the rest of the committers appear to be happy with the approach.

Best,

Christo

[Adam Retter]

Excellent. Glad I could help