K/V and file checksum

[Micheal Okutubo]

Hey folks, I want to try out checksum verification in RocksDB (via RocksJava) and it seems RocksDB supports the following:

1. Whole file checksum: I don’t see how to enable this in RocksJava. Even the VerifyFileChecksums api in c++ returns not supported error but I saw a DB implementation for it.  Any reason why this isn’t supported? Is there a way to enable file checksum in RocksJava?

2. Per Key Value checksum: According to this blog, it seems this exists. How do I enable it in RocksJava? Also does it still not support memtable entries? Any other limitations?

Thanks

[Alan Paxton]

HI Michael,

You can certainly verify the block checksums from the Java API simply by calling `db.verifyChecksum()`

Re (1) whole file checksum. The API (https://github.com/facebook/rocksdb/blob/main/include/rocksdb/db.h#L2016) you link to is just the default implementation. Are you running code which returns this error or have you concluded this is not supported by inspection ?
The `VerifyFileChecksums` API has not been added to the Java API; it should be a straightforward task to add it; we could look at implementing that, and of course you are welcome to submit a PR yourself.

Re (2) This does not appear to have been added to the Java API. It looks as if we would only need to add the `protection_bytes_per_key` write option to Java `WriteOptions` to enable this. Also, the changelog for 7.6.0 says:

Added an option `memtable_protection_bytes_per_key` that turns on memtable per key-value checksum protection

which looks like the memtable support you were asking about. Adding it to the Java API would just be another option, again.

--

Kind regards,

Alan Paxton

Senior Engineer

--
You received this message because you are subscribed to the Google Groups "rocksdb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rocksdb+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/rocksdb/edd6d0e6-9170-4515-b6eb-14755893e7b8n%40googlegroups.com.

[Micheal Okutubo]

Yeah was just trying to check to make sure there wasn't any known issue with it before making the changes. Thanks Alan.