[Feature Request] Expose ExportColumnFamily() and CreateColumnFamilyWithImport() in Public C-style API

[Dave Oda]

Greetings RocksDB community,
Part-time lurker, first-time contributor. 

Background
We are leveraging RocksDB on Windows platforms in managed (C#) via the RocksDBSharp application library.  That library exposes much of the essential RocksDB functionality. 

Problem
However, ExportColumnFamily() and CreateColumnFamilyWithImport() methods are not exposed in the public C-style API; therefore, they cannot be leveraged by non-native applications, such as exposed in the RocksDB-Sharp application/wrapper (for managed C# use).  These methods would be extremely useful, if not essential, for our purposes. 

Proposed Changes
We'd like to see the ExportColumnFamily() and CreateColumnFamilyWithImport() methods exposed in the public C-style API.  For proper usage, by any external calling application, this will require additional support functions (especially to facilitate marshalling of LiveFileMetadata structure and the ImportColumnFamilyOptions classes/structs via properties and objects from native to C# applications and back) to be implemented and exposed in the RocksDB public API.

Proof of Concept
We have prototyped the working RocksDB fork/build with changes to the public API (primarily, c.h and c.cc files) on Windows (and VS2022) to expose the ExportColumnFamily() and CreateColumnFamilyWithImport() methods.  Unit tests are still in development and feedback from the community will need to take place (which is starting with this missive).

What does the community think about this Feature Request?  Is it worthwhile?

[Radek Hübner]

Hello Dcoda, 

Thank you for reaching out community and sorry for late reply. 

I think your feature request makes sense and I expect that more people will be able to benefit from it. If you have any existing code, feel free to submit PR. Even if it's just draft PR, we can work together to make it productions ready. Just let me know the URL of PR.

Details how to contribute you can find in RocksDB wiki: https://github.com/facebook/rocksdb/wiki/RocksDB-Contribution-Guide

Radek 

[Pavel Tcholakov]

Hi folks,

I am resurrecting an old conversation - I created a parallel implementation sometime last year, before I discovered this thread. I have opened a PR with my changes:

https://github.com/facebook/rocksdb/pull/13874

I believe this has some advantages over Dave's existing PR:

  - adds a smaller surface area
  - better encapsulates metadata adding a rocksdb_export_import_files_metadata_t wrapper
  - follows existing C API patterns a bit more closely

We've been using the changes above for just under a year in production via our own fork of the rust-rocksdb crate. In the end, I'm not attached to which PR gets merged, I'd just love to see FFI column family export/import support land in main. If anyone is available to take a look at the PR, I'd be super grateful!

Many thanks,
Pavel