Adding new type of grid

[Emmanuel Lévêque]

Hello, I am wondering about the relevance of using openvdb for lattice Boltzmann CFD simulation in complex sparse geometries, such as porous materials. In that case, a new datatype should be used to handle the set of  velocity distributions in each voxel, i.e. a vector with 15, 19 or 27 components. How straightforward would it be to add a new type of grid in the openvdb library?

Thank you for your advices.

Regards, Emmanuel

[Ken Museth]

Hi Emmanuel,

Great question. I see two possible solutions - either use one grid with a custom ValueType (as you're suggesting) or use multiple grids. The latter is obviously much easier and in many cases pretty performant as well since the overhead of encoding tree topology is typically very small compared to the memory footprint of the values. The (multi-grid) approach also has the advantage that the sparseness  of the various fields are allowed to change, which might even result in better compression than the single-grid approach. However, the single grid with a custom ValueType is also possible thought far more complex. While it's relatively easy to template a VDB Grid and Tree on your own ValueType (see BlindData in openvdb/tools/ParticlesToLevelSet.h) most of our tools and I/O methods will likely not work with your custom grid. In other words, I would only take this approach if you're prepared to write custom tools as well. Incidentally, we've actually talked about adding N-size vectors to OpenVDB in the future, but I can't tell you exactly when that will be available.

-Ken

[Ken Museth]

Hi Emmanuel,

I just announced the release of OpenVDB version 9.0.0 which actually includes a feature that could be very useful for you, namely "transient data" in all the tree nodes. You can think if it as blind-data that is not written to disk. It could be used to index into "side-cars" of data. Imagine you create a regular FloatGrid or Vec3fGrid. Then you allocate external arrays with additional values for your lattice Boltzmann simulation, and use the transient data in each tree node to point to those auxiliary arrays. Very straight forward and yet powerful techniques. In fact, we have secretly been using this technique in OpenVDB for many years but hacking the array index into the origin of each tree node (which is safe in certain scenarios), but in 9 we're include an extra int32_t that is explicitly intended for this purpose - so not hacking needed anymore :)

-Ken

[Jasper Lu]

Excited to see the extra int32_t that will be added to OpenVdb!

Another potential suggestion for Emmanuel -- something we've done in the past to accommodate custom data types is look for an existing data type in openvdb that has a size >= our desired size (so in your case, assuming you need 27 floats, you would need an existing data type of size >= 27 * 4 = 108 bytes). Then, we create a packed struct of the same size as the existing data type. From there, all we need to do is reinterpret_cast the data type into a struct whenever we access data (easily hidden behind an API).

So, for example, in the case of a 27-length velocity vector of floats, you could choose the Mat4D type (which is 16 * 8.= 128 bytes) and do:

struct __attribute__((packed)) VelocityVoxel {

    using StorageType =  Mat4d;

    float components[27]; // 108 bytes

    int padding[5];

};

openvdb::tree::Tree<VelocityVoxel::StorageType, ...>

[Ken Museth]

Very clever, though definitely not for the faint of heart :)

[Emmanuel Lévêque]

Hello, Thank you very much for your suggestions. I like the transient data attached to the LeafNode to point to some auxiliary data atlas.  The trick of pack structure is also very appealing.

Regards, Emmanuel