16bit precision VDB volumes ?

[Michal Maciejewski]

Hello,

I am trying to minimize memory consumption of OpenVDB Rasterize Points. I am creating Fog volume for density rendering, basically range of values is expected to be in 0-1 range. Seems like I dont need 32-bit precision.

Would it be possible to add option to rasterize to 16-bit or introduce compression ?

Thanks!

[Kévin Dietrich]

For in-memory "compression", what you can do is define is a half-float grid, like so:

typedef openvdb::tree::Tree4<half, 5, 4, 3>::Type HalfTree;
typedef openvdb::Grid<HalfTree>::Type HalfGrid;

You can also register a HalfMetadata type if you'd like.

If you intend to save the grid to a file and read it back, then you should also register the grid, before reading:

HalfGrid::registerGrid();

Though, for saving a grid using half floats you can use a regular float grid and set a "save float as half" metadata on it:

grid.setSaveFloatAsHalf(true);

[Michal Maciejewski]

Thanks Kévin,

Is defining a half-float grid exposed in any of OpenVDB nodes in Houdini ?

Unfortunately, I am not a programmer so its hard for me to make use of your solution.

I can find OpenVDB Metadata to define 16bit float, but its for writing on disk only. I dont intend to write my volume to file.

[Harry Biddle]

Hey Michael,

Houdini doesn't have much support for half-precision volumes unfortunately. Are you rendering with Mantra? Are you sure that the file size is causing a memory bottleneck?

[Michal Maciejewski]

Im pretty sure. I simply run out of memory when I set desired resolution.

I have 20mil cell volume for each:

- density - float

- Cd - color (3xfloat)

At voxel size 1 I have 3,4 GB memory usage.

Once I get down to 0.5 to get the detail right, Im out of my limit.

I am looking for ways of optimizing my volumes. I wish there was the same compression options as you guys did in VDBPoints. All my volumes are in 0-1 value ranges. I have heard such case can be very efficiently packed in memory.

Any other quick workarounds or tips ? 

[Harry Biddle]

Ah, sorry, I've misunderstood. Are you running out of memory during the rasterise stage? You could consider doing spatially coherent sections of the volume and merging the result. Are you hoping to do all of this in Houdini?

[Michal Maciejewski]

yes, all in Houdini. Thanks, slicing volume in relation to Camera, and
setting lower resolution further away from Cam will do the job.

[Mihai Aldén]

Hi, the latest version of this tool, which is available on github now, supports frustum aligned grids and subregion masking.

[Dan Bailey]

We only have that build released for H14 at Dneg currently, H15 coming shortly! :)

Mihai, is there an advantage to rasterizing each attribute independently if we're really close to the memory limit? From skimming the code it looks like we could possibly trade off a little performance against saving some memory? (All SOPs are already being unloaded here).

Also, one quick suggestion - the transform / mask / frustum options are typically hidden beneath all the vex processing options on the OpenVDB Rasterize Points SOP. Others may disagree, but I might suggest putting them above the vex processing options would them a little easier for an artist not to miss them.

Finally, as Tas says, it might not be a bad idea to consider 16-bit in-memory support for VDB, particularly for use cases like the Cd attribute where we don't need very much precision but still want a high voxel count. Not sure if this has come up at DWA or not.

[Mihai Aldén]

Hi Dan,

" is there an advantage to rasterizing each attribute independently if we're really close to the memory limit? From skimming the code it looks like we could possibly trade off a little performance against saving some memory?"

- I don't think the memory overhead of the tool is significant.  So this strategy will probably only help if you write out each grid to disk.

"Also, one quick suggestion - the transform / mask / frustum options are typically hidden beneath all the vex processing options on the OpenVDB Rasterize Points SOP. Others may disagree, but I might suggest putting them above the vex processing options would them a little easier for an artist not to miss them."

- The UI is already structured like you are proposing. :)   But the creation script only gets executed once so you need to drop down a new node to get the updated UI.

This is how the new UI should look:

[Dan Bailey]

Good to know, thanks Mihai.