Comparing the performance of different voxel libraries

[Magnus Wrenninge]

A while back, I was interested in comparing the performance of Field3D with some other voxel storage libraries, most notably OpenVDB. Personally, I was interested in the memory- and performance implications of using different data structures, because when Field3D was first developed, our use of multiple data structures came from the assumption that different uses and applications require different data structures for optimal performance. Thus, it was of interest to us to see what each data structure's performance was under a range of different tasks, so that we could make educated decisions about when to use each one.

The result of this is a suite of tests located at http://github.com/imageworks/Field3D/blob/master/test/misc_tests/lib_perf_test/

The test code was initially written by me, but as the various tests took shape, I implemented the Field3D parts, and people from the VDB team implemented the VDB parts. 

Hopefully, the tests can be used as a basis for determining which data structure is more suitable for a range of different contexts: dense data, sparse data, narrow band levelsets, coherent lookups, incoherent lookups, memory overhead, etc., etc. 

A summary of the test results is available in PDF form here:

http://magnuswrenninge.com/visualeffects/field3d-performance-test

(Original source http://github.com/imageworks/Field3D/blob/master/test/misc_tests/lib_perf_test/test_results/graypage_tests.pdf?raw=true)

I'm sure there will be lots of questions and discussions spawned from this, and I hope we can explore the topic further together, and perhaps come up with new tests that highlight other aspects of volumetric data structures.

Magnus

[Steven Caron]

thanks for sharing this magnus...

am i reading this right? field3d seems to use slightly less memory for most operations? and for incoherent tasks, ray marching, and a few other field3d is more performant?

thanks

steven

[Maik Figura]

Hey Magnus,

thanks for sharing the summary and the testing code, pretty interesting stuff!

Cheers
Maik

[Magnus Wrenninge]

Hi Steven,

Regarding memory use, yes, although keep in mind that most of the data sets in the current tests are dense. VDB is definitely more memory efficient for very sparse data sets, like narrow band level sets. But for clustered and/or only partially sparse data sets, SparseField can be a better fit.

For performance, SparseField (and DenseField even more so) is faster than VDB because it's inherently a simpler data structure with less overhead. Depending on what your application/plugin is doing, one structure or the other may be more efficient.  Especially for small data sets, regular dense buffers can be drastically faster than either VDB or SparseField. For production code that may run tens or hundreds of thousands of proc hours over the course of a show, it's definitely worth considering.

Keep an eye out in the coming months for some Field3D 2.0 announcements related to all this.

Magnus

--
You received this message because you are subscribed to the Google Groups "Field3D dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to field3d-dev...@googlegroups.com.
To post to this group, send email to field...@googlegroups.com.
Visit this group at http://groups.google.com/group/field3d-dev.
For more options, visit https://groups.google.com/groups/opt_out.

[Frederic Servant]

Hi Magnus,

Very cool initiative and very fair comments.

Thanks for this.

--

Fred