Speeding up the voxelization algorithm

[emanuele.gissi]

Hi to all,

I am very satisfied with the current BlenderFDS release. One of the
main remaining issues of BlenderFDS is the speed of the voxelization
algorithm. Exporting complex and large geometries from BlenderFDS to
the FDS input file can take a long time.

I think that there is room for improvement and speed up of the
voxelization algorithm, but sadly this is beyond my coding
abilities...

Today I probably found a specific financing source, and I would like
to commit this task to a professional developer.
The resulting code will be open-sourced.

1 - Is anybody interested in this task?
2 - Or,do you know anybody that could help us out on that?

I'll send a similar request to the blender user list.

Best regards,
Emanuele Gissi

[Ron]

Hi Emanuele,

I am not a professional developer but I have been writing a program
for work in python for the past 5 months. The one thing I discovered
(if you don't know this already) is that python "for loops" are
dreadfully slow. What I did was to replace the for loops I could with
numpy arrays. My solution time went from about 30 seconds to less
than 1 second and I still have more loops to replace.

A simple example:

out = []
for i in range(10000):
out.append(i * 50)

change to:

i= np.arange(0, 10000)
out = i * 50

This is simple but if the for loop has a lot of increments the speed
increase will be dramatic.

Good luck
Ron

[Emanuele Gissi]

Thank you, Ron! That's a good idea.
I'll try that. But first I shall discover how to use numpy inside Blender embedded Python.
I'll keep you informed.
Emanuele

--
http://www.emanuelegissi.eu/

[Ron]

No problem, glad to help.

I just found the numpy command "tofile" that writes the data in the
numpy arrays out to either ASCII or binary data format.
I haven't used it but it might help speed things up.

Ron

[Matti]

Hello Emanuele,

I think the best way to speed up is to use numpy.
Do you have any ideas to use numpy in Blender?

I tried a few things, but with the embedded Python the voxelization is
very slow.


Matthias

[Johannes Dimyadi]

Thanks, Matti

Trust all is well with your business.

I don't have a lot of experience with Python, so can't contribute anything in this area.

Johannes

[Emanuele Gissi]

Sorry for not replying earlier, I am very taken by a SAR exercise here in Genova.
I will do some research on numpy and tell you in some days, ok?
Sorry once more,
Emanuele.

--
http://www.emanuelegissi.eu/

[Kristopher Overholt]

I use Numpy quite a bit in my research, and the speedups are significant as everyone has mentioned. However, I don't think there is an easy way to work with a Numpy dependency in Blender because Numpy requires compilation on the local platform and cannot simply be included in the BlenderFDS plugin like some other Python modules.

--
Kristopher J. Overholt
Ph.D. Student, Fire Research
The University of Texas at Austin
http://www.koverholt.com
Mobile: (832) 736-3473

[Matti]

Hi Emanuele,
Hi Kristopher,

has one of you connections to the blender developers?

I think numpy is the most efficient way. But it is unfortunately no
python module.
I will do a few experiments with python to find the fastest blender-
python version for voxelization.

Matthias

On 5 Mai, 07:38, Kristopher Overholt <koverh...@gmail.com> wrote:
> I use Numpy quite a bit in my research, and the speedups are significant as
> everyone has mentioned. However, I don't think there is an easy way to work
> with a Numpy dependency in Blender because Numpy requires compilation on the
> local platform and cannot simply be included in the BlenderFDS plugin like
> some other Python modules.
>

> On Thu, May 5, 2011 at 00:30, Emanuele Gissi <emanuele.gi...@gmail.com>wrote:
>
>
>
>
>
> > Sorry for not replying earlier, I am very taken by a SAR exercise here in
> > Genova.
> > I will do some research on numpy and tell you in some days, ok?
> > Sorry once more,
> > Emanuele.
>

[Johannes Dimyadi]

Hi Matti

Your help would be very valuable to the project.

Johannes

[Emanuele Gissi]

Hi Matti, hi all,

the SAR exercise came to a conclusion, and everything was ok: nobody was hurt!
Back to the office, time to switch on again the computer now.

Many BlenderFDS users privately suggested Numpy. And I read wonders about it.

As Kris stated, the main problem is importing Numpy while running inside the Blender embedded Python interpreter.
Once this problem is solved I am pretty confident we can overcome our issue.

I have some connections with Blender developers.
They are rather busy at moment, while trying to release Blender 2.58
But, in general, they are rather responsive.

Two persons that could help us are Yorik (http://yorik.uncreated.net/) and Allan Brito (http://www.allanbrito.com/). Though not Blender core developers, they can point us in the right direction. Matti, you can write to them directly if you want: they both are very kind.

One of the best forums for obtaining good answers is the following:
http://www.blenderartists.org/forum/

Some time ago I found the BioBlender project (http://bioblender.eu/)
I think they are using Numpy in Blender. They heavily modified Blender code.
Not a viable solution for us. But they can help us, a little.
They are italians, next week I will perform a deeper research and report back here in some days.

Best regards,
Emanuele

On Thu, May 5, 2011 at 8:36 AM, Matti <matthias....@googlemail.com> wrote:

Hi Emanuele,
Hi Kristopher,

has one of you connections to the blender developers?

I think numpy is the most efficient way. But it is unfortunately no
python module.
I will do a few experiments with python to find the fastest blender-
python version for voxelization.

Matthias

--
http://www.emanuelegissi.eu/

[Emanuele Gissi]

Hi Matti, hi all,

In my spare time I continued to work on this issue and I probably found a solution.

Thanks to the ctypes module, it's very easy to call a C library from Python code.
I made some test and it worked like a charm
So what we need to do is implementing the voxelization algorithm to a pure C library with as few dependencies as possible.
Pure ANSI C would be the best choice.

As a side note, I add that this work prepares our project for a likely revolution in geometry description in FDS 6 or further revision.

At the moment my C skills are not good enough for such an endeavour, I am still studying.

If your offer is still valid, Matti, could I kindly ask you to work on that?
Here is a very good (and open source) starting platform:
http://www.cs.princeton.edu/~min/binvox/

Best regards,
Emanuele Gissi

--
http://www.emanuelegissi.eu/

[Matti]

Hi Emanuele,

I will test this.

The idea with c is perhaps the faster way.
I will talk to a few students of my university. Perhaps they will help
us to code c.

Greetings,

Matti

On 20 Sep., 13:35, Emanuele Gissi <emanuele.gi...@gmail.com> wrote:
> Hi Matti, hi all,
>
> In my spare time I continued to work on this issue and I probably found a
> solution.
>
> Thanks to the ctypes module, it's very easy to call a C library from Python
> code.
> I made some test and it worked like a charm
> So what we need to do is implementing the voxelization algorithm to a pure C
> library with as few dependencies as possible.
> Pure ANSI C would be the best choice.
>
> As a side note, I add that this work prepares our project for a likely
> revolution in geometry description in FDS 6 or further revision.
>
> At the moment my C skills are not good enough for such an endeavour, I am
> still studying.
>
> If your offer is still valid, Matti, could I kindly ask you to work on that?
> Here is a very good (and open source) starting platform:http://www.cs.princeton.edu/~min/binvox/
>
> Best regards,
> Emanuele Gissi
>

> > On Thu, May 5, 2011 at 8:36 AM, Matti <matthias.zaehrin...@googlemail.com>wrote:
>
> >> Hi Emanuele,
> >> Hi Kristopher,
>
> >> has one of you connections to the blender developers?
>
> >> I think numpy is the most efficient way. But it is unfortunately no
> >> python module.
> >> I will do a few experiments with python to find the fastest blender-
> >> python version for voxelization.
>
> >> Matthias
>
> > --
> >http://www.emanuelegissi.eu/
>
> --http://www.emanuelegissi.eu/

On 20 Sep., 13:35, Emanuele Gissi <emanuele.gi...@gmail.com> wrote:
> Hi Matti, hi all,
>
> In my spare time I continued to work on this issue and I probably found a
> solution.
>
> Thanks to the ctypes module, it's very easy to call a C library from Python
> code.
> I made some test and it worked like a charm
> So what we need to do is implementing the voxelization algorithm to a pure C
> library with as few dependencies as possible.
> Pure ANSI C would be the best choice.
>
> As a side note, I add that this work prepares our project for a likely
> revolution in geometry description in FDS 6 or further revision.
>
> At the moment my C skills are not good enough for such an endeavour, I am
> still studying.
>
> If your offer is still valid, Matti, could I kindly ask you to work on that?
> Here is a very good (and open source) starting platform:http://www.cs.princeton.edu/~min/binvox/
>
> Best regards,
> Emanuele Gissi
>

[Emanuele Gissi]

Thank you Matti!
Emanuele

--
http://www.emanuelegissi.eu/