Simple utility to merge EXR tiles

[Fredrik Averpil]

Hi,

After having rendered a still image using slicing/tiling on our render farm, I would like to automate the merging of all slices. I was initially thinking I'd use Nuke. But I was thinking there must be some simple utility that were able to do just this ...

And so I started looking and I even attempted to build OpenImageIO although that would also require me to write the whole app... so I think I'd rather just go with Nuke rather than write a whole new app.

Does anyone know of any such utility, which is preferably available for both Windows and Linux and primarily supports merging a series of EXR files into one big EXR?

Kind of like this:

./merge -infile=render.slice1.exr -infile=render.slice2.exr -output=final_image.exr -compression=zip_scanline

// Fredrik

[matt estela]

Doesn't one ship with vray?

--
You received this message because you are subscribed to the Google Groups "maya_he3d" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maya_he3d+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Fredrik Averpil]

No, not that I know of. There are img2tiledexr.exe and vrimg2exr.exe but none of them accepts a series of EXR images as input and will output one single EXR from them.

[Deke Kincaid]

If you compile OpenExr 2.0 or higher then there is a utility called "exrmultipart" which will let you combine multiple exr files as separate parts.  This makes an EXR 2.0 only image though.

Usage: exrmultipart -combine -i input.exr[:partnum][::partname] [input2.exr[:partnum]][::partname] [...] -o outfile.exr [options]

   or: exrmultipart -separate -i infile.exr -o outfileBaseName [options]

   or: exrmultipart -convert -i infile.exr -o outfile.exr [options]

If you want OpenEXR 1.x images there is a nice set of python bindings for openexr here which can combine a bunch of exr channels from different files into one:

http://excamera.com/sphinx/articles-openexr.html#openexrpython

http://excamera.com/articles/26/doc/intro.html#exr-to-exr

[Fredrik Averpil]

That's excellent, Deke. I actually found a precompiled OpenImageIO build for 64-bit Windows, Python 2.7 which I ended up using. That was EXR version 1.3.13:

http://www.lfd.uci.edu/~gohlke/pythonlibs/#openimageio

But eventually I will have to do this thoroughly and then perhaps it will be easier to compile OpenEXR 2.0.

// Fredrik

[Szabolcs Horvatth]

Hi,

A little while ago I wrote a python script for merging EXR image slices.
It was not too complicated but there were a few surprises:
- The http://excamera.com/sphinx/articles-openexr.html#openexrpython
python bindings are for a quite old OpenEXR version, so some of the
newer EXR attributes are not supported. And adding custom attributes for
example were not working properly.
- You need either PIL or Numpy to work with the arrays returned by the
bindings and Numpy is the better option, because it supports the half
datatype too. But still its an extra dependency.
- The Numpy version that came with our Ubuntu distro was old and it was
tricky updating it. :)
- If there is no data window defined than the merging can be really
slow, because you do have to check the pixel values to avoid "overing"
the full black image on your content. And when you have lots of layers
than its even slower.
- When I started doing this it sounded like a good solution to use
python and get rid of the EXR dependencies and compiling but if I had to
do it again I'd go with C++...

And if you need the script just drop me a mail.

Cheers,
Szabolcs

[Fredrik Averpil]

Hey Szabolcs,

Thanks for the pointers. I was thinking I would go for Deke’s advice and attempt building OpenEXR 2.1.0 and give “exrmultipart” a try to get a proper, working solution to merge slices on both Windows and Linux (since I only have a quick fix in effect at the moment). Any particular reason why you didn’t go for that?

Did you end up using the older Open EXR version / Python bindings or did you end up somehow successfully being able to utilize OpenEXR 2.0 via Python bindings?

So, what I need to do is merge multiple V-Ray-rendered EXR slices on the commandline (and preferably via Python), and I need to do it on both Linux and Windows, to later be read into Nuke 8.

I found a quick fix to this; a pre-compiled binary for Windows of OpenImageIO, and so I’m using its python bindings. I read each EXR into ImageBuf(), added them together and saved out a new EXR. This works fine for now. Quick psuedo-like code of what I’m doing at the moment in case that helps anyone else trying to do the same thing:

import OpenImageIO as oiio
from OpenImageIO import ImageInput, ImageOutput
from OpenImageIO import ImageBuf, ImageSpec, ImageBufAlgo

folder = '/some/folder'
merged_buf = ImageBuf()
buffers = []

for filename in os.listdir( folder ):
  filepath = os.path.join(folder, filename)
  buffers.append( ImageBuf( filepath ) )

for i in range(0, len(buffers)):
  if i == 0:
    ImageBufAlgo.add( merged_buf, buffers[i], buffers[i+1])
  if (i > 1) and (i <= len(buffers) ):
    ImageBufAlgo.add( merged_buf, merged_buf, buffers[i])

merged_buf.write( os.path.join(folder, 'merged.exr') )

Regards,
Fredrik

​

--
You received this message because you are subscribed to the Google Groups "maya_he3d" group.

To unsubscribe from this group and stop receiving emails from it, send an email to maya_he3d+unsubscribe@googlegroups.com.

[Szabolcs Horvátth]

Hi Fredrik,

> Any particular reason why you didn’t go for that?

Yes, I haven't tested OpenEXR 2.0 and Nuke 7.x and I also thought that it would be a lot easier than it actually turned out to be. ;)

> Did you end up using the older Open EXR version / Python bindings or did you end up somehow successfully being able to utilize OpenEXR 2.0 via Python bindings?
>

I used the older bindings.

> found a quick fix to this; a pre-compiled binary for Windows of OpenImageIO <http://www.lfd.uci.edu/%7Egohlke/pythonlibs/#openimageio>, and so I’m using its python bindings. I read each EXR into ImageBuf(), added them together and saved out a new EXR. This works fine for now. Quick psuedo-like code of what I’m doing at the moment in case that helps anyone else trying to do the same thing:
> ...
Thats nice. I could not use OIIO because we use different bit depths for various layers and OIIO does not handle that in a single EXR file.

Cheers,
szabolcs