Hugin++ direct reading of ARW (raw) files

[johnfi...@gmail.com]

I made a fork of vigra, plus a tiny change to hugin++ so that hugin++ can directly use ARW files rather than first converting them to TIFF.  Very little extra would be needed to support other formats supported by libraw.

I strongly dislike the directory clutter of the current hugin method of dealing with raw files (convert to tiff and then use the tiff file in the hugin project).  I keep hugin projects around a long time while experimenting with ways of doing things a little better.  I don't want to keep the tiff files.

This hugin++ change is tiny and clean.  It just prioritizes vigra over raw conversion for extensions that are in both lists.  If you use the standard vigra with hugin++ that change makes zero difference and you don't need libraw.

This hugin++ change only makes a difference if you use my fork of vigra, which uses (and requires) libraw.

My changes to vigra are quite ugly.  Apologies in advance to anyone trying to build it (but I'll provide whatever support I can and answer any questions and appreciate any suggestions on doing any of that better).  My lack of cmake knowledge made that part of the changes (getting the vigra build to find/use libraw) especially bad.  The Windows/Mingw64 build is working for me, as is the Fedora build.

I'm happy to zip up and share all the required mingw64 binaries if someone else wants to test this on windows. There is no .dll compatibility for any C++ symbols between visual C++ and mingw64, so you either need 100% visual C++ or 100% mingw64.  For visual C++, I could only answer questions, not provide any binaries.

At some point I'll learn how the windows hugin binary distribution creation process works and post an installer (for the mingw64 binary version) on the hugin++ site.  So far no one has asked for that.

For fedora, I think building hugin++ and my fork of vigra from the repositories (and getting libraw and all other dependencies the normal way) would be easier than getting any binaries from me.  But ask me for help if you want to test this and have some problem.

I'm still working on several aspects of this.  I think it is ready for other to try.  I don't think it is yet ready for anyone to rely on.  I would appreciate any feedback.

Relevant repositories are:

https://sourceforge.net/projects/vigraraw/

which is my fork of:

https://github.com/ukoethe/vigra

https://sourceforge.net/projects/huginplusplus/

https://github.com/LibRaw

[Tommy Hughes]

Hi, I want to try this but not sure how to build it?

➜  build cmake ..
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring VIGRA version 1.11.1
-- Checking template recursion depth: ok
-- Searching for ZLIB
--    in default locations
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11")  
-- Searching for RAW
--    in default locations
--  RAW_NAMES: libraw;libraw_r
--  RAW_LIBRARY_RELEASE: RAW_LIBRARY_RELEASE-NOTFOUND
-- Searching for TIFF
--    in default locations
-- Could NOT find TIFF (missing: TIFF_LIBRARY TIFF_INCLUDE_DIR)  
-- Searching for JPEG
--    in default locations
-- Could NOT find JPEG (missing: JPEG_LIBRARY JPEG_INCLUDE_DIR)  
-- Searching for PNG
--    in default locations
-- Could NOT find PNG (missing: PNG_LIBRARY PNG_PNG_INCLUDE_DIR)  
-- Searching for FFTW3
--    in default locations
-- Could NOT find FFTW3 (missing: FFTW3_LIBRARY FFTW3_INCLUDE_DIR)  
-- Searching for FFTW3F
--    in default locations
-- Could NOT find FFTW3F (missing: FFTW3F_LIBRARY FFTW3F_INCLUDE_DIR)  
-- Searching for HDF5
--    in default locations
-- Could NOT find HDF5 (missing: HDF5_INCLUDE_DIR)  
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)  
-- Found PythonInterp: /usr/bin/python (found version "2.7.18")  
-- Searching for Boost (at least version 1.40.0)
--    in default locations
-- Could NOT find Boost (missing: Boost_INCLUDE_DIR) (Required is at least version "1.40.0")
-- Checking for threading support:   std::thread
--     (added compiler flags: -pthread -std=c++11)
-- Detected C++ version: 201103
-- Checking VIGRANUMPY_DEPENDENCIES
-- Using Python 2.7.18 at /usr/bin/python
-- Found Python includes:  /usr/include/python2.7
-- Could NOT find Python library (looking for version 2.7)
-- Could NOT find boost_python library (looking for version 2.7)
-- Searching for Python numpy: ok
Traceback (most recent call last):
 File "<string>", line 1, in <module>
ImportError: No module named nose
-- Could NOT find Python nosetests ('import nose' failed)
-- Could NOT find sphinx documentation generator
-- Could NOT find VIGRANUMPY_DEPENDENCIES (missing: PYTHON_LIBRARIES Boost_PYTHON_LIBRARY)  
-- ** WARNING: test_classifier::RFHDF5Test() will not be executed
-- ** WARNING: fftw not found. test_correlation will not be executed
-- ** WARNING: test_features will not be executed
-- ** WARNING: test_fourier will not be executed
-- ** WARNING: test_hdf5impex will not be executed
-- ** WARNING: fftw not found. Fourier-domain registration tests will not be executed
-- ---------------------------------------------------------
-- VIGRA configuration information:
-- ---------------------------------------------------------
--   Using RAW   libraries: RAW_LIBRARY_RELEASE-NOTFOUND
--   Using ZLIB  libraries: /usr/lib/x86_64-linux-gnu/libz.so
--   PNG libraries not found (PNG support disabled)
--   TIFF libraries not found (TIFF support disabled)
--   JPEG libraries not found (JPEG support disabled)
--   OpenEXR disabled by user (WITH_OPENEXR=0)
--   FFTW libraries not found (FFTW support disabled)
--   HDF5 libraries not found (HDF5 support disabled)
--   Boost Graph Library disabled by user (WITH_BOOST_GRAPH=0)
--   LEMON graph library disabled by user (WITH_LEMON=0)
--   Vigranumpy dependencies not found (vigranumpy disabled)
-- ---------------------------------------------------------
--   building shared lib
--   binaries will be generated in: /home/tommy/Downloads/hugin++/vigraraw-code-559c737e4cfcb207b3626f224fcb17c1b74f8e3e/build
-- ---------------------------------------------------------
--   includes will be installed at: /usr/local/include
--   libraries will be installed at: /usr/local/lib
--   binaries will be installed at: /usr/local/bin
--   vigra manuals cannot be generated (Doxygen or Python not found)
--   vigranumpy will NOT be installed
--  
-- Consider setting DEPENDENCY_SEARCH_PREFIX to find missing libraries
-- ---------------------------------------------------------
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
RAW_LIBRARY_RELEASE (ADVANCED)
   linked by target "vigraimpex" in directory /home/tommy/Downloads/hugin++/vigraraw-code-559c737e4cfcb207b3626f224fcb17c1b74f8e3e/src/impex

-- Configuring incomplete, errors occurred!
See also "/home/tommy/Downloads/hugin++/vigraraw-code-559c737e4cfcb207b3626f224fcb17c1b74f8e3e/build/CMakeFiles/CMakeOutput.log".

[John Fine]

First, a warning:  After posting the above, my testing determined there is quite a bit of processing needed to massage a raw image into a useful 16 bit image.  I don't know how to do that extra processing.  I looked at other programs using the same code as libRaw and was unable to follow what they were doing and empirically found even they were better than not doing any extra processing, but not really correct:

The conversion to JPG done inside the camera does a better job of noise elimination and a better job of interpolation than any program I've tried for reading the raw file.  The JPG loses lots of range vs. the raw file.  I'd like to know how to get similar noise reduction and interpolation to what the camera would do itself but preserve the range.  RawTherapee does that badly;  Other programs do it worse;  My own code in my fork of Vigra is still worse, likely bad enough to not really be any use.

It can import raw images and preserve the extra bit bits (that the JPG format has thrown away) but the result is generally worse than the JPG (depending on how much you needed the extra bits).  While it is more convenient than first converting to tiff with RawTherapee, it isn't as good quality.

I hope to someday more carefully study the code in RawTherapee and figure out what I should have done.  But for now, I've set it aside and I'm working on other aspects of Hugin++

So trying my code might be a waste of time, unless you know enough about what ought to be done to fix it, or you are just curious.

On Sun, Jul 31, 2022 at 11:18 AM Tommy Hughes <irthoma...@gmail.com> wrote:

Hi, I want to try this but not sure how to build it?

I had a hard time myself in getting Vigra to build.  I'm incompetent with Cmake and always fumbling around in the dark when it comes to getting Cmake to find things.

I get things to work mostly by editing the CmakeCache.txt file and fixing the NOTFOUND entries to point to the right places.  It would be more correct to convince Cmake to find things, but with only one big exception that I've run into, editing the cache to make it think an earlier run found the thing works.

The hard part is guessing what it wanted:  Typically there is some file and it wants the path of the directory that contains that file and if you figure out what file you can find it and give it the directory path.  But it just identifies the package it wants, which is split across multiple directories and you don't easily know which of those it wants.

I assume you went directly to building my vigra without ever learning to build the original vigra.  Mine is flawed in how it searches for libraw (I'd really appreciate someone competent in cmake telling me what I should have done).  It looks like more is wrong than that (things that would also fail if you were building the official vigra).  But I'm not sure.

It certainly looks like you are missing some of the packages that are described as optional here:

http://ukoethe.github.io/vigra/doc-release/vigra/Installation.html

such as libjpeg and libtiff

For practical use of vigra with hugin, those really aren't optional.

The problem that is almost certainly my fault is:

Using RAW   libraries: RAW_LIBRARY_RELEASE-NOTFOUND

My (fedora) build of my vigra has these lines in CMakeCache.txt

//Path to a file.
RAW_INCLUDE_DIR:PATH=/usr/include/libraw
//Path to a library.
RAW_LIBRARY_RELEASE:FILEPATH=/usr/lib64/libraw.so

I suggest verifying that those exist on your system (libraw is installed) and then editing those lines in your CMakeCache.txt and rerunning cmake and seeing how much gets fixed.

[Tommy Hughes]

I see, thank you for the detailed reply. Does that mean that you are working with jpg files? I haven't dug into the code. Why not just use some temporary folder for the tiffs, and erase them when finished?

[johnfi...@gmail.com]

On Sunday, July 31, 2022 at 2:49:59 PM UTC-4 irthoma...@gmail.com wrote:

I see, thank you for the detailed reply. Does that mean that you are working with jpg files? I haven't dug into the code. Why not just use some temporary folder for the tiffs, and erase them when finished?

I never think I'm "finished".  I haven't yet gotten a single panorama as correct as it looks like it could have been.  So they are all sitting around waiting for me to try some improvement either to Hugin itself or to my skill at using hugin.

I do work mainly with the jpg files.  The fact that RawTherapee (at least on default settings) doesn't do the noise reduction and interpolation as well as the camera does itself, combines with the inconvenience of keeping the temp files around.

In other cases, the jpg has lost too much detail in the darker sections (because 8 bits just isn't enough) so pre-converting to tiff is necessary.

[David W. Jones]

Hmm, I shoot raw files (no jpg), and have no problems with Rawtherapee's noise reduction. I do my conversions to 16-bit per channel TIFF in Rawtherapee, then feed that into Hugin.

In my experience, the amount and type of noise reduction needed varies from image to image, even the same ISO. The amount of contrast in the scene, how bright and dark things are, produces different types (and degrees) of noise.

My Sony A58 and my previous Minolta 7D both use 16-bit per channel sensors, but only output 12-14 bit RAW files. I think in-camera noise reduction is done on the 16-bit image data, right? I think that's the one point where in-camera could do a better job, only with much less control over how it's done.


--
David W. Jones
gnome...@gmail.com
exploring the landscape of god
http://dancingtreefrog.com

Sent from my Android device with F/LOSS K-9 Mail.

[Frederic Da Vitoria]

I used to use a noise reduction software to which you could "teach" what NR settings to apply automatically according to the camera brand, the ISO, the aperture and the speed. The results were good, but the NR was still applied globally to the whole image, and sometimes I had to create several output images using different NR settings and use masks in order to achieve a good result.

Where the camera has an advantage is that it knows precisely how the sensor behaves, when and where it will create the most noise and what kind of noise, maybe even detect different kinds of surfaces and apply different noise reduction settings on different parts of a photo in order to preserve more details when possible. But of course this all relies on the camera's "AI" (I am not sure the word "AI" really applies here), the AI could guess incorrectly, and the user could achieve a better result with enough experience and time.

[johnfi...@gmail.com]

On Monday, August 1, 2022 at 6:54:46 AM UTC-4 davi...@gmail.com wrote:

Where the camera has an advantage is that it knows precisely how the sensor behaves, when and where it will create the most noise and what kind of noise, maybe even detect different kinds of surfaces and apply different noise reduction settings on different parts of a photo in order to preserve more details when possible. But of course this all relies on the camera's "AI" (I am not sure the word "AI" really applies here), the AI could guess incorrectly, and the user could achieve a better result with enough experience and time.

I wish I understood this stuff better.

What (if any) part of the jpg looking better is due to the lossy compression  of jpg actually improving quality?  Areas that should be smooth in color are smooth in jpg, but grainy looking when converting arw to tiff.  Edges that are sharp in jpg are fuzzy when converting arw to tiff.

The arw format as produced by my alpha 7 III camera is a fairly stupid lossy compression, using 8 bits per pixel.  It carries much better information than a simple 8 bits per pixel, but much less information than uncompressed would have.  Clearly, it had at least 11 bits per pixel (maybe more) before the lame compression.  I'd far rather have pictures take twice the space and not lose that.  But I'm pretty sure the alpha 7 has no such option.

Likely the interpolation and noise reduction creating the image that feeds the jpg compression has the original (at least 11 bit per pixel) data, rather than the reconstruction of that data after lossy compression.  Maybe there is nothing one can do to the arw file that gets the interpolation and noise elimination as good as in the jpg, because you don't have the original data.  So you have a tradeoff of which information to lose when choosing jpg vs. arw, not an option to keep the maximum.

Sony's website tells you to use the ImagingEdge software they distribute to convert arw to tiff.  That does a slightly worse job than RawTherapee.  If knowledge of the characteristics of the sensor were the major difference, then ImagingEdge ought to do a better job.

For a tripod shot of a stationary target, the extra detail in dark areas in the arw format is such an advantage over jpg that arw is is clearly better.  For faster exposure, especially with a narrow aperture to increase depth of focus (forcing higher ISO), the garbage in the darker areas of the tiff outweigh the extra detail and the jpg is much better.  The exposure bracketing feature of this camera is garbage, so it is generally not the answer.  If I was just doing the images from tripod with decent lighting, even my own calls to libraw, that do a generally worse job than ImagingEdge, would be usable and better than just using the jpg. 

I also don't know what is lost (or gained) by delaying some of that processing until after stitching.  Since I don't know the concepts behind either better interpolation or noise reduction, I really have no clue what info is lost by delaying.  The gains might be more obvious:  Where stitching is making automatic decisions based on which image has an area of overlap "better", the noise reduction etc. is more likely to misguide it, vs choose the better before correcting.  Where the user must intervene to guide the process (such as in applying a different tone mapping by mask to different parts of the image) it may be much less work and avoid wasted effort if done after stitching.

 

[dkloi]

A7m3 does uncompressed 14 bit raw files https://helpguide.sony.net/ilc/1720/v1/en/contents/TP0001661967.html https://www.sony.co.uk/electronics/support/articles/00229990. The A7m3 sensor has a 14 bit ADC (IMX410 https://www.sony-semicon.com/files/62/pdf/p-13_IMX410CQK_Flyer.pdf https://www.techinsights.com/products/def-1806-802).

Raw files are monochrome in that each pixel only records one color channel, the final image (3 colours channels per pixel) has to be interpolated. Colorimetric conversion of the recorded raw pixel values is not trivial, it's not a simple matter of equating a "Red" pixel as equal to the red channel of the output file, the spectral response function of the colour filter array will not usually be exactly like the CIE colour matching functions. Unless the sensor spectral response curve satisfies the Luther's condition, then there is no perfect way of converting the sensor response to match exactly the ideal observer response under all illuminants and surfaces. In practice, one can use a colour conversion matrix that transforms the sensor RGB triplet into an output rgb triplet that is optimised over a set of real-world colours.

Interpolation algorithms have become extremely sophisticated. Rawtherapee allows you to try different ones. There are many different algorithms in the research literature you could implement as well.

I use the free (as in free beer, not open source) Capture One Express (for Sony) to convert A7m3 ARW files to 16-bit TIFF, usually with highlight recovery cranked to max as well as liberally apply shadow recovery. Also tweak black/white levels, clarity, chromatic aberration, vignetting, white balance, although I rarely modify the noise reduction settings.

Not sure what you are doing wrong to get worse looking converted raw files compared with out of camera JPGs.

[John Fine]

On Mon, Aug 1, 2022 at 8:36 PM 'dkloi' via hugin and other free panoramic software <hugi...@googlegroups.com> wrote:

A7m3 does uncompressed 14 bit raw files https://helpguide.sony.net/ilc/1720/v1/en/contents/TP0001661967.html https://www.sony.co.uk/electronics/support/articles/00229990.

Thanks!  I remember long ago seeing that in the documentation and being annoyed that it wasn't in the camera menu system and concluding it was about some different model.

But I checked again and it is right there where it is supposed to be.  Somehow I have been failing to notice it all this time.

I quickly took a picture that needs more range than jpg has.  Simply viewing the arw with the microsoft viewer shows just as much noise in the dark areas as I'm used to.  But converting arw to tiff with rawtherapee gave me much better results than I'm used to.  The result still has more noise than the JPG but only a little bit more.  So the compression seems to be destroying information needed for noise reduction, and I've been using raw incorrectly all this time.

[David W. Jones]

On 8/2/22 01:53, John Fine wrote:
>
>
> On Mon, Aug 1, 2022 at 8:36 PM 'dkloi' via hugin and other free
> panoramic software <hugi...@googlegroups.com

> <mailto:hugi...@googlegroups.com>> wrote:
>
> A7m3 does uncompressed 14 bit raw files
> https://helpguide.sony.net/ilc/1720/v1/en/contents/TP0001661967.html

> <https://helpguide.sony.net/ilc/1720/v1/en/contents/TP0001661967.html>
> https://www.sony.co.uk/electronics/support/articles/00229990
> <https://www.sony.co.uk/electronics/support/articles/00229990>.

>
>
> Thanks!  I remember long ago seeing that in the documentation and being
> annoyed that it wasn't in the camera menu system and concluding it was
> about some different model.
>
> But I checked again and it is right there where it is supposed to be.
> Somehow I have been failing to notice it all this time.
>
> I quickly took a picture that needs more range than jpg has.  Simply
> viewing the arw with the microsoft viewer shows just as much noise in
> the dark areas as I'm used to.  But converting arw to tiff with
> rawtherapee gave me much better results than I'm used to.  The result
> still has more noise than the JPG but only a little bit more.  So the
> compression seems to be destroying information needed for noise
> reduction, and I've been using raw incorrectly all this time.

I've been using RAW ever since I first got my old (no longer working)
Maxxum 7D. I saw advice (I think on this list) saying that the best
quality results came from starting with RAW, sticking with
high-dynamic-range color (16-bit per channel) for all processing, and
only produce a final JPG *when you're done*.

In other words, don't throw away any image information until you no
longer need it - when it's going out the door.

I've occasionally made an "HDR" image from JPGs, but histograms showed
spikes in each chennel, because JPG is missing 8 bits of image data per
channel.

I think it's also best to archive the source RAW files (they're like
negatives from the days of film) and your processing tool's settings
(PP3 files for Rawtherapee) in case I want to go back and rework them.
Perhaps I've learned better how to deal with noise, or adjust
color/contrast/etc since I processed it previously.

Thanks for the links for the online guides, learned a lot more about the
A7R IVA than I might have otherwise. I still want one. :)

--
David W. Jones
gnome...@gmail.com

wandering the landscape of god
http://dancingtreefrog.com
My password is the last 8 digits of π.