[hugin-ptx] multi-image tiffs generate tiff warnings? (was: error logs)

[Harry van der Wolf]

Hi Micah,

2010/5/13 Micah Chaney <micah...@gmail.com>

harry,

here are the error logs for the TIFF lib error, and hugin crashing when opening a PTO with the preview window open.  i didn't want to clutter the google group with this, its extremely long.

i will continue to give you feedback as i find bugs.  thanks for your time and effort.

Unfortunately I can't do much with your crash log as it doesn't give me a clue where it fails.

Your tiff warning is due to the fact that your tiff contains multiple tiff files.

For the others in this list, the repeated warnings are:
"TIFF library warning
tiff module: TIFFReadDirectory"

I'm sure that your tiffs are created from a DSLR (a canon or nikon?) either directly, or via the RAW convertor delivered with your camera. If you open such a tiff in OSX Preview you will see that it has 2 images. A full resolution 16bit tiff, most probably with an alpha-mask, and a small preview tiff. Programs that do support this can load the small preview fast and display the large tiff a bit later, or show it as a preview in a file manager or thumbs gallery. The library automatically loads the big (default) image from the tiff and not the preview.

If have seen this also with enfuse often. Enfuse can't handle these tiffs and crashes on it. I had to write a patch for my ImageFuser(1) program to overcome that behavior from enfuse.
Even more: I think that the only reason that hugin does not crash (on all platforms) when having to handle these kind of tiffs in stacked panos, is because nona (and align_image_stack) are not that intelligent and just grab the first image in the tiff that they encounter. And then they deliver a single (cropped) tiff to enfuse and enblend which than can handle these new intermediate "single tiff" images.

Please let me know whether I'm correct in my analysis of the tiff behavior.


Harry

(1): http://imagefuser.sourceforge.net/

--
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugi...@googlegroups.com
To unsubscribe from this group, send email to hugin-ptx+...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/hugin-ptx

[cspiel]

Harry -

On May 13, 11:47 pm, Harry van der Wolf <hvdw...@gmail.com> wrote:
> For the others in this list, the repeated warnings are:
> "TIFF library warning tiff module: TIFFReadDirectory"

It would be terribly helpful to know
which program issues this message, wouldn't it?

> I'm sure that your tiffs are created from a DSLR (a canon or nikon?) either
> directly, or via the RAW convertor delivered with your camera. If you open
> such a tiff in OSX Preview you will see that it has 2 images. A full
> resolution 16bit tiff, most probably with an alpha-mask, and a small preview
> tiff.

Again some definitive answer would be
appropriate, e.g., what does tiffinfo(1) say
about the image files?

> If have seen this also with enfuse often. Enfuse can't handle these tiffs
> and crashes on it. I had to write a patch for my ImageFuser(1) program to
> overcome that behavior from enfuse.

Enfuse (and Enblend) ought to process
multi-directory TIFF files. In particular they
should not crash. If fact, there never has been
any failure reported on multi-layer images.

Of course it is utter nonsense to feed a
2-directory TIFF image as described above (full
quality image plus thumbnail) into Enfuse (or
Enblend for that matter), because the different
layers are not meant to be fused (or blended),
whereas both Enblend and Enfuse process all
directories. This behavior is documented in
Section 1, "Overview", of the Enblend and the
Enfuse manual.

> Even more: I think that the only reason that hugin does not crash (on all
> platforms) when having to handle these kind of tiffs in stacked panos, is
> because nona (and align_image_stack) are not that intelligent and just grab
> the first image in the tiff that they encounter. And then they deliver a
> single (cropped) tiff to enfuse and enblend which than can handle these new
> intermediate "single tiff" images.
>
> Please let me know whether I'm correct in my analysis of the tiff behavior.

Many applications ignore all but the
first directory in a TIFF file.

However, the true support of multi-directory
TIFFs implies quite an extra effort. We have
not even reached a conclusion on the syntax to
process multi-layer images for Enblend and
Enfuse! See, e.g. the brief "discussions" at

http://sourceforge.net/tracker/?func=detail&aid=2885858&group_id=123407&atid=696409
http://sourceforge.net/tracker/?func=detail&aid=2977905&group_id=123407&atid=696412


/Chris

[Harry van der Wolf]

Hello Chris,

2010/5/16 cspiel <csp...@freenet.de>

Harry -

On May 13, 11:47 pm, Harry van der Wolf <hvdw...@gmail.com> wrote:
> For the others in this list, the repeated warnings are:
> "TIFF library warning tiff module: TIFFReadDirectory"

       It would be terribly helpful to know
which program issues this message, wouldn't it?

You are completely right. The message originated from a private mail which I redirected to the hugin mailing list. As it was a very long mail (containing a crash log) I copied only the "necessary" parts, but indeed too brief.
When running external binaries like enblend/enfuse, Hugin redirects stdout and sterr messages and displays them in a progress window. That's where the tiff warnings are displayed. Binaries exiting with 0 (no error) make the progress window close nicely, warnings and everything else are displayed as a warning/error message, after having closed the progress window. In this case a warning error was displayed when Hugin was reading the tiff images, that is before enblend and/or enfuse are being used.
 

> I'm sure that your tiffs are created from a DSLR (a canon or nikon?) either
> directly, or via the RAW convertor delivered with your camera. If you open
> such a tiff in OSX Preview you will see that it has 2 images. A full
> resolution 16bit tiff, most probably with an alpha-mask, and a small preview
> tiff.

       Again some definitive answer would be
appropriate, e.g., what does tiffinfo(1) say
about the image files?

I agree but please note that most Mac users do not have the tiff libraries/tools on their Mac. I wanted to ask Micah to privately send me one/some of his tiffs for further analysis, but I have been busy the last days with both ImageFuser and PTBatcherGui/Hugin builds, and as the warnings were only warnings and not (fatal) errors I put the priority a bit lower. To be continued.


 

> If have seen this also with enfuse often. Enfuse can't handle these tiffs
> and crashes on it. I had to write a patch for my ImageFuser(1) program to
> overcome that behavior from enfuse.

       Enfuse (and Enblend) ought to process
multi-directory TIFF files.  In particular they
should not crash.  If fact, there never has been
any failure reported on multi-layer images.

My wording is not accurate (or plain incorrect actually). Enfuse and enblend don't crash, but both enfuse and enblend exit with error messages. For the user that's the same. He/She has tiff images most programs handle without problem, either because most of them don't recognise them as such like in this case align_image_stack and nona, or they do recognise them properly and act accordingly (macs preview, photoshop, etc.). But enblend and enfuse don't. Again: I know the difference, but the user either doesn't know or doesn't care.

 

Of course it is utter nonsense to feed a
2-directory TIFF image as described above (full
quality image plus thumbnail) into Enfuse (or
Enblend for that matter), because the different
layers are not meant to be fused (or blended),
whereas both Enblend and Enfuse process all
directories.  This behavior is documented in
Section 1, "Overview", of the Enblend and the
Enfuse manual.

I agree as such that these single image files containing mulit-tiffs are not meant at all to be fused or blended, but I think we should not close our eyes and ignore the existence of these images. More and more cameras and raw-convertors create these images and they are slowly becoming the new tiff standard creared by DSLR and high-range compact cameras (my view on the near future). Based on the size difference of the "original" and the preview, and the fact that the original has an alpha-mask and the preview not, it should be possible to recognise these kind of tiffs. Unfortunately I lack the knowledge to write a patch for this, let alone that enblends/enfuses current behavior is "by design" to handle these tiffs now.
The mentioned ImageFuser patch is nothing more than letting sips, a command-line tool comparable to ImageMagick convert and available on every mac, split the images.
I fully understand your point and the implications of implementing the functionality to try to handle these images. And most probably new exceptions and thereby "(end user) bugs" will arise when it should be implemented.

 

> Even more: I think that the only reason that hugin does not crash (on all
> platforms) when having to handle these kind of tiffs in stacked panos, is
> because nona (and align_image_stack) are not that intelligent and just grab
> the first image in the tiff that they encounter. And then they deliver a
> single (cropped) tiff to enfuse and enblend which than can handle these new
> intermediate "single tiff" images.
>
> Please let me know whether I'm correct in my analysis of the tiff behavior.

       Many applications ignore all but the
first directory in a TIFF file.

Agreed. See my comments above.

 

However, the true support of multi-directory
TIFFs implies quite an extra effort.  We have
not even reached a conclusion on the syntax to
process multi-layer images for Enblend and
Enfuse!  See, e.g. the brief "discussions" at

       http://sourceforge.net/tracker/?func=detail&aid=2885858&group_id=123407&atid=696409
       http://sourceforge.net/tracker/?func=detail&aid=2977905&group_id=123407&atid=696412


/Chris

I understand and respect your opinion. For the time being it remains an inconvenient and unsatisfactory situation for the users.

Sometimes I behave like a user in this mailing list, sometimes as a "builder" or troubleshooter (hardly ever as a developer as I'm simply not).
This double role sometimes causes mails that are "easy"  for the end user (at least I hope so) and might be unclear or incorrect to the developer.
Even though I try to be as accurate as possible I might (will?) generate more of these unclarities in the future due to my double role.

best,
Harry

 

[cspiel]

Harry -

THX for your detailed answer.

On May 16, 3:05 pm, Harry van der Wolf <hvdw...@gmail.com> wrote:
> I understand and respect your opinion. For the time being it remains an
> inconvenient and unsatisfactory situation for the users.

I'm not happy with the current
capabilities of Enblend/Enfuse when it comes to
multi-layer images, either. They allow them,
but they do not support them -- to use
Stroustrup's distinction.

A quick-and-dirty solution to your problems
could be a command-line option that dumbs down
the new algorithm, e.g.
--first-layer
And given this option, Enblend/Enfuse will only
process the first directory of _each_ TIFF image
that has been passed to them.


I have played around with Ingemar's
suggestion to add kind of an index expression to
each input file, like
image1.ext[m] image2.ext[n] ...
where m, n, ... are the layers' indexes. This
syntax, however, raises at least two problems.

(i) What happens with filenames that end with
with exactly our index expression? Do we need a
special quoting syntax, and how ghastly should
it look like?

(ii) Attaching the index expression directly to
the filename knocks out the shells' filename
expansion. This is as ugly as it is hard to
circumvent. Consider:
enblend `for x in image*.tif; do echo $x[1]; done`


Another idea that has come to my mind is
introducing layer-selection operators as
arguments. I'm not sure yet, whether to make
them prefix or postfix. The added
complexity would drive Dr. Cuddy up the wall and
through the roof -- principally a good thing. :-)

Anyhow, the essence is separating the image
filenames from the layer specifications. This
makes the new command line look like the
following (show for prefix operators):

enblend '[n]' image11.ext image12.ext ... \
'[m]' image21.ext image22.ext ... \
...

The layer-selection operator is valid for all
files to its rhs until another operator shows
up, thus minimizing the number of keystrokes,
yet retaining all the power of layer-selection
expressions.

BTW, problem (i) remains, but at least we got
rid of (ii). And please remember, that we'd had
to teach the response files the new operator
syntax, too.


Cheers,
Chris