Vertical panorama exposure issue

[johnfi...@gmail.com]

I'm trying to assemble a specific vertical panorama, but also trying to learn methods for assembling a vertical panorama.

One major issue (that I'm furthest from figuring out on my own) is the exposure issue:

Taken on a cell phone (fixed F 1.9, Focal length 2.91, aperture 1.85, not sure those even have the same meaning as on big camera) with auto varying time:

Bottom photo is 0.025 sec exposing the street well and including some totally blown out sky.

Middle photo is 0.0083 sec exposing buildings well with sky blown out and street too dark.

Top photo is 0.0073 sec exposing the sky well.

I definitely don't want to compress the range.  I'd rather be dishonest than correct for that.

If the alignment were perfect, then doing no exposure correction would work with the blending from one image to the next smoothing the exposure transition.  But I haven't gotten the alignment near that good and would like to learn how to live with poor alignment.  That involves narrower transition zones in controlled places, which would then generate ugly exposure jumps.

What I think I want is a height based exposure shift in each photo (rather than the usual exposure to exposure nonlinear mapping to compress the range without losing much detail).  In each photo I think I want a smooth transition from making the bottom a little brighter to making the top a little darker.  Then the same final pitch of each photo would match in brightness.

I think this height based exposure issue may be a common problem, not just something in this set of three.

1) Have I misunderstood the issue and there is a very different basic solution?

2) Or is there a way already in hugin to adjust exposure the way I described?

3) If I need to use an external tool, how should I measure the amount I want to adjust exposure and/or compute it from the exposure times reported in EXIF?

4) What open source tool is easiest to use to pre-apply an exposure shift based on vertical position.

Also, is there good info on the other techniques for a narrow vertical panorama?  I don't know what projection works OK.  The best I found with a google search is to turn the whole thing sideways and use the usual methods for that kind of panorama, then rotate it afterward.  Is that really the best way to do narrow vertical?

I can put the photos and project etc. on my google drive and provide a link, if that would really help.  But I would prefer generic advice that fit this category of problem rather than specific help for this set of 3 images.

[Bruno Postle]

On Fri, 21 Jan 2022 at 21:47, johnfine2017 wrote:
>
> I'm trying to assemble a specific vertical panorama, but also trying to learn methods for assembling a vertical panorama.
>
> One major issue (that I'm furthest from figuring out on my own) is the exposure issue:
> Taken on a cell phone (fixed F 1.9, Focal length 2.91, aperture 1.85, not sure those even have the same meaning as on big camera) with auto varying time:

They have the same meaning except all the dimensions are smaller for a
small sensor, Hugin should interpret them ok.

> Bottom photo is 0.025 sec exposing the street well and including some totally blown out sky.
> Middle photo is 0.0083 sec exposing buildings well with sky blown out and street too dark.
> Top photo is 0.0073 sec exposing the sky well.
>
> I definitely don't want to compress the range. I'd rather be dishonest than correct for that.
>
> If the alignment were perfect, then doing no exposure correction would work with the blending from one image to the next smoothing the exposure transition. But I haven't gotten the alignment near that good and would like to learn how to live with poor alignment. That involves narrower transition zones in controlled places, which would then generate ugly exposure jumps.
>
> What I think I want is a height based exposure shift in each photo (rather than the usual exposure to exposure nonlinear mapping to compress the range without losing much detail). In each photo I think I want a smooth transition from making the bottom a little brighter to making the top a little darker. Then the same final pitch of each photo would match in brightness.

Hugin will by default map the exposure of the photos to the scene,
meaning that it will brighten your sky image and darken your ground
image so that they match the exposure of the middle image - the end
result (if you stitch to an 8bit format like JPEG) is that you may as
well have taken the photos on the middle exposure in the first place.

Actually most of the time this is wanted, most panoramas I shoot these
days are taken with my phone on auto-exposure, and I don't worry that
the individual photos have different exposure times, Hugin blends them
together seamlessly.

But it doesn't work if you want to see the clouds and undergrowth in
the same final panorama - for this you need tonal range compression -
things you can do:

1. Stitch to a file format that can hold this large dynamic range,
such as EXR, and 'tone map' this into a viewable 8bit image using a
tool like LuminanceHDR.

2. Force Hugin to ignore the exposure recorded in the photos (and
discard any fine-tuning by the Hugin photometric optimiser). Select
the photos in the Photos tab, edit the Eev and set them all to the
same number. It doesn't matter what this new Eev number is, but you do
need to set the Eev of the output panorama to the same number, or the
exposure of the output will be way-off (in the Preview window there is
a little handy arrow button marked 'EV' that resets the output
panorama exposure to the average of the input photos). This will give
you your clouds and undergrowth visible in the same panorama, but the
seams might not be exactly where you want them, so create masks that
remove the overexposed clouds in your middle photo and that remove the
underexposed buildings from your sky photo - don't try and follow the
edge exactly, leave a bit of sky and building for Hugin to use for the
transition - this will move the seam to the boundary between buildings
and sky, and with luck this seam is completely invisible.

> Also, is there good info on the other techniques for a narrow vertical panorama? I don't know what projection works OK. The best I found with a google search is to turn the whole thing sideways and use the usual methods for that kind of panorama, then rotate it afterward. Is that really the best way to do narrow vertical?

Most of the Hugin output projections are designed for very wide angle
of view scenes. You can try a fisheye projection like Stereographic,
these are symmetrical and treat horizontal the same as vertical. But
what you probably want is Transverse Mercator, this projection is
designed specifically for tall and narrow panoramas.

--
Bruno

[johnfi...@gmail.com]

On Friday, January 21, 2022 at 6:33:52 PM UTC-5 bruno...@gmail.com wrote:

Hugin will by default map the exposure of the photos to the scene,
meaning that it will brighten your sky image and darken your ground
image so that they match the exposure of the middle image - the end
result (if you stitch to an 8bit format like JPEG) is that you may as
well have taken the photos on the middle exposure in the first place.

That is what I most want to avoid.  It would ruin the panorama.

Actually most of the time this is wanted, most panoramas I shoot these
days are taken with my phone on auto-exposure, and I don't worry that
the individual photos have different exposure times, Hugin blends them
together seamlessly.

Maybe you mean horizontal panoramas, for which I can see that what you usually want is to undo what the camera did with varying auto exposure.

That is harder to believe for vertical.  I don't think this exposure issue I have is at all uncommon within vertical examples.

  for this you need tonal range compression -

That is also something I want to avoid.  Compressing the tonal range is not nearly as bad as clipping the range (undoing the auto exposure) would be.  But it is still bad.

I want to leave the well chosen exposure choices made by the phone roughly intact, but somehow avoid the problems that implies for seams.

 

In fact, for this specific panorama, I think with a lot of tedious work I could mask to boundaries at which a shift in exposure would look like something legit rather than like a defective seam.  I might even try that.  But I'd rather learn a better and more general method.

 Select
the photos in the Photos tab, edit the Eev and set them all to the
same number.

I hadn't thought that was needed.  I don't really understand other choices.  I thought I could opt out of any exposure correction more directly than that.

But if I misunderstood other choices, thanks for the reminder of a clean way to disable exposure correction.

 

what you probably want is Transverse Mercator, this projection is
designed specifically for tall and narrow panoramas.

Bruno

Thanks.  There were so many choices and names in a terminology I don't know.  It helps a lot to know the likely best to try and if necessary compare to turning the whole project sideways.

I still think that what I want for the main problem is to pre-shift the exposure within each photo based on relative (within each image) vertical position.  I think I know which open source tool will make that fairly easy (once I learn how to use it).  So I'll reread that documentation and experiment a lot (documentation makes it easy to see such effects are supported but hard to see exactly how you ask for them).

I'll assume you know hugin well enough that your not mentioning anything about doing that inside hugin implies it isn't there.

I don't mind using multiple tools.  I'd want to learn the purely-in-hugin way if it exists.  But otherwise whatever works.

[Bruno Postle]

On Sat, 22 Jan 2022 at 00:16, johnfine2017 wrote:
>
> I still think that what I want for the main problem is to pre-shift the exposure within each photo based on relative (within each image) vertical position. I think I know which open source tool will make that fairly easy (once I learn how to use it). So I'll reread that documentation and experiment a lot (documentation makes it easy to see such effects are supported but hard to see exactly how you ask for them).
> I'll assume you know hugin well enough that your not mentioning anything about doing that inside hugin implies it isn't there.
> I don't mind using multiple tools. I'd want to learn the purely-in-hugin way if it exists. But otherwise whatever works.

Yes, just tell Hugin to ignore the Exposure EXIF information and use
the images as-is - your camera has already decided how best to expose
the different parts of the scene, so you can use the images as
provided instead of letting Hugin adjust the exposure - this does what
you want entirely within Hugin.

--
Bruno

[johnfi...@gmail.com]

On Friday, January 21, 2022 at 6:33:52 PM UTC-5 bruno...@gmail.com wrote:

Most of the Hugin output projections are designed for very wide angle
of view scenes. You can try a fisheye projection like Stereographic,
these are symmetrical and treat horizontal the same as vertical. But
what you probably want is Transverse Mercator, this projection is
designed specifically for tall and narrow panoramas.

Stereographic is marginally the best of what I've tried so far.

Most of the projections make the overall image bulge outward at the top and bottom.  I'm pretty sure they are magnifying area at the top and bottom relative to the middle.  Some of them have vertical lines pretty close to vertical.

Turning the whole project sideways (which was easier than I had expected, just change the roll value on the anchor image and recalculate) did the reverse bulge, out in the center and had vertical lines very far from vertical.

Transverse Mercator just exaggerated all the faults  of turning the project sideways (with the more common projection choices):  The overall panorama bulged in the middle and vertical lines exaggerated that bulge.

Stereographic has vertical lines bulge out, but not nearly as badly as I get from turning the project sideways.

There isn't actually a lot in the panorama above or below the middle image.  The top and bottom images overlap each other a lot in addition to each mostly overlapping the middle.  The original middle image isn't very distorted (vertical lines bulge inward just a very small amount).  So I would expect to be able to get a less distorted panorama.  But I don't know about projections, so I don't know what to try.

It seems like there must be some metric wrong in the exif and/or some user error to have the whole panorama this distorted compared to the middle image alone, which covers most of the total.

[Bruno Postle]

On Sat, 22 Jan 2022 at 18:24, johnfine2017 wrote:

> There isn't actually a lot in the panorama above or below the middle image. The top and bottom images overlap each other a lot in addition to each mostly overlapping the middle. The original middle image isn't very distorted (vertical lines bulge inward just a very small amount). So I would expect to be able to get a less distorted panorama. But I don't know about projections, so I don't know what to try.
> It seems like there must be some metric wrong in the exif and/or some user error to have the whole panorama this distorted compared to the middle image alone, which covers most of the total.

The only projection that renders straight lines as straight is
'rectilinear', the other projections may give you more viewing angle,
but they do this by curving straight lines.

The photos should align seamlessly, if they look to have different
distortions then something is wrong.

--
Bruno

[johnfi...@gmail.com]

On Saturday, January 22, 2022 at 1:34:37 PM UTC-5 bruno...@gmail.com wrote:

The photos should align seamlessly, if they look to have different
distortions then something is wrong.

I apparently stated things unclearly.  I was discussing the target overlap of the three original images (mostly overlapped) not the panorama results.

The point of noting that very high overlap is that the total viewing angle of the panorama isn't much greater than that of the middle photo alone.  So it is surprising that the whole panorama must be so much more distorted than the original middle photo.

I haven't yet gotten near seamless alignment.  Sections that are blended are terrible.  But that is an entirely separate issue from the distortion of the projection.

In the past (before you explained things about the optimize process to me) I got past poor alignment with lots of masking to control where the poor blending occurs.  Masking might also answer the original exposure question of this thread.  I'm currently trying masking as a solution for comparison to other solutions:  Better control points and optimize to improve alignment combined with using an external tool to shift the exposure based on vertical coordinate.

[johnfi...@gmail.com]

I finally went back to that set of 3 photos and got a good result.  I'm not happy with the methods required.  There is probably a better way.  I'm open to advice on what to try next time.  There should be a better way, so in any case I'll look through the relevant parts of the source code to see what could be made to work better.

There are three basic issues here, two of which were solved with masks.  But editing those masks required fixing a flaw in the mask editing code (see my recent new thread).

1) Exposure.  Each photo has the correct average exposure for its part of the scene and I didn't want to change that much.  But the top photo had shorter exposure than the middle which is shorter than the bottom.  So sections from any one photo have good exposure unchanged.  But narrow seams would highlight the exposure difference, while broad seams would be garbage due to issue (2).  I still think the better general solution would be to first apply an exposure vertical gradient to each photo (in a separate program if necessary).  But I didn't find time to try that.  A mask worked a lot better than I expected, to force narrow seams in places that don't highlight the exposure jumps.

2) Alignment.  I never got good enough alignment for real blending of any sections.  My solution to (3) made the misalignment 1.5 times as bad.  The masks intended to fix (1) fixed (2) with no extra effort.

3) Distortion.  The resulting image was very distorted.  I tried most of the projections and some were worse and none were significantly better than the default.  After much experimentation, I started without the lens a,b,c,d,e,g,t parameters, then rotated the anchor image -90 and reoptimized (which didn't hurt alignment much), then cut the Hfov in half, and reoptimized which destroyed alignment (I assume because the original Hfov was correct) then added  a,b,c,d,e,g,t parameters to get back to decent alignment (I'd really like to know what those are doing and how they compensate for an intentionally wrong Hfov.  But I'm already investigating in too many different subtopics at once).  Finally, I stitched with Cylindrical and used an external program to rotate +90.

Given the distortion this had when simply done sideways with Cylindrical, I expected that cutting the Hfov would do what it did.  But I have no clue why it had that distortion.  It seems like there should be a less distorting projection without needing to mess with the alignment.

[dkloi]

If provide the raw files, maybe we could give it a shot. Are you able to show what you are getting?

In high contrast situations, I will use exposure blending with enfuse and this gives quite natural looking results.

[johnfi...@gmail.com]

On Friday, February 11, 2022 at 3:24:32 PM UTC-5 dkloi wrote:

If provide the raw files, maybe we could give it a shot. Are you able to show what you are getting?

Thanks for the offer.  But this time I'd rather not share the photos (and they were not raw, they were from my cell phone, which has a much worse camera then most cell phones).

I hope on my vacation (which will be later this month) to be using my new camera and mostly with a 105mm lens shooting mostly scenes that ought to be much wider, and so will need hugin when I get home (and those will be raw).  I expect some of those will be better examples with which to ask for help.

In high contrast situations, I will use exposure blending with enfuse and this gives quite natural looking results.

I don't yet have a good understanding of what enfuse and enblend are doing  nor how to use their options.  I will gradually be learning that.  But this panorama has large continuous areas of high contrast detail that need to be split by seams.  I think I understand the concept of a program identifying higher contrast areas and forcing the seams to not be there.  But I can't see how that could work in this example and I haven't yet gotten it to work in examples where I think it ought to.

This one needed avoiding exposure correction, so I chose "Exposure fused from any arrangement" (possibly due to not really understanding the choices).  But I don't have stacks and the panorama would be wrecked by any exposure correction that I've seen in action.  Many upper parts of the image need to end up darker than many lower parts even though in the real view the darkest thing in the upper third was brighter than the brightest of the lower third.  As I understand it, exposure correction starts with undoing the "error" from combining images with very different exposure and then might mitigate the consequences by a nonlinear mapping of a brightness range with extra bits back onto a normal brightness range.  But in many of my photos, no such mapping can fix the dynamic range problem.  Anyway, that choice on the Stitcher tab selects both enfuse and enblend, then I'm unclear on what each does in this example.

Before the masks were improved, fusing or blending (I don't understand which) was happening in areas with great detail and with about a 1.5 pixel misalignment between the images in that area.  I couldn't get better results than that from optimize (even before I starting forcing Hfov to be wrong).  That was enough misalignment to turn sharp sections of both originals into a blurred section of the result.