How does "Straighten" really work?

[ChameleonScales]

I'm wondering how the "Straighten" button in the Move/Drag tab works. My assumption is that it goes as follow:

• determine a non-flat circle that passes through each image center
• flattens the circle through some average calculation
• determine the pitch & roll of that flat circle
• applies the inverse of those pitch & roll on each photo.
  

Note that I haven't looked at the code and the wiki doesn't go in much detail, so I'm only assuming from my own experience.

[ChameleonScales]

I did not make a very thoughtful assumption. Let me make a new one for what I think it does:

• determine a single set of pitch and roll values that is closest to as many photos as possible
• apply the inverse of this set of values to each photo
  

Am I getting closer ?

It's been a little more than a year. Does anyone have an answer ?

[David W. Jones]

Hmm, haven't a real clue, but maybe this will help?

https://hugin.sourceforge.io/tutorials/two-photos/en.shtml

Are you asking about the process behind it? The above link lists the steps of the process the Straighten button applies.

This link discusses the Straighten tool (at the bottom of the page):

https://wiki.panotools.org/Hugin_move_drag_tab

"Straightening the panorama optimises the roll and pitch of the input images without changing their relative positions, levelling the panorama in the process. This normally produces good results; if you need more accurate positioning, try adding vertical control points in the Hugin Control Points tab and reoptimise."

Hope that helps!

-- 
David W. Jones
gnome...@gmail.com
wandering the landscape of god
http://dancingtreefrog.com
My password is the last 8 digits of π.

[ChameleonScales]

Thank you. That answered part of my question.

I'm still wondering how it determines the orientation of the panorama though.

Let's say I take photos with a panoramic head at random y,p,r orientations (so there can even be photos taken in portrait format, others in landscape and all the in-between angles).

If I stitch them in Hugin and then click on "Straighten", will it determine a "dominant" orientation so that the equator of the panosphere is closest to as many photo centers as possible or will it do something else ?

I may do some tests to understand it better.

--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/466cc222-e495-4ec9-b9e0-18e30105ec08%40gmail.com.

[Saleh Saeed]

Hi,

I cannot answer all of your questions but I hope my description will help you.

Assume you have multiple images (Img1, Img2, Img3, Img4, Img5) covering a 360x180 field-of-view. The images are stitched using hugin and projected onto a full spherical surface. The assumption can be extended to images with limited field-of-view and different projection surfaces. Hugin does the stitching with optimization (or bundle adjustment) of orientation (yaw-y, pitch-p, roll-r) and lens correction parameters by using control points and line points.

Control points: The control points are matching points in the overlap region across multiple images (Read more about matching control points here [1]). Let's assume Img1, Img2 and Img4 have an overlapping area with one matching point (x_Img1, y_Img1) <-> (x_Img2, y_Img2) <-> (x_Img4, y_Img4).  The SIFT or similar features match corresponding points in other images, irrespective of the orientation, size etc [1]. So it does not matter if your photos are landscape or portrait or of different size. Assume that other images (Img3,Img5) have matching control points as well but for simplicity, I am not adding more points here.

Line points:  Assume that Img4 has two line points (one horizontal line H1(x1_Img4,y1_Img4 <-> x1_Img4,y2_Img4)  and one vertical line V1(x3_Img4,y3_Img4 <-> x4_Img4,y3_Img4)). Hugin extracts all edged lines per image and checks for the angles, arcs etc to find line points which are exactly horizontal or vertical. I don't have a link to documentation. You can check it in the code to get more details.

Orientation: Assume that the orientations (y,p,r) for images are Img1<->(y1,p1,r1), Img2<->(y2,p2,r2), Img3<->(y3,p3,r3), Img4<->(y4,p4,r4), Img5<->(y5,p5,r5)

Lens Correction: Lens correction can have multiple parameters but for the sake of simplicity, assume that there is no lens correction.

Optimization: The optimization in hugin starts with 0 initial orientation (y1,p1,r1,y2,p2,r2,y3,p3,r3,y4,p4,r4,y5,p5,r5 = 0). It varies the orientation in each iteration and checks for the distances between matching control points (x_Img1-x_Img2, x_Img1-x_Img4, x_Img2-x_Img4, y_Img1-y_Img2, y_Img1-y_Img4, y_Img2-y_Img4) and line points (y1_Img4-y2_Img4, x3_Img4-x4_Img4). It's like a jigsaw puzzle where Hugin tries different incremental values for y p r to check if the control and line point distance is reduced or increased. 

The straighten button will probably not determine a dominant orientation if it is using the optimization. There is one way to search for dominant orientation as explained in section 5 of [2].

 

You can test the straightening process by rotating your photos.

[1] SIFT or similar features https://link.springer.com/article/10.1023/B:VISI.0000029664.99615.94

[2] Panorama stitching in openCV, a bit different from Hugin https://link.springer.com/article/10.1007/s11263-006-0002-3

To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/84aO0BheH2Obf-0CMZ3jdw4W0sZXefAdRoS0AV45tCPOnzOzU_-REcbKhvx00vBuKg2JtguamYhysRTBGlPpeUk07VNms7SImuw0l7Pt59c%3D%40protonmail.com.

[Sean Greenslade]

On Tue, May 14, 2024 at 12:58:52PM +0000, 'ChameleonScales' via hugin and other free panoramic software wrote:
> Thank you. That answered part of my question. I'm still wondering how
> it determines the orientation of the panorama though.
>
> Let's say I take photos with a panoramic head at random y,p,r
> orientations (so there can even be photos taken in portrait format,
> others in landscape and all the in-between angles). If I stitch them
> in Hugin and then click on "Straighten", will it determine a
> "dominant" orientation so that the equator of the panosphere is
> closest to as many photo centers as possible or will it do something
> else ?
>
> I may do some tests to understand it better.

I did a quick stare at the source code (namely
hugin_base/algorithms/basic/StraightenPanorama.cpp), and from what I can
tell, it does the following.

All the images in a panorama are iterated over and an "up" vector for
each image is produced based on the local coordinate system of the
image. The "up" vector of each image is processed into a covariance
matrix so that an average / composite "up" vector for the entire
panorama can be calculated. A rotation is then calculated to rotate the
entire panosphere so that its "up" matches this composite "up" of all
the images.

In short, it figures out the way to rotate the whole panorama (without
moving any individual images with respect to each other) such that all
of the individual images are as level as possible.

Note that there are a bunch of caveats mentioned in the code. It
attempts to detect portrait-rotated images based on exif data, so if
that is not present / correct, it will not work.

If any image translation variables are in use, they will not be
accounted for and the straighten will likely fail.

Since it does not move individual images, just the whole pano,
straighten works best on panos that have a good geometric optimization.

And finally, there is the baked-in assumption that leveling the
individual images will actually level the panorama. If the input images
were not shot level with respect to the horizon, then straighten will
not help.

--Sean

[ChameleonScales]

Thank you very much for this in-depth explanation. One aspect I wasn't 100% sure about and which your explanation seems to confirm, is that if you shoot your photos on a tilted plane like this (brown=earth, blue=sky) :

then the Straighten button should produce a panorama with a tilted horizon like this :

However, if you shoot while keeping the horizon level, even if you tilt each photo up or down (drawing a cone shape with your camera) like this :

and if for some reason your first geometric optimization in Hugin produces a tilted result like this (which happens to me sometimes) :

then the Straighten button should level the horizon but not move the photo centers on the equator of the sphere (thus keeping the "conical" tilt of your shoot) :

To my experience and understanding of your explanation, this is how it behaves and what makes the most sense to me.