Hugin Translation x PTGui Viewpoint optimization

[Carlos Eduardo G. Carvalho (Cartola)]

Hi,

recently I've been doing many hand held panoramas with a philopod. Some other people who do the same and use PTGui usually make comments about the PTGui "viewpoint correction". They say that it makes very good results, allowing more tolerance in the philopod rotation.

In this page there is an example of a displaced nadir picture being added to a stitch using this optimization. I guess I've already done that once optimizing translation of the nadir picture, but usually it works better if I work with it separately adding vertical and horizontal lines to correct its perspective and then add it to the complete stitch.

I've been trying to use translation to improve a final result of a handheld panorama, but it always mess up everything, making the result worse that it was, showing like a quad, like this:


​

No matter what I try, only one picture, all of them, only some translation parameters or all with or without positions, it never improves the stitch.

Is there a way to use it in spherical panoramas? Would it be the viewpoint correction shown in PTGui? Could we have a function like that in Hugin?

Thanks!

Carlos E G Carvalho (Cartola)
http://cartola.org/360
http://www.panoforum.com.br/

[panostar]

Not long ago I worked out a technique for stitching hand held panoramas (no philopod or plumb line) using viewpoint correction in PTGui that has been surprisingly effective.  Normally, a hand held nadir shot aligns with images shot from a tripod mounted camera without any difficulty, using vp correction for the nadir alone.  When all the images are hand held, the horizontal row of images won't themselves align with each other well enough to accept the nadir image alignment.  The trick I have devised involves adding the horizontal row of images a second time into the project.  The additional images are treated as nadir shots and aligned with viewpoint correction applied.  Image areas that are not part of the flat floor/ground are masked away.  A few control points around the edge of this floor area are assigned to link it to the normal row of images.  The features used for this are in the reasonably far distance and therefore relatively immune from parallax effects.  The floor areas in the normal horizontal row of images are masked away.  The combined sets of horizontal images then provide a nicely aligned floor area to which the handheld nadir shot can be aligned very well.

This technique can be startlingly successful.  This is the first panorama I tried this technique on:

http://www.johnhpanos.com/spherical/peterbor2/peterbor2.html

Paving is notoriously difficult to stitch well in hand held panoramas, yet what you see in the above panorama is straight out of the stitcher without any Photoshop post processing to correct minor glitches (other than coping with 3D features like people, who are not part of the flat floor and therefore distorted by the viewpoint correction transformation).

I should think the same technique could be used in Hugin too.

John

[Bruno Postle]

On Tue 14-Oct-2014 at 09:54 -0300, Carlos Eduardo G. Carvalho (Cartola) wrote:
>
> recently I've been doing many hand held panoramas with a philopod.
> Some other people who do the same and use PTGui usually make
> comments about the PTGui "viewpoint correction". They say that it
> makes very good results, allowing more tolerance in the philopod
> rotation.

> I've been trying to use translation to improve a final result of a
> handheld panorama, but it always mess up everything, making the
> result worse that it was, showing like a quad, like this:

The Hugin mosaic mode fits images to a flat plane, usually in the
middle of the output panorama. So if your mosaic is a floor rather
than a wall, you need to arrange your output panorama so the nadir
is in the centre of the canvas - This is probably not what you want
a spherical panorama to look like.

Hugin 2014.0 has Tpy and Tpp parameters to tilt this plane and put
it anywhere you like - This is what you need.

--
Bruno

[Brandon]

Could a person stitch the pano together with everything other than the handheld down shot. Stitch it take the result and turn it into a cube. Then take the floor side of the cube and using the mosaic stuff that hugin has put the hand held down shoot onto the floor side of the cube?

I have not had a down shot bad enough that I have done it yet, but I have wondered about it and if that would be possible.

[Carlos Eduardo G. Carvalho (Cartola)]

2014-10-14 19:51 GMT-03:00 Brandon <bra...@flyingtsalers.com>:

Could a person stitch the pano together with everything other than the handheld down shot. Stitch it take the result and turn it into a cube. Then take the floor side of the cube and using the mosaic stuff that hugin has put the hand held down shoot onto the floor side of the cube?

I have not had a down shot bad enough that I have done it yet, but I have wondered about it and if that would be possible.

This is more or less what I've done here: http://cartola.org/panoforum/viewtopic.php?f=19&t=32 - it is in Portuguese, but the images show the workflow. A cube and a displaced nadir with perspective corrected using vert and horiz lines. It works perfectly, but nowadays I am frequently using no tripod.

[Carlos Eduardo G. Carvalho (Cartola)]

2014-10-14 18:11 GMT-03:00 Bruno Postle <br...@postle.net>:

The Hugin mosaic mode fits images to a flat plane, usually in the middle of the output panorama.  So if your mosaic is a floor rather than a wall, you need to arrange your output panorama so the nadir is in the centre of the canvas - This is probably not what you want a spherical panorama to look like.

Hugin 2014.0 has Tpy and Tpp parameters to tilt this plane and put it anywhere you like - This is what you need.

Tks Bruno.

[Carlos Eduardo G. Carvalho (Cartola)]

2014-10-14 16:07 GMT-03:00 panostar <houghto...@gmail.com>:

Not long ago I worked out a technique for stitching hand held panoramas (no philopod or plumb line) using viewpoint correction in PTGui that has been surprisingly effective.  Normally, a hand held nadir shot aligns with images shot from a tripod mounted camera without any difficulty, using vp correction for the nadir alone.  When all the images are hand held, the horizontal row of images won't themselves align with each other well enough to accept the nadir image alignment.  The trick I have devised involves adding the horizontal row of images a second time into the project.  The additional images are treated as nadir shots and aligned with viewpoint correction applied.  Image areas that are not part of the flat floor/ground are masked away.  A few control points around the edge of this floor area are assigned to link it to the normal row of images.  The features used for this are in the reasonably far distance and therefore relatively immune from parallax effects.  The floor areas in the normal horizontal row of images are masked away.  The combined sets of horizontal images then provide a nicely aligned floor area to which the handheld nadir shot can be aligned very well.

This technique can be startlingly successful.  This is the first panorama I tried this technique on:

http://www.johnhpanos.com/spherical/peterbor2/peterbor2.html

Paving is notoriously difficult to stitch well in hand held panoramas, yet what you see in the above panorama is straight out of the stitcher without any Photoshop post processing to correct minor glitches (other than coping with 3D features like people, who are not part of the flat floor and therefore distorted by the viewpoint correction transformation).

I should think the same technique could be used in Hugin too.

John

Tks John. I guess I've already seen a page where you describe this workflow. Quite interesting. In my workflow I don't have a nadir picture. I am usually doing 4 pics with pitch variation and I am searching for ways to minimize eventual parallax errors. I use to see Zeljko Solectic panos and he frequently mentions the use of viewpoint correction in similar situations.

I hope I will have time until next week to study all options suggested here and if I come to any conclusion I will share here.

Thank you all!

[Nicolas Pelletier]

So when adding this Nadir with a different viewpoint, you would optimize for Yaw Pitch and Roll, and also Tpy and Tpp and this should do the trick. Correct? Or am I missing something?

Thanks, this will be quite usefull.

[Bruno Postle]

On Mon 17-Nov-2014 at 17:14 -0800, Nicolas Pelletier wrote:
>So when adding this Nadir with a different viewpoint, you would optimize
>for Yaw Pitch and Roll, and also Tpy and Tpp and this should do the trick.
>Correct? Or am I missing something?

I wouldn't optimise them, at least at first, you might have to do
this eventually if the floor isn't level enough.

Try setting 'plane yaw' to '-90' for all your nadir photos and see
if this lets you assemble the nadir as a mosaic (untested, you may
have to use '90' instead of '-90', let us know).

--
Bruno

[Nicolas Pelletier]

Hi,

My tests were inconclusive.

To be sure I express what I'm trying to do, I have a spherical panorama, 6 around, and one up (zenith). They stich flawless. I tool a shot of the nadir from my tripod, but with about a 45 degree angle (exactly as they would do here: http://www.ptgui.com/examples/vptutorial.html )

I "removed" the fisheye and made it into a recilinear before adding (and set the lens info accordingly).

And I was trying to achieve the same effect as the viewpoint correction you see in the link to stich the nadir.

I tried with the tpy parameter at both 90 and -90. I saw no effect on the opengl viewer. But I don't know if I should. I tried to optimize those values (tpy and tpp) but nothing moved there. Only when I let the optimizer also work on TrX, TrY and TrZ did this value change (it went from 90 to 68). But the match was not better (it was actuall worse then when I tried using the lens parameter d and e to compensate).

So my conclusions are that I may trying to do it the wrong way, or I may not really inderstand what these values do.

Thanks for the help.

nick

[Bruno Postle]

On Tue 18-Nov-2014 at 18:44 -0800, Nicolas Pelletier wrote:
>
> My tests were inconclusive.
>
> To be sure I express what I'm trying to do, I have a spherical
> panorama, 6 around, and one up (zenith). They stich flawless. I
> tool a shot of the nadir from my tripod, but with about a 45
> degree angle (exactly as they would do here:
> http://www.ptgui.com/examples/vptutorial.html )
>
> I "removed" the fisheye and made it into a recilinear before
> adding (and set the lens info accordingly).

This shouldn't be necessary, Hugin can map a fisheye image to a
plane.

> And I was trying to achieve the same effect as the viewpoint
> correction you see in the link to stich the nadir.
>
> I tried with the tpy parameter at both 90 and -90. I saw no effect
> on the opengl viewer. But I don't know if I should. I tried to
> optimize those values (tpy and tpp) but nothing moved there. Only
> when I let the optimizer also work on TrX, TrY and TrZ did this
> value change (it went from 90 to 68). But the match was not better
> (it was actuall worse then when I tried using the lens parameter d
> and e to compensate).

Again this is untested, but this is what I would do:

Align your equirectangular panorama using just your 'normal' photos,
leaving a hole for the nadir shot.

None of these photos want to move any further, so you need to set
'custom parameters' and in the Optimiser tab turn off all
optimisation for _all_ your existing pictures.

Add your nadir photo to the project, and create some control points
linking it to the existing photos.

Set 'plane pitch' for this photo to -90 (I said 'plane yaw' before,
oops!), nothing will appear to change, since this is just
positioning the virtual plane that the photo will be mapped-to.

For just this photo, optimise roll, pitch, yaw, X, Y, Z - You don't
need to optimise lens parameters as this photo has been taken with
the same lens as all your other photos (and you didn't convert it to
rectilinear).

If this doesn't align the nadir shot, it could be because there are
too many parameters being optimised, try and drag it to the
approximate position and optimise again. Also you may have better
luck setting 'plane pitch' to 90 instead of -90, though I'm not sure
it makes any difference.

If the floor isn't perfectly level, you can try optimising
'plane yaw' and 'plane pitch' to see if this gets better results,
but only do this if it almost good enough already.

> On Tuesday, November 18, 2014 12:36:34 PM UTC-5, Bruno Postle wrote:
> > On Mon 17-Nov-2014 at 17:14 -0800, Nicolas Pelletier wrote:
> > > So when adding this Nadir with a different viewpoint, you
> > > would optimize for Yaw Pitch and Roll, and also Tpy and Tpp
> > > and this should do the trick. Correct? Or am I missing
> > > something?
> >
> > I wouldn't optimise them, at least at first, you might have to
> > do this eventually if the floor isn't level enough.
> >
> > Try setting 'plane yaw' to '-90' for all your nadir photos and
> > see if this lets you assemble the nadir as a mosaic (untested,
> > you may have to use '90' instead of '-90', let us know).

> > > > The Hugin mosaic mode fits images to a flat plane, usually
> > > > in the middle of the output panorama. So if your mosaic is
> > > > a floor rather than a wall, you need to arrange your output
> > > > panorama so the nadir is in the centre of the canvas - This
> > > > is probably not what you want a spherical panorama to look
> > > > like.
> > > >
> > > > Hugin 2014.0 has Tpy and Tpp parameters to tilt this plane
> > > > and put it anywhere you like - This is what you need.

--
Bruno

[Nicolas Pelletier]

Hi,

It almost worked, but still had a little wiggle that I could not get rid of. But the instructions were clear anough for me to understand.

If my understanding is correct, there is another set of transforms that are missing, perspective correction (something like here http://tutorial9.s3.amazonaws.com/uploads/2008/04/skew-distort-perspective.gif ). Probably another one to compensate for the distance that is different. But I'm not an expert here.

This would enable a viewpoint correction. I though I had seen a post by somebody that had coded it a few years back, but I might be mistaken.

(If these exist but are not available in the GUI, don't hesitate... I'll try them command line)

Second thing I understand is another way I'll be able to shoot a nadir.

If I take a nadir picture that is 100% parallel to the ground and at the same height as the other pictures but that I moved the tripod off from over the nodal point, then the steps you described would also enable me to do great job here. correct?

Many thanks.

nick

[Bruno Postle]

On 22 November 2014 02:57:26 GMT+00:00, Nicolas Pelletier wrote:
>
>It almost worked, but still had a little wiggle that I could not get
>rid of. But the instructions were clear anough for me to understand.
>
>If my understanding is correct, there is another set of transforms that
>are missing, perspective correction (something like here
>http://tutorial9.s3.amazonaws.com/uploads/2008/04/skew-distort-perspective.gif
>
>). Probably another one to compensate for the distance that is
>different. But I'm not an expert here.

The Hugin roll, pitch and yaw parameters are enough to do the perspective transformation you need; the XYZ viewpoint parameters compensate for the different camera position; and the new (in 2014.0) 'plane pitch' and 'plane yaw' parameters allow you to map this photo to the floor rather than a wall.

>Second thing I understand is another way I'll be able to shoot a nadir.
>
>If I take a nadir picture that is 100% parallel to the ground and at the
>same height as the other pictures but that I moved the tripod off from
>over the nodal point, then the steps you described would also enable
>me to do great job here. correct?

Yes this should work too, the angle of the camera shouldn't make any difference.


--
Bruno