Solving the low-frequency alignment problem using image order and overlap differencing

[Alex Stanhope]

PTGui is fantastic.  It's ability to identify control points is brilliant, especially when there's lots of high frequencies in the image, like sharp edges or transitions between low luminance to high luminance pixels.  It's slightly weaker at identifying control points for low-frequency images, like smooth transitions or graduated colour changes.  The thing is that human beings are pretty poor at that too.  When manually adding control points into the overlapping space between two images, I look for identifiable markers, but they're very hard to find in low-frequency images.  It would be fantastic if PTGui or a plugin could deal with this scenario, which I imagine is quite common in:

* out of focus areas

* skies

I've got two ideas for improving the accuracy of automatic control points for this tricky group of images:

1. Image order and overlap pattern

I shoot my images using a Nodal Ninja, so they're very accurately spaced.  Using the order of the image, PTGui could derive roughly where in the pano space it should go.  This isn't going to be pixel accurate, but it should be within a degree.

2. Overlap differencing

All images have a fairly common overlap.  Again it's not pixel accurate, but I shoot about 25% overlap between the images and that's accurate to about a degree.  By looking at the overlap between all the previous images, PTGui could expect the same kind of overlap with the new image.  After it's found that overlap area, it could probably find a good match for alignment based on a log-polar transform of the image difference (subtraction).  The point where the two gradients overlapped should produce a good 'spike'.

Does anyone have any recommendations for handling these areas without code changes?  I know PTGui supports plugins, but I don't think the API is public so I'm not sure how to evaluate writing one myself.  Is there any appetite to improve this aspect of PTGui's already very good game?

[Geoff - Spherical Visions]

If you look at Projects|Align grid you will see that this allows the images to be aligned, however it doesn't prevent CP's being created beyond adjacent images at present.

[PTGui Support]

Hi Alex,

Thanks for the feedback. Improving the control point generator so that it takes into account any predefined alignment of the images is on the wish list, as well as improving its accuracy for out of focus details.

I'm sorry but there's no generic plugin interface in PTGui; support for the current plugins (autopano, enblend) was custom written.

Joost

> --
> You received this message because you are subscribed to the Google Groups "PTGui" group.
> To post to this group, send email to pt...@googlegroups.com
> To unsubscribe from this group, send email to ptgui+un...@googlegroups.com
> Please do not add attachments to your posts; instead upload your files at a file sharing site (for example http://ge.tt/ ) and include a link in your message.
> For more options, visit this group at http://groups.google.com/group/ptgui

[Tom Sharpless]

Right On, Alex!

I've been grumbling for years about PTGui's  blythe (or is it foolhardy?) disregard of the shooting pattern.  When known, it is the strongest guide to good CP placement, and it is known in 99% of the image sets PTGui will be asked to stitch.  PTGui inherited this blind spot froim PanoTools, and shares it with the other PT family stitchers.

The present 'grid layout' tools miss the main point, which is that a shooting pattern is an arrangement on the sphere, not a flat map.  Trying to represent it as an arrangement of rows and columns guarantees that it will fail for most spherical panos.   But we can describe our spherical shooting patterns rather accurately with angles.  I'm sure you know what I mean when I say "6-around up 55 degrees, 8-around level, 6-around down 55 degrees, nadir".  I am still hoping to see the day when PTGui will understand that too.

-- Tom

[Joost Nieuwenhuijse - PTGui Support]

On Nov 10, 2012, at 12:33 AM, Tom Sharpless <tksha...@gmail.com> wrote:

> Right On, Alex!
>
> I've been grumbling for years about PTGui's blythe (or is it foolhardy?) disregard of the shooting pattern. When known, it is the strongest guide to good CP placement, and it is known in 99% of the image sets PTGui will be asked to stitch. PTGui inherited this blind spot froim PanoTools, and shares it with the other PT family stitchers.

I would disagree about the 99% statement, but indeed this would be extremely useful. It's high on my list.

> The present 'grid layout' tools miss the main point, which is that a shooting pattern is an arrangement on the sphere, not a flat map. Trying to represent it as an arrangement of rows and columns guarantees that it will fail for most spherical panos. But we can describe our spherical shooting patterns rather accurately with angles. I'm sure you know what I mean when I say "6-around up 55 degrees, 8-around level, 6-around down 55 degrees, nadir". I am still hoping to see the day when PTGui will understand that too.

It's nearly impossible to create a GUI that would accommodate each and everyones shooting patterns. In your above example it would need to deal not only with the number of images in each row but also the starting point (yaw offset) for each row. This would lead to a very complex user interface. Therefore Align to Grid is deliberately limited to a basic row*column grid. But many (most?) non-spherical gigapans are actually shot that way.

The more complex shooting patterns can still be entered in Align to Grid by applying multiple times for each row. See "Non uniform grids" in

http://www.ptgui.com/examples/creating_gigapixel_panoramas_with_a_robotic_panohead.html

Or you can enter the angles manually in the Image Parameters tab. You only have to do this once if you save to a template.

Joost

[Erik Krause]

Am 10.11.2012 12:02, schrieb Joost Nieuwenhuijse - PTGui Support:
> Or you can enter the angles manually in the Image Parameters tab. You
> only have to do this once if you save to a template.

Actually PTGui has the feature Tom wants already. It's the "Fill yaw"
feature: On Image Parameters tab select the images you want to set yaw
in the Pitch column, Press "Fill yaw..." and enter the correct values.
PTGui defaults to the full round, so if you selected 4 images it will
use an increment of 90�. Now enter the desired value for Pitch and press
enter. Done.

--
Erik Krause
http://www.erik-krause.de

[Jim Watters]

On 2012-11-10 7:48 AM, Erik Krause wrote:
> Actually PTGui has the feature Tom wants already. It's the "Fill yaw" feature:
> On Image Parameters tab select the images you want to set yaw in the Pitch
> column, Press "Fill yaw..." and enter the correct values. PTGui defaults to

> the full round, so if you selected 4 images it will use an increment of 90�.

> Now enter the desired value for Pitch and press enter. Done.

"Fill Yaw" helps but it is not perfect.

I don't shoot with an automated pan head. My images are only roughly equally
spaced apart.
The sky may not cover the entire area. There may be scattered images that do
have features that anchor to the ground with control points. The images between
these anchors need to be evenly spaced between them.

Currently I force all images to have the same roll. Force each row of images to
have the same pitch. Use a calculator to calculate the incremental value needed
to space the images between the first and last. Then select the yaw value of the
fist image in the row that in an anchor. Use Fill Yaw and past in the fist yaw
value to keep it from changing. use the calculated increment value. If done
right the last image on the row should not change position. But because it is
already anchored with control points it can be optimized back to the correct
position. I have not tried one of these panoramas with version 9.1.4/5

My feature requests, that has already been recolonized by Joost, but not
implemented, is:

1/
Instead of only Fill Yaw, have a fill any parameter.
Add an option to have start and end values instead of a increment value. These
two values should be auto filled by current values. This will allow images in
the sky to be adjusted even if the pano is not level. Because of the way Yaw,
Pitch, and Roll is laid out on the Image Parameters tab all three could be
adjusted in one go.

2/
Allow to enter values for Yaw, pitch, and roll as a reference not just absolute
values. Like the original Panotools scripts. Put "=1" right in the Yaw value for
image 2. To force them to have the same value.

This is what the Link roll and Link pitch was suppose to accomplish on optimizer
tab. But that was before I started to shoot multi row panoramas. Globally
linking all images is only useful for single row panoramas. It was meant to be
used first pass on the optimization to help remove really bad placed control
points and roughly optimize the yaw values. This was before robot heads and
gigapixel panoramas. When images have control control points joining them
together, it is best to ensure linking Roll and Pitch is not checked before the
final optimization even if it is a single row.

Being able to assign all images the same value in the Image Parameters tab by
clicking on the Column header, say Roll, then just type a value and press enter
has made the Link Roll and Link Pitch obsolete.

Does anyone actually use the Link roll, Link pitch feature on the Optimizer tab
anymore?

3/
The lens model for PTGui could use an update too.

I have shot some multi-row panoramas with two or three zoom settings. Shoot most
of the panorama at a moderate zoom level. Shoot the sky at the widest zoom and
then detail areas at a narrower setting. For best alignment it is best to
optimize all images at the same time.

Assigning reference values for lenses could help this. The current lens model
for PTGui allows for one global lens and then all the rest have individual
settings. These individual settings can be set a predefined lens parameters in
the Image Parameters tab using Lens db... at the bottom, but that does not allow
to further optimize these values as a group.

This would also allow optimizing two or more different lens settings. Selecting
to optimize any one of group of images that share the same value because they
were set to be the same using a reference value would optimize that value for
the group. Or just remove or grey out any value that is set by reference. Error
checks are needed to ensure no circular references are made.

Additionally when an image is assigned a lens from the Lens db remember that. On
the optimizer tab under Optimize globally: Add a new column for each lens in the
project.
On the "Lens Settings" tab instead of checking off individual parameters for
Lens, Shift, Shear, and Crop, have a drop down list that would would be set to
Global by default but have the choice of lenses defined in Lens database, and
New Lens #1, The first image would be assigned the values for that lens.
Subsequent images would be set to the same by using reference value.

--
Jim Watters
http://photocreations.ca

[Geoff - Spherical Visions]

Jim, Another route is to record the Azimuth, Pitch and Roll of each shot using a GPS tagger with with 3-axis compass facilities, this then allows you to position all the shots no matter what order you take them in!  I use the technique whenever I shoot more than a single row.  Joost also has the requirement to log the relevant Exif data on his features list, in the meantime there is a workflow which gets a round the lack here http://www.sphericalvisions.com/cameraattitude/cameraattitude.htm