mosaic, confusion and ignorance - some experiments

[paul womack]

I thought I'd step back from my large map project,
and grope my way towards an understanding
of the underlying model and capabilities of
mosaic mode, on the simplest possible example.

I suspect all I've done is better define my ignorance.

In the hope that I may help others, and
that other can help me I present my experiment.

I took two shots of a pretty nigh 2D target;
it's a financial advert (yawn) printed on stiff paper,
and delivered to me unfolded.
The shots were deliberately taken at very different angles,
to see wether this could be resolved.

I carefully (and manually) set a large number
of well distributed control points between the 2 images.

I then attempted to see if one image
could be arbitrarily fitted to another "base" image
using the flexibility of mosaic mode.

So I left image 0 fixed (all parameters zero except roll
of 90 to put the image right way up and keep me sane).
I optimised (in various sequences) YPR,XYX.

All gave me dreadful CP errors (avg 19, max 74).
(attached as base.pto)

So - if I can't match one onto a base "arbitrarily", is there
a place where they will "mutually" fit?

I allowed the optimiser to change YPR on image 0.
Perfect (in its way). With this extra freedom,
the optimiser found a really good solution (avg 0.5, max 1.2)
(attached as YPR.pto)

Of course, this isn't what I finally wanted, since
the image isn't in the right orientation.

So I tried a different approach, again fixing the base image,
but allowing the optimiser "plane Y" and "plane P".

Again, optimisation was nigh perfect avg 0.58, max 1.2),
so this is a way to transform one image to fit another.
(attached as YPR_YP.pto)
Progress!

With the power of plane Y and P now at my command,
I was confident that I could transform the base image to
be upright and square, and that the second image would be able
to follow.

So I set vert and horiz CPs on the base image, allowed the optimise
to optimise YPR on the base image, and got, again, Avg 0.63, max 1.9.

And plane Y P went close to zero... !!
(attached as SQ.pto)

Indeed, knowing that a good result is possible, I
started afresh, and simply allowed the optimiser
YPR on base, YPRXYZ on the other, and "bang".

So it appears that squareness is actually *needed* to avoid
having non zero plane Y/P.

Which implies that leaving squareness as an option extra at the end
of optimising one of my maps sets may not be wise.

(sidebar:
If nona is used to make a multi-layer tiff
nona -m TIFF_multilayer -o multi_layer eg.pto

and Gimp layers used as a sort of blink-comparator, it can be seen
that even this target is not truly 2D, and shows some "flexing" as you toggle
layers on and off.
)

BugBear (in a quagmire of choices)

[Roger Broadie]

I agree that your test, where you have photographed the same subject –
a properly flat, rectangular sheet – from two very different angles and
tried to align one with the other, does indeed reveal a case where the
optimiser fails.

In base.pto you sought to make the alignment by leaving image 0, the
base image unchanged and optimising y, p, r, X, Y and Z for the image 1,
the image that is to be aligned with the other. It didn't work. I agree
that it should have, but I tried introducing the parameters in various
different orders and did no better than you. The errors in pixels
(mean/max) remained around 49/188 (after calculating the optimum size,
as, for consistency, I have done for the other examples). The X, Y
and Z parameters seem to have worked well, but the angular alignment was
still well adrift. Maybe the optimiser couldn't cope here with the very
large difference in the alignments of the two images.

On the other hand, simply adding the Plane Yaw and Pitch parameters into
the optimisation of image 1 yields a good result, with error figures at
optimum size of 1.5/3.1 - for your SQ.pto, the equivalent figures at
optimum size are 1.5/5.8. And the alignment of the superimposed image
now matches that of the underlying one. The optimiser has recovered its
mojo. Obviously this is not what the Plane parameters are designed for
and I am not at all clear how the angle values are distributed between
the normal and Plane yaw and pitch parameters. It may simply be that,
with more parameters to play with, the optimiser does a better job.

That may also be why your example YPR.pto, which adds optimisation of
those parameters for the base image 0 also works. It yields error
figures at optimum size of 1.5/3.2. Of course the whole image is
rotated, but that is not surprising, since the roll is allowed to vary
for both images. It can be prevented by leaving the roll of image 0 at
90 deg and non-optimisable. The error figure is hardly distinguishable,
but, remarkably, the yaw and pitch values that the optimiser settles on
are such that the image of the sheet is now nearly rectangular, as the
attached screen-shot shows. It certainly seems that the optimiser works
best when the area of interest is, or can be made, at least roughly
rectangular. That probably explains why it helps to add horizontal and
vertical control points at an early stage, as I tend to do anyway when
stitching maps, .

One warning I should give is that while using Plane Pitch and Yaw
parameters can improve the error figures and eliminate some stitching
flaws it can also, I have found, introduce some distortion of the
overall image. It's probably only worth doing if there are problems and
it may be possible to restrict it to individual images.

Roger Broadie