morph-to-fit and Hugin with ptomorph
Bruno Postle
that helps with stitching Hugin panoramas with parallax errors. To
give an idea of what it does, here is a 'before and after' example
of a handheld panorama with quite a lot of stitching errors:
http://www.flickr.com/photos/36383814@N00/6948262808
What it does is read all the control points in an existing PTO
project, and use these to generate a 'morph' distortion for each
photo that results in all the control points lining up exactly.
The morphing itself is done with ImageMagick.
I'd describe this as experimental, it isn't going to work with HDR,
may do strange things with masks and circular cropping, and probably
doesn't work on Windows. I'd like to hear some feedback, do you
think something like this would be useful in Hugin? Is there a
better approach than forcing _all_ the control points to fit?
To try it you need the current Panotools::Script from SVN, something
like this:
svn co https://panotools.svn.sourceforge.net/svnroot/panotools/trunk/Panotools-Script
..or wait for the Panotools::Script 0.27 release (not sure when that
will be).
--
Bruno
Erik Krause
> I recently added a new tool called 'ptomorph' to Panotools::Script
> that helps with stitching Hugin panoramas with parallax errors. To
> give an idea of what it does, here is a 'before and after' example
> of a handheld panorama with quite a lot of stitching errors:
> http://www.flickr.com/photos/36383814@N00/6948262808
Very interesting. What is the difference to the old panotools Morph-to-fit?
--
Erik Krause
http://www.erik-krause.de
Bruno Postle
The panotools morph-to-fit isn't available in Hugin/nona...
Panotools also uses a not-very-nice distortion where it splits the
image up into triangles and performs an affine transformation on
each triangle separately, so you get steps between adjacent
triangles (I believe, I haven't looked at it for ten years or
something). The ImageMagick morph is a 'rubber-sheet' distortion,
so it is much smoother.
The reason morph-to-fit was never implemented in Hugin is that there
were obviously better ways of doing it.
Panotools also uses a whole separate interface, control points are
specified with 'c-lines', whereas panotools morphing uses 'C-lines'
and these have a completely different format. ptomorph just uses
the existing normal 'c-lines'.
These 'C-lines' seem a bit superfluous to me, though I suppose there
are circumstances where you might want to control morphing with a
reduced number of points - I'm not sure if there ever have been any
tools that support a reduced set like this.
--
Bruno
Rogier Wolff
On Thu, Apr 19, 2012 at 09:36:18PM +0100, Bruno Postle wrote:
> doesn't work on Windows. I'd like to hear some feedback, do you
> think something like this would be useful in Hugin? Is there a
> better approach than forcing _all_ the control points to fit?
Yes, I think this is useful.
The drawback of forcing ALL points to fit is that you force all
control points to be perfect: If there happens to be a rogue control
point in there, this will have a huge influence on the result.
Also there might be "unfixable" parallax errors. One photo might have
much more between two images.
Consider
1 a b c d 2
3 e 4
The 1, 2, 3 and 4 are control points. 1-3 are a vertical line. 2-4
too. And 1-a-b-c-d-2 are evenly spaced, as are 3-e-4 on a different
picture.
On the other hand, you seem to be getting pretty good results in
practise. I guess practise trumps theoretical drawbacks.. :-)
Roger.
--
** R.E....@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
** Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
The plan was simple, like my brother-in-law Phil. But unlike
Phil, this plan just might work.
Jim Watters
The results look great. will it be possible to implement in Nona so warping and
morphing is done is one pass?
On 2012-04-19 6:40 PM, Bruno Postle wrote:
> Panotools also uses a whole separate interface, control points are specified
> with 'c-lines', whereas panotools morphing uses 'C-lines' and these have a
> completely different format. ptomorph just uses the existing normal 'c-lines'.
I am guessing I would not be able to feed one image into ptomorph and have it
morph with 'c-line' control points? This is a good reason to use the 'C-line'.
>
> These 'C-lines' seem a bit superfluous to me, though I suppose there are
> circumstances where you might want to control morphing with a reduced number
> of points - I'm not sure if there ever have been any tools that support a
> reduced set like this.
PTGui has been able to use morph-to-fit for at least 10 years when stitching
with Panotools. It can be disabled, enabled for all control points, or enabled
for only selected control points. I believe PTAssembler does it similarly.
I also wonder how it works when three images meet at the same place. Will it
create vortexes? Rik Littlefields outlined the issue in an old email.
https://www.email-lists.org/pipermail/ptx/2004-May/001597.html
--
Jim Watters
http://photocreations.ca
kfj
> Yes, I think this is useful.
welcome indeed. Never mind the theory - at times you can use a spanner
as a hammer and the result will be just as well.
> The drawback of forcing ALL points to fit is that you force all
> control points to be perfect: If there happens to be a rogue control
> point in there, this will have a huge influence on the result.
entities. Instead I see them like a surface attribute with a low
sampling rate compared to the pixels. Upping the cp coverage is simple
- running cpfind or apsc with the right parameters (or woa :) will
create hundreds or even thousands of cps, and just throwing away the
worse half of them is easy enough, eliminating any rogue cps.
> Also there might be "unfixable" parallax errors. One photo might have
> much more between two images.
parallactic errors which have produced distortions which aren't
covered by the the panotools optical model. Proper parallactic errors
may in some cases be made invisible by morphing, but some of them just
won't yield. There are other mechanisms to deal with them which try
and identify coherent regions and shift and deform them to match in
the result, and morphing just isn't powerful enough to do that sort of
magic.
> Consider
>
> 1 a b c d 2
> 3 e 4
>
> The 1, 2, 3 and 4 are control points. 1-3 are a vertical line. 2-4
> too. And 1-a-b-c-d-2 are evenly spaced, as are 3-e-4 on a different
> picture.
round it...
> On the other hand, you seem to be getting pretty good results in
> practise. I guess practise trumps theoretical drawbacks.. :-)
Kay
Erik Krause
> Panotools also uses a not-very-nice distortion where it splits the
> image up into triangles and performs an affine transformation on
> each triangle separately, so you get steps between adjacent
> triangles (I believe, I haven't looked at it for ten years or
> something). The ImageMagick morph is a 'rubber-sheet' distortion,
> so it is much smoother.
That sounds very nice and the result is indeed superb. The not-so-nice
distortions in the panotools version was the reason why I never
investigated further years ago...
Bruno Postle
On 20 Apr 2012 15:27, "Erik Krause" <erik....@gmx.de> wrote:
>
>> The ImageMagick morph is a 'rubber-sheet' distortion,
>> so it is much smoother.
>
> That sounds very nice and the result is indeed superb. The not-so-nice distortions in the panotools version was the reason why I never investigated further years ago...
I mean to experiment with more real data and see what problems there are, hence this ptomorph script.
It currently uses a 'Shepards' distort, which forces each point to fit exactly. Though ImageMagick also has a polynomial distort where it applies a 'best fit' average. This looks promising since you can pick the order of the polynomial, and it will apply a distortion overall, just using the control points as a guide rather than requiring an exact fit.
So there is lots of potential here.
--
Bruno
Erik Krause
> It currently uses a 'Shepards' distort, which forces each point to fit
> exactly. Though ImageMagick also has a polynomial distort where it
> applies a 'best fit' average. This looks promising since you can pick
> the order of the polynomial, and it will apply a distortion overall,
> just using the control points as a guide rather than requiring an exact fit.
>
> So there is lots of potential here.
Sounds very interesting indeed! I just had a quick look on the
imagemagick distortion page. The possibilities are overwhelming. IMHO
this is an entirely new step - like enblend some years ago....
kfj
> I recently added a new tool called 'ptomorph' to Panotools::Script
> that helps with stitching Hugin panoramas with parallax errors.
> think something like this would be useful in Hugin?
I tried it today with a scan of a map. I had 8 individual images, and
because the map had been folded and I couldn't get it totally flat, I
couldn't get the stitch right. There were jumps by up to about 10
pixels, and the result wasn't really good enough. So I thought that
since I actually had flat images which were merely distorted and not
parallax errors your tool might be perfect for the purpose. It was.
The stitch from the pto your tool generated came out near perfect, I
could only see 1-2 pixel errors in areas which weren't covered by CPs,
close to the edge, on the black frame lines of the map; inside the map
image I couldn't detect a single fault.
I think 'useful in Hugin' is a gross understatement. I haven't yet
unleashed it on a 'real' panorama, but I sure will soon and report on
that as well, and it looks like the others who have tested it have had
good results, too.
Thanks
Kay
Bruno Postle
>
> I mean to experiment with more real data and see what problems
> there are, hence this ptomorph script.
>
> It currently uses a 'Shepards' distort, which forces each point to
> fit exactly. Though ImageMagick also has a polynomial distort
> where it applies a 'best fit' average. This looks promising since
> you can pick the order of the polynomial, and it will apply a
> distortion overall, just using the control points as a guide
> rather than requiring an exact fit.
polynomial order between 1 and 5:
http://www.flickr.com/photos/36383814@N00/6951181084
http://www.flickr.com/photos/36383814@N00/6951183554
http://www.flickr.com/photos/36383814@N00/7097254813
http://www.flickr.com/photos/36383814@N00/7097256599
http://www.flickr.com/photos/36383814@N00/7097258147
It looks like 1 and 2 will be useful, and these should be quite
robust in a project with 'bad' control points, unlike the default
'Shepards' distort.
3, 4 and 5 order polynomials clearly need a good spread of control
points or they get a bit weird ;-)
--
Bruno
Bruno Postle
>The results look great. will it be possible to implement in Nona so
>warping and morphing is done is one pass?
Yes, though I don't know if it should be in libpano13 or in nona, or
if the distortion should be applied to the source photos (as
implemented in ptomorph), or if it should be applied to the remapped
images.
>>Panotools also uses a whole separate interface, control points are
>>specified with 'c-lines', whereas panotools morphing uses 'C-lines'
>>and these have a completely different format. ptomorph just uses
>>the existing normal 'c-lines'.
>I am guessing I would not be able to feed one image into ptomorph and
>have it morph with 'c-line' control points? This is a good reason to
>use the 'C-line'.
Yes, but there are better ways of interactively morphing just one
photo.
>PTGui has been able to use morph-to-fit for at least 10 years when
>stitching with Panotools. It can be disabled, enabled for all control
>points, or enabled for only selected control points. I believe
>PTAssembler does it similarly.
This seems like another GUI complication that Hugin could do
without, the alternative is that you just delete control points
after optimisation and morph this pruned project.
Also the polynomial morphing may turn out to be robust enough that
there is no need to use a subset of control points. Need to do more
tests.
>I also wonder how it works when three images meet at the same place.
>Will it create vortexes? Rik Littlefields outlined the issue in an
>old email.
>https://www.email-lists.org/pipermail/ptx/2004-May/001597.html
No idea...
--
Bruno
Kornel Benko
So have I. The only thing I found, was a crash of nona due to
some "errors?" in resulting EXIF data in morphed images.
(It does not happen on every project, so I do not wonder that nobody else had seen it)
In my work-flow I had to add a call to exiftool "-all= " on each image to overcome the problem.
And yes, I am impressed too, especially when using together with "Warped Overlap Analysis".
Kornel
Cristian Marchi
~/unstable/compilati/panotools/bin/ptomorph -p 2 -o morphed.pto IMG_0479-IMG_0494.pto
I've got the following error:
Can't locate object method "InitTrafo" via package "Panotools::Script" at /home/cristian/unstable/compilati/panotools/bin/ptomorph line 27.
Thanks for any help
Kornel Benko
> I've just compiled the svn version of the Panotools and tried to test
> ptomorph by running the following command to an existing pto file of a
> Huign project with stitching errors (360x180 handheld pano made of 16
> photos):
> ~/unstable/compilati/panotools/bin/ptomorph -p 2 -o morphed.pto
> IMG_0479-IMG_0494.pto
>
> I've got the following error:
> Can't locate object method "InitTrafo" via package "Panotools::Script" at
> /home/cristian/unstable/compilati/panotools/bin/ptomorph line 27.
>
> Thanks for any help
You should install first (then the required modules should be placed on proper places)
The related files are probably in /home/cristian/unstable/compilati/panotools/Panotools-Script/lib/Panotools
Kornel
Bruno Postle
On 21 Apr 2012 09:16, "Kornel Benko" <Kornel...@berlin.de> wrote:
> Am Samstag, 21. April 2012 um 00:26:55, schrieb Cristian Marchi <cri....@gmail.com>
> > Can't locate object method "InitTrafo" via package "Panotools::Script" at
> > /home/cristian/unstable/compilati/panotools/bin/ptomorph line 27.
> You should install first (then the required modules should be placed on proper places)
Or you can use the new version of the library without installing, this is how I do development:
export PERL5LIB=/home/cristian/unstable/compilati/panotools/Panotools-Script/lib
This will work until you log out.
--
Bruno
Bruno Postle
On 20 Apr 2012 22:59, "Bruno Postle" <br...@postle.net> wrote:
> On Thu 19-Apr-2012 at 23:54 -0300, Jim Watters wrote:
>>
>> The results look great. will it be possible to implement in Nona so warping and morphing is done is one pass?
>
> Yes, though I don't know if it should be in libpano13 or in nona, or if the distortion should be applied to the source photos (as implemented in ptomorph), or if it should be applied to the remapped images.
Also, the polynomial morph actually does optimisation of distortion parameters, I always imagined this warping would be part of the normal optimisation process rather than something that only happens at stitch time.
..or maybe not, need more info.
--
Bruno