This is more or less what the cpclean -p options does.
--
Bruno
I am inclined to add them to the main repository after 2010.4.0 is declared
final. We'll have to see how to integrate neatly in the file tree. I would,
of course, prefer to have the initiator and main contributor having access to
the repo directly, but this would require you to sign up to a SourceForge
account.
> The script is python, I wrote it using Python 2.6 on a
> Kubuntu 10.10 system; you may have to grab the python argparse module
> which, I think, isn't yet in the standard library.
confirmed. And indeed I updated already the Wiki instructions [0]
> If you're happy with Bruno's approach to rather remove a 'bad'
> percentage, take that - it's a fine idea as well and it's already
> mainstream ;-)
both approaches have their merits and deserve to be available in mainstream.
Yuv
[0] <http://wiki.panotools.org/Hugin_Compiling_Ubuntu#Dependencies>
not only me
https://bugs.launchpad.net/hugin/+bug/685489
Yuv
Allow me to add that I really don't think that removing all the "far"
points is the best way to do this.
Often hugin will find control points in the grass near my
feet. Sometimes one or two near the horizon where it actualy matters.
Now the 5 or 10 control points in the grass will "twist" the images so
that the one or two good ones are some of the furthest from their
proper mapping. But in fact they are the ones that matter most.
Throwing them away (well ok, commenting them out) isn't going to help
the pano.
You know what I'd like to do? I'd like to assign weights to the
control points. For example, all control points would start with
weight 10. But then I can set the weight of those in the distance to
100 to increase their weight. Or I can set the weight of those closeby
to 1 to reduce it. (for an identical effect).
One step further we'd find different ways to set the weights
automatically. For example the similarity index might come into
play. Or I might be able, as a user, to indicate: the further up an
image the higher the weight.
When I shoot panos, I take portrait pictures. These overlap in a
narrow region with the next image. So I think I'd like about
--sieve2width 2 --sieve2height 5 --sieve1size 3
To have points near both edges let and right, and a uniform
distribution along the height of the image. If in all ten areas 3
control points are found, 30 is a bit much. On the other hand, it
doesn't always find control points. Hmm. cpfind can't know the
overlapping area, right? So I would need say a 5x5 grid to guarantee
some points in the overlapping area. Oh well. Then we might have
even more control points if things match up well....
Roger.
--
** R.E....@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
** Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement.
Does it sit on the couch all day? Is it unemployed? Please be specific!
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ
there should be a better way to enable/disable CPs selectively...
> You know what I'd like to do? I'd like to assign weights to the
> control points.
... and weighting would help in this and many other cases. For example in the
past the wish has been expressed to discern between user-generated CPs and
computer-generated CPs.
instead of commenting out CPs, you would set them to a weight of 0.
Still: I don't think CPs are the ultimate tool for the image alignment
process. They are historically grown, because humans would pin down printed
photos to each other before digital stitching came along; and when digital
stitching came along it was the human user interface to it mimicking the pre-
digital interaction and pinning down images to each other by the use of CPs.
The availability of feature detection/matching algorithms enabled the
replication of the pinning process, but there must be a more efficient way of
aligning images that does not require CPs.
Then we can always use CPs as the human-entry interface to the process, or to
try to detect orientation in a situation with less than perfect input (no
orientation data in EXIF).
Yuv
Indeed. And a user-preference can set the default weight for both of
them. Some users trust themselves, others trust the computer
better. :-)
> instead of commenting out CPs, you would set them to a weight of 0.
Right. Or at least very low.
> Still: I don't think CPs are the ultimate tool for the image
> alignment process. They are historically grown, because humans
> would pin down printed photos to each other before digital stitching
> came along; and when digital stitching came along it was the human
> user interface to it mimicking the pre- digital interaction and
> pinning down images to each other by the use of CPs.
What I think should be possible is that you optimize the cross
correlation of say a 10x10 area of the image with another image. This
is computationally expensive. This would only be feasable to do for
example for a 5 pixel radius of a point 20 pixels north of a
controlpoint.
I can't think of how to get an initial point to start working from
besides asking the user or doing the feature detection thingy....
But this is WAY beyond addition of an extra field in the
"controlpoint" structure.
first and foremost is a calibrated lens. then you only deal with image
orientation and not with imperfections in the image.
> > I can't think of how to get an initial point to start working from
> > besides asking the user or doing the feature detection thingy....
>
> or use two feature detectors. My idea would be to use the fastest
> available feature detector, pick it's best few results in a ROI and
> then verify that these locations are indeed corresponding with a heavy-
> duty, most-likely-to-succeed detector.
or an image pyramid, starting detection on the low scale version of the
image...
Yuv
done (yesterday).
> If you look at the purpose of the script in a
> different way, you can maybe appreciate it's usefulness better: it's
> an insurance that a certain number of control points will be kept for
> each image pair when throwing out less well fitted CPs.
makes sense and is indeed the weakness of keeping the top X% overall.
> At some point the domain of GUIs or
> command line parameters isn't sufficient anymore to cater for specific
> needs - this is where scripting begins.
While I agree with you that your script is useful, I disagree with the above
statement. This is not about GUI vs. CLI. It is about workflow.
Our GUI represent the workflow at a very superficial level:
load > generate cp > align > stitch
One level lower there are multiple steps involved, and problems happen
inevitably when we try to execute those multiple steps simultaneously.
For example, we use the optimizer to:
- correct for lens distortion
- align images on the panosphere
- correct for variations (intentional or unintentional) of the viewpoint
What we need are flows, and a script is indeed a very powerful way to express
and modify flows; and we need a strategy playing conditionally across flows.
In the case of CP generation, Thomas has introduced a simple but very powerful
flow based on the assumption that the images are taken in sequence.
We need a similar approach to the rest of the process; and probably even going
forth and back at the superficial level a few time, simulating what an expert
user would do.
When designing these flows (or scripts), we need to ask ourselves the
questions: what key metrics determines if the step is necessary or not? what
key metrics determine if the step succeeded or failed?
from the very very start (and completely spontaneous, so there may be errors):
1. are EXIV data available? if no: the "smartsisstant" is screwed. display a
message asking the user about their pre-processing workflow (in most cases we
can assume digital input nowadays)
2. based on the EXIV lens identification, FOV, F-Stop: do I have lens
correction parameters stored? if yes, load them. If not suggest lens
calibration (as a separate process) and recommend the user interrupts the
stitch process until the lens is calibrated.
3. is there a pattern to the exposure values? if they are all the same, this
is a single panorama. if they are regular (e.g. -2/0/+2) this is a user who
knows how to shoot HDR. if they are all over the place it can be a panorama
shoot in automatic mode (additional info in EXIV too); or it can be that there
are one or two straying extra exposures (e.g. for a door or window in the
scene). And I have not considered yet the case that multiple panoramas could
be in the same set of images.
and so on, and so on.
> Designing the UI, one walks a
> fine line between keeping it simple and comprehensible even for less
> experienced users, and nevertheless offering powerful enough features.
> A scripting interface is provided for those who want more, if not
> total, control.
this is not (yet) about designing the UI. It is one level deeper.
> My current endeavours of producing a few simple Python scripts to deal
> with the issue are just bait, really - and born from my Python
> background and my dislike of perl.
likes and dislikes aside, I agree with you that Python is the right way to go.
Python bindings into the Hugin codebase are long time on my wishlist.
> there isn't a scripting interface yet
well, theoretically you can shell-script or perl-script or Python-script all
of the CLI tools. That's not optimal, but sufficient for a start.
The 'old way' of doing things was to script something for the CLI tools and if
it would make sense wait for a coder to implement it in the main code and
speed it up. With Python binding that second part would not be necessary
since your Python script would share the same memory and variables as the code
without going through slow and expensive read/write operations.
> To facilitate experimentation along these lines,
> I'll do the following: In my Python script, I'll provide the set of
> control points as a set of objects with more properties than the ones
> which can be gleaned from the mere coordinates in the pto. I'll
> include the 'distance' and the coordinates in pano space for a start.
> Then everyone who's capable of writing a bit of code can interface
> with that and flag the points they want deleted.
Good. Next in those properties we'd need to identify hand-picked from
generated CPs, and here is the limitation of the approach: we need to
interface into code and extend that structure there too. And the PTO file, of
course.
currently a CP line in the PTO file looks like:
c n0 N1 x2319.75430140533 y1306.89102657256 X635.409481924115
Y1295.02701452379 t0
what would it take to expand it to:
c n0 N1 x2319.75430140533 y1306.89102657256 X635.409481924115
Y1295.02701452379 t0 s0 a0 w100
with
s: source (0=unknown, 1=human, 2=cpfind, etc.)
a: active (0=yes, 1=no)
w: weight (0 to 100)
the big question is: would the tools that currently parse the PTO script
stumble on this, or is it safe to add?
if it is safe to add to the specs, let's do it. Then we can go about adding
support for the s/a/w script parameters, all while making the assumption that
they may or may not be present.
...
> More room for scripting experiments. Let me just finish with a remark
> I've made over and over: if your lens is well calibrated, you really
> don't need that many CPs. Either you use a set with a great number of
> CPs to calibrate your lens, or you have a well-calibrated lens and
> only need the CPs to nudge your images in place. It shouldn't be
> necessary at all to optimize lens parameters with every pano to 'bend'
> the images to fit, and if you don't, what do you need so many CPs for?
EXACTLY! Every time I see users coming back with thousands of CPs I wonder if
this is another meaningless attempt to assemble the largest quantity of boring
pixels or if it is an exercise in global warming by CPU strain.
Theory is that three strategically placed CPs per image pair are enough when
the lens is already calibrated. I work with five CPs per image pair, and for
small projects (e.g. full sphericals with six fisheye shots) repeating the
left-click-right-click dance on the CP tab is equally fast and yield better
results than any CP generator I've tried before. A CP generator becomes
useful only when dealing with a large number of input images.
Yuv
It is used for Hugin GUI metadata that doesn't materially alter the
project.
Your extra info for control points would presumably modify the
behaviour of the optimiser, so they need to be 'normal'
control-point parameters instead.
--
Bruno
you'll be surprised to know that Hugin took a hand-crafted pto file with s/a/w
parameters without any problem, and ran the optimization as if they were not
there. So from that perspective, we can add them and then add support to them
in the relevant tools.
> #-hugin cpWeight s0 a0 w100
>
> AFAIK this is established practice for introducing additional
> ('noncritical') values into the pto without breaking compatibility
> with other pto-using programs.
Really? what other pto-using programs? and what compatibility?
I tried feeding a simple PTO file straight out of Hugin to PToptimizer, with
whom the specs are supposedly 'shared'.
It chokes... already on the p line, so compatibility is broken anyway.
Even after removing the E/R/S parameters from the p line, it chokes again on
the Eev in the i line.
And: what you call 'established practice' is just a confusing hack. Let's do
things right.
A. PUT THE HISTORY OF THE PTO FORMAT TO REST
It was once derived from the original panotools syntax specified by Helmut
Dersch. In that sense it shares some DNA and a common ancestor with the
syntax of other PT based tools. But they have all evolved in different
directions and it can be safely assumed that:
1. the PTO format is specified by the Hugin project and for Hugin.
2. it has evolved to fit the growing needs of Hugin (e.g. adding photometric
variables, translation variables, etc.)
3. we are on our own and can keep it growing and try to steer the growth
toward a sensible solution
B. SPECIFYING THE NEW NEEDS
1. there is a case/request to activate/deactivate CPs
2. there is a case to weight CPs
3. there is a case to discern the source of CPs (human or computer generated)
C. EXTENDED SPECIFICATION OF THE PTO FORMAT TO MATCH THESE NEEDS
New variables on the 'c' lines:
- s: source (0=unknown, 1=human, 2=cpfind, etc.)
- a: active (0=yes, 1=no)
- w: weight (0 to 100)
Note that the these params are optionals and they have sensible default for
backward compatibility if a parser is properly coded, maybe with the exception
of weight - should it be in a range from 1 to 100 to avoid that all CPs have
weight of 0?
If there are no objections I will updated the specs both in Hugin and in
Libpano to make developers aware of things to come.
Ideally
http://hugin.hg.sourceforge.net/hgweb/hugin/hugin/file/default/doc/nona.txt
http://panotools.svn.sourceforge.net/viewvc/panotools/trunk/libpano/doc/stitch.txt
http://panotools.svn.sourceforge.net/viewvc/panotools/trunk/libpano/doc/Optimize.txt
should share a single, synchronized and coordinated spec.
D. PARSING A PTO FILE (or a PT dialect script)
I don't know how many parsers there are out there that do parse pto files. I
hope for them that they are designed as robustly as your Python parser or
Hugin's own parser. A good parser should be able to ignore noise such as
extra parameters that it does not need; or have a 'strict' syntax option to
throw warnings with a switch to disable those warnings. Feature request for
Panotools.
https://bugs.launchpad.net/panotools/+bug/696673
I don't know if Bruno's Perl tools are affected?
E. MAKING SENSE OF THE NEW PARAMETERS
It may or may not make sense for different tools to implement the new
parameters. Once they are specified and the parsers are robust enough, there
is no hurry. You can start playing around with these parameters in your
Python scripts; I can add the facility into the Hugin GUI to discern which
points are manually entered by the user; and so on. Eventually they will make
sense in the big scheme of things.
Yuv
But do you find this fulfilling work? Setting aside the fact that
you've become convinced that the algorithms of today generate worse
results than you, Wouldn't you prefer to have the computer do this
boring work for you?
You apparently do the "pano-with-six-fisheye-images" dance regularly.
Calibrating the lens and then just using that is much easier.
I have a zoomlens.
Focal lenght: 18-135mm
F-stop: f/2.0- f/32
focus: 1.5m - inf
So worst case I have to have a 3D matrix of lens calibration settings
for two analog and one digital parameter. (one of which doesn't go
into the exif data). I think that calibrating the lens on the current
project is a reasonable compromise.
Roger.
--
** R.E....@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
** Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Hey, do you see a need for an active-weight-zero control point?
Do you see a need for an inactive weight-not-zero control point?
If you set the weight to zero for control points that are not active,
the optimizer just has to add "cp->weight * " to the line that says
toterr += this_distance;
How about: negative weights are "inactive"?
Then my suggestion already doesn't work. Maybe then it would even be
better from a software engineering standpoint to just do it your way.
Why do you thing weight is an integer?
When you start algorithmically assigning weights, it would make sense
to allow fractional weights.
What about setting the weight to "1" by default?
Then you can lower the weight by usign values 0-0.99 and increase it
by using values above 1. So if you manually tag a controlpoint as
being very valid, you can increase it outside the normal range.
There are two cases where above-default weights are useful. In the
first case, you simply have a very well defined control point. Say a
perfectly black 90 degree angle on a white background. That will match
up very well with another photo. So you have more confidence in this
point than the noisy controlpoints that cpfind found in a perfectly
blue sky. A weight of say 2 or 3 may be appropriate here.
In the second case, you have some feature that ends up looking
crooked(sp?) even if just a few pixels off. So you want that control
point to carry more weight than the others. It should be able to pivot
around that point, but land the parameters such that it aligns almost
perfectly. A weight of 100 might be appropriate here (with my
default-is-1.0 scheme).
There is no need to update the docs until the feature exists, the
implementation often determines the details anyway. e.g. weights
are unlikely to be integers as already noted.
Note that an existing hack to simulate weighting is to simply
duplicate control points, it works ok.
>I don't know how many parsers there are out there that do parse pto files. I
>hope for them that they are designed as robustly as your Python parser or
>Hugin's own parser. A good parser should be able to ignore noise such as
>extra parameters that it does not need; or have a 'strict' syntax option to
>throw warnings with a switch to disable those warnings. Feature request for
>Panotools.
>
>https://bugs.launchpad.net/panotools/+bug/696673
>
>I don't know if Bruno's Perl tools are affected?
Panotools::Script is ok, it will just ignore any unknown parameters
so long as they are implemented using the existing syntax.
I don't think there is a problem here, if extra data needs to be
added to the Hugin .pto format then it can be added, the other tools
can catch up. See the addition of photometric parameters and masks
to see how it can be done painlessly.
--
Bruno
The wiki pages can be just a general description. Usually the
definitive guide to any of these tools is given on the command-line,
e.g:
$ cpclean -h
cpclean: remove wrong control points by statistic method
cpclean version 2010.4.0.a26eaba2eda3
Usage: cpclean [options] input.pto
CPClean uses statistical methods to remove wrong control points
Step 1 optimises all images pairs, calculates for each pair mean
and standard deviation and removes all control points
with error bigger than mean+n*sigma
Step 2 optimises the whole panorama, calculates mean and standard deviation
for all control points and removes all control points with error
bigger than mean+n*sigma
Options:
-o file.pto Output Hugin PTO file. Default: '<filename>_clean.pto'.
-n num distance factor for checking (default: 2)
-p do only pairwise optimisation (skip step 2)
-w do optimise whole panorama (skip step 1)
-h shows help
This is definitive because this is the bit written by the
programmer. If there is more relevant but technical info then it
could also be in a man page or on the wiki (or both sometimes).
--
Bruno
No it should only remove some of the points with an error-distance
greater than the average, you will always have some points left.
--
Bruno
You could use PToptimizer which writes all this statistical
information about control point errors to the project file. Though
you have to clean a Hugin project file to make it acceptable to the
panotools parser in PToptimizer.
There are two tools by Iouri Ivliev in Panotools::Script to help
with this, unfortunately I have never had a chance to play with
them (it would be really useful if somebody could evaluate all this
stuff and report their experiences):
ptsed is a command-line editor for .pto projects, amongst other
things it will clean a .pto file so it can be read by
panotools/PTmender/PToptimizer.
ptscluster is another statistical filter for control points (like
ptoclean and cpclean), it uses the information in the PToptimizer
output and tries to leave a spread of control points rather than
just deleting the 'worst'.
>I could make a temporary pano with equirect output and 360X180 fov,
>then I could calculate the distance on the sphere from the polar
>coordinates. But of course this is clumsy; maybe you can think of a
>more elegant way?
You could reimplement a panorama transformation and measure the
angular distance directly, this isn't so difficult (there is one of
these in Panotools::Script used by ptoclean).
--
Bruno
>I've run into another problem with finding the CP errors here - if I
>call the optimizer on the Panorama object, the errors are filled in,
>but now I wonder if there isn't a way to just ask for the calculation
>of the distances without actually optimizing the CPs?
Sorry, I'm not familiar with this code. Hugin updates all the
control point distances after optimisation and on initially loading
a project (so it isn't necessary to run the optimiser to get this
info).
--
Bruno
that would be really nice! I once tried with boost::python, but I hit
some strange roadblocks, which prevented me from creating a useful
interface...
> I've run into another problem with finding the CP errors here - if I
> call the optimizer on the Panorama object, the errors are filled in,
> but now I wonder if there isn't a way to just ask for the calculation
> of the distances without actually optimizing the CPs? After all, if
> the transformations for the images are known, the distances should be
> calculable without optimization. cpclean also calls the optimizer,
> probably because it had the same problem?
I don't think there is a ready made function for computing the distances
inside the hugin source. The distances can also be computed using the
PTools::Transform objects (transform both points from input image
coordinates into panorama space equirect and compute the great circle
distance (haversine formula) there. Note that the distances depend on
the output image projection and size.
ciao
Pablo
this is the wrong way of doing things. first write the specs, then
implemented them. as an extra bonus, well written specs are the documentation
that often lacks^H^H^H^H^Hfollows implementation.
> Note that an existing hack to simulate weighting is to simply
> duplicate control points, it works ok.
Hack. Hugin also has a function to remove duplicate CPs. I already see the
unpredictable results from improperly mixing the two.
> Panotools::Script is ok, it will just ignore any unknown parameters
> so long as they are implemented using the existing syntax.
good to know.
> I don't think there is a problem here, if extra data needs to be
> added to the Hugin .pto format then it can be added, the other tools
> can catch up. See the addition of photometric parameters and masks
> to see how it can be done painlessly.
thanks. I went by the book [0], posted [1] to panotools-devel and if there is
no objection I will unify the specs. Once the specs are unified there will be
work to do to bring the parser(s) up to date; and to actually implement the
new parameters. I know I will move forward at a snail pace, so if somebody is
faster than me, be my guest and take me over. Otherwise, I will appreciate
the mentorship of more experienced devs as I am making my mistakes while
trying to implement this.
Yuv
[0]
http://panotools.svn.sourceforge.net/viewvc/panotools/trunk/libpano/doc/developmentPolicy.txt?revision=1060&view=markup
[1] http://article.gmane.org/gmane.comp.graphics.panotools.devel/1761
of course not, but I'll do what is faster.
> Setting aside the fact that
> you've become convinced that the algorithms of today generate worse
> results than you, Wouldn't you prefer to have the computer do this
> boring work for you?
I am not convinced that today's algorithms generate worse results than me. I
had timed them to be slower back when I was doing this and I am convinced that
there is room for improvement.
> You apparently do the "pano-with-six-fisheye-images" dance regularly.
not at all. I can count on a hand the number of full sphericals I have shot
in the past two years. Maybe two hands. I made more partial stitches at
medium to long focal distance, and cpfind is really great on them. See
attached example.
> Calibrating the lens and then just using that is much easier.
>
> I have a zoomlens.
>
> Focal lenght: 18-135mm
> F-stop: f/2.0- f/32
> focus: 1.5m - inf
>
> So worst case I have to have a 3D matrix of lens calibration settings
> for two analog and one digital parameter. (one of which doesn't go
> into the exif data). I think that calibrating the lens on the current
> project is a reasonable compromise.
sure, calibrating the lens on the current project is a reasonable compromise -
and I even show it in my last tutorial [0].
the 3D matrix can be approximated / interpolated with multiple points in this
3D space, and if Hugin runs a calibration on every project you do, and store
the result in that 3D space, it will become increasingly better at processing
your panos.
Yuv
>
>
> Roger.
[0] http://panospace.wordpress.com/2010/09/19/linear-panoramas-mosaic-
tutorial/
Not really, weight is relative.
> Do you see a need for an inactive weight-not-zero control point?
Yes - when I want to deactivate it without losing the weight information (as
you do presently implicitly by commenting the line rather than deleting it).
The active/not active toggles are like the visibility toggles in the fast
preview.
> Why do you thing weight is an integer?
actually I did not, but I realize that my specs were ambiguous. Should have
read w100.0 to remove that ambiguity.
> What about setting the weight to "1" by default?
though of it. the problem is that legacy projects will come with no s/a/w
parameters assigned. setting all defaults to 0 makes sure that there are no
unexpected behaviors when parsing old PTO files. But we can expect parsers to
be written to specification, right? so, yes 1 makes it for a sensible default.
Yuv
thanks for this, Tom. I'll update the work-in-progress specs accordingly by
transforming the active flag from a one bit to a two bits parameter:
- s: source (0=unknown, 1=human, 2=cpfind, etc.) - default 0
- a: active (0=yes, 1=not for optimization 2=not for morph-to-fit 3=neither
for optimization nor for morph to fit) - default 0
- w: weight (0.0 to 100.0) - default 1.0
let me know if this looks good enough for you or if you have better ideas.
Yuv
I still say: Don't mention an upper limit.
Weight: floating point, default 1.0. We currently don't expect
negative weights to be useful, but who knows.
You can add a remark: currently Yuval thinks values between about 0
and 100 make sense and outside are not really useful, but who knows
what may become useful in the future.
Do not write unneccesary things into the spec. From your spec someone
might implement a parser that reads 1 or two digits, a period followed
by one more digit. Or special case 100.0.
Now weight 1 is invalid. Should be 1.0 for that parser. We can't
specify 1.05. 0.1 and 0.2 are a factor of two apart, and whereas we
have a 1/1000 resolution in specifying high weights we only have 0/10
resolution of specifying lower weights.
OK
> You can add a remark: currently Yuval thinks values between about 0
> and 100 make sense and outside are not really useful, but who knows
> what may become useful in the future.
not necessary.
> Do not write unneccesary things into the spec.
OK
Yuv