questions about cpfind
kfj
I have some questions about cpfind:
- if my images have been taken with a stereographic lens (I use the
Samyang fisheye), will cpfind notice the fact (it's in the input pto)
and omit remapping the image, even though the hfov is above the
remapping threshold?
- cpfind only ever matches image pairs. Other CPGs, like autopano-sift-
c, seem to take the feature points of all images together and run one
matching step on the lot. If I understand the process correctly, the
pairwise approach is more likely to join images which aren't so well-
matched, but with lots of images and without constraints like linear
matching the pairwise approach is very time-consuming. Is there a
chance that the 'match all feature points in one go' approach could be
implemented in cpfind?
- other CPGs have a 'maxmatches' feature that limits the number of CPs
returned. In cpfind the only way to influence the number of CPs
gnerated seems to be via the sieve parameters. Wouldn't it be possible
to employ a mechanism similar to cpclean to the end result to kick out
the 'worst' few points if there are more than the desired number?
- the documentation is thin when it comes to explaining the sieving
mechanism. I think 'buckets' and 'sieves' are technical terms straight
out of the matching algorithms, but aren't so easy to understand for
someone who merely wants to use cpfind. I'd like to know more about
the 'buckets' involved. I think what happens is that the image is
subdivided into sections and each section is treated so that it
doesn't get too many CPs, or has at least some even if they aren't so
good, resulting in a more even spread than if the whole image was
looked at. If that is so, wouldn't it be a good idea to look at the
aspect ratio of the images to calculate appropriate bucket numbers?
This may seem of little use for the standard case of the images being
3:2 or 2:3, but I often have images of greatly varying aspect ratio
and a mechanism where I can just say I want, say, roughly 24 buckets
spread so that they all cover as-near-as-possible square areas would
be helpful. This question is particularly relevant for use of cpfind
with woa, where I intend to make cpfind the default CPG, now that the
--fullscale issue is settled.
- for the finetuning parameters, the terminology is straight out of
the statistical methods empoyed. I have a rough understanding of what
the ransac algorithm does and what a kdtree is, but I suppose I'm not
average in that respect. Meaning that these parameters might be more
useful if some explanation along the lines of 'you may want to raise
this number to achieve that' was given - or there was an attempt made
at making these parameters more palatable by introducing flags which
are more concerned with their effect than their implementation.
- in woa, I use pre-warped images and feed them to the CPG. The iamge
pairs cpfind gets to see are therefore quite similar geometrically.
What ransac mode should I use? So far I have set hfov to 30 and
projection to rectilinear for all my prewarped images - after all I
have already warped them to be geometrically compatible and I dont
want them to be reprojected in any way - so the mode chosen would be
'hom'. Would I maybe be better off with 'rpy'?
Kay
Pablo d'Angelo
> Hi all!
>
> I have some questions about cpfind:
>
> - if my images have been taken with a stereographic lens (I use the
> Samyang fisheye), will cpfind notice the fact (it's in the input pto)
> and omit remapping the image, even though the hfov is above the
> remapping threshold?
yes.
> - cpfind only ever matches image pairs. Other CPGs, like autopano-sift-
> c, seem to take the feature points of all images together and run one
> matching step on the lot. If I understand the process correctly, the
> pairwise approach is more likely to join images which aren't so well-
> matched, but with lots of images and without constraints like linear
> matching the pairwise approach is very time-consuming. Is there a
> chance that the 'match all feature points in one go' approach could be
> implemented in cpfind?
I have implemented a simple version of that, but its not yet commited,
as the results are worse than that of the pairwise matching. I'll look
into commiting that change.
> - other CPGs have a 'maxmatches' feature that limits the number of CPs
> returned. In cpfind the only way to influence the number of CPs
> gnerated seems to be via the sieve parameters. Wouldn't it be possible
> to employ a mechanism similar to cpclean to the end result to kick out
> the 'worst' few points if there are more than the desired number?
Yes, that might be possible, but one the simple approach will loose the
well distributed characteristic. Maybe some more advanced heuristrics
might be found for keeping that, but I haven't thought about that yet.
> - the documentation is thin when it comes to explaining the sieving
> mechanism. I think 'buckets' and 'sieves' are technical terms straight
> out of the matching algorithms, but aren't so easy to understand for
> someone who merely wants to use cpfind. I'd like to know more about
> the 'buckets' involved. I think what happens is that the image is
> subdivided into sections and each section is treated so that it
> doesn't get too many CPs, or has at least some even if they aren't so
> good, resulting in a more even spread than if the whole image was
> looked at.
Its something similar to that. There are two thinning filters:
Sieve 1:
filters the interest points (based on location and interest point
strength). This minimizes the computation expense in the descriptor
computation and matching steps. Uses sieve1width by sieve1height
buckets/sections over the whole image. Only sieve1size interest points
are kept in each bucket.
Sieve 2:
thins the successfully matched points. The rectangular region covered by
the matches is divided into sieve2width and sieve2height sections. Note
that not the full image area is used here, but the area where matches
have been found.
Each bucket gets --sieve2size matches (if enough are available)
> If that is so, wouldn't it be a good idea to look at the
> aspect ratio of the images to calculate appropriate bucket numbers?
> This may seem of little use for the standard case of the images being
> 3:2 or 2:3, but I often have images of greatly varying aspect ratio
> and a mechanism where I can just say I want, say, roughly 24 buckets
> spread so that they all cover as-near-as-possible square areas would
> be helpful. This question is particularly relevant for use of cpfind
> with woa, where I intend to make cpfind the default CPG, now that the
> --fullscale issue is settled.
That might be a nice feature, so one specify the number of buckets for
the larger dimension and computing the number of buckets for the smaller
dimension based on the aspect ratio.
> - in woa, I use pre-warped images and feed them to the CPG. The iamge
> pairs cpfind gets to see are therefore quite similar geometrically.
> What ransac mode should I use? So far I have set hfov to 30 and
> projection to rectilinear for all my prewarped images - after all I
> have already warped them to be geometrically compatible and I dont
> want them to be reprojected in any way - so the mode chosen would be
> 'hom'. Would I maybe be better off with 'rpy'?
Both should work. If you just want to allow shift and rotations 'rpy',
together with a small fov (<10) would be best. the homography will also
allow shearing and mirroring, which is probably not needed for your case.
cpfind (and SIFT and related algorithms in general) are however not
ideal for the refinement of matching points, as it expects precisely
repeatable interest point detection. A direct, intensity based matching
is preferrable for that case, for example Least square matching.
http://www.photogrammetry.ethz.ch/general/persons/AG_pub/ALSM_AWGruen.pdf
This typically yields points with an accuracy of 1/10th of a pixel.
The drawback is that is requires good initial approximations (~2 pixel
accurate) of the relative orientation of the images. It would be a good
refinement technique for inclusion into cpfind and the hugin control
points tab.
ciao
Pablo
kfj
> Am 03.06.2011 08:29, schrieb kfj:
> > I have some questions about cpfind:
> > ... Is there a
> > implemented in cpfind?
>
> I have implemented a simple version of that, but its not yet commited,
> as the results are worse than that of the pairwise matching. I'll look
> into commiting that change.
substantial speedup. Of course you're the one who's looked into the
matter, but I guess that's what it may amount to, and if taht is so I
think the match-all-in-one-go approach might make the better default -
if the (qualitatively superior) pairwise approach is deemed necessary,
it can still be there as an option.
> > - other CPGs have a 'maxmatches' feature that limits the number of CPs
> Yes, that might be possible, but one the simple approach will loose the
> well distributed characteristic. Maybe some more advanced heuristrics
> might be found for keeping that, but I haven't thought about that yet.
You might even be able to simply call it to clean up the points, just
like you have a facility to perform a celeste run on the data. And so
add a --cplean parameter.
> > - the documentation is thin when it comes to explaining the sieving
> > mechanism.
thanks for the clarification. Your concise explanation might make a
nice addition to the man page.
> > What ransac mode should I use?
>
> Both should work. If you just want to allow shift and rotations 'rpy',
> together with a small fov (<10) would be best. the homography will also
> allow shearing and mirroring, which is probably not needed for your case.
that I assume that locally the effect of a scene being seen through
different lenses is similar to an affine transformation, even though
overall it's more complex. What differences remain after I've woa-ed
the images usually looks like they are sheared a bit, (especially if
they come from very different lenses, like one from the fisheye and
one from a rectilinear lens) - so I'll stick with the homography. Is
the 'rpy' any faster, though?
> cpfind (and SIFT and related algorithms in general) are however not
> ideal for the refinement of matching points, as it expects precisely
> repeatable interest point detection. A direct, intensity based matching
> The drawback is that is requires good initial approximations (~2 pixel
> accurate) of the relative orientation of the images. It would be a good
> refinement technique for inclusion into cpfind and the hugin control
> points tab.
approach. Since I use woa mainly to match images from different
lenses, the woa-ed images often difffer a lot in sharpness, so the
implied stages of low-pass-filtering in SIFT and SURF help deal with
this matter.
As you mention the point, I think the 'finetuning' of CPs in hugin
needs a work-over - my feeling is that especially with images from
different lenses, if I manually set a CP and then 'finetune' it, the
result is much worse than my manual placement. I suppose it's old code
and based on cross-correlation, which may fail with widely differing
distortion in both images. The placement of the CPs generated with a
CPG usually leave little to be desired - they're either spot-on or
just plain wrong. And with control points from woa-ed images they tend
to be very good indeed, as this removes most of the geometric
differences between the images in the first place.
I'll have a look at the paper quo quote - I'm interested in detectors
for matching and I appreciate that it's a fine art to find something
precise and at the same time computationally inexpensive - usually
beyond my mathematic horizon. I still would like to know what the
difference between the detector in cpfind and SURF is, though - in a
nutshell ;-)
Kay