How to use geometry vs. feature based control points?
Bill Sherman
I just compiled Hugin two days ago, and love that there is an
open-source panorama stitching tool (and that it works on Linux).
I've read through a few tutorials (and watched a couple on YouTube),
but there are still a few things I don't have a good handle on.
Plus I have a handful of basic questions.
I thought I'd make my first post to the list one of the bigger
issues that I'd like to work through -- stitching an MxN array
of images from a robotically controlled camera, which may include
a featureless sky and/or limited/changing features in water.
I use a GigaPan EpicPro device for the image sequence capture, and
I get considerable overlap in the images, so I should have a decent
set of images for my panoramic captures. The GigaPan unit comes
with software (not for Linux) that will automatically do the
stitching. This software does a decent job, but there are clear
seeming issues, and also it doesn't do any context-aware analysis
to determine from which image to take a possibly moving object. All
the stitching parameters are determined by the fact that it knows
it's taking an MxN array of images, so it doesn't need feature-based
control points to do the work.
But because of the fact that this stitching isn't perfect (or really
sufficiently good), I've been looking for other solutions, and I
recently found Hugin. Hugin is also nice because I'll be able to
do the work on my desktop (though I suppose when it's sucking all
the resources from my machine I won't appreciate it so much).
So after taking a couple hours to get Hugin and it's dependencies
compiled, and reading through/watching a tutorial or two, I began
some practice panorams. It happens that my first trial is a
night image of the Washington Monument, and so there are some
featureless sky segments. I managed to get a panoram if I removed
those images, but I would like to widen the image, which requires
the inclusion of pure sky (or ripply water).
So for the areas where there are good features, Hugin worked well
in finding control points. But since the relationship between all
my images is known from the mechanized capture process, I would
like to be able to resort to this knowledge when no feature matches
can be found.
Has anyone successfully created a workflow that adds control points
based on geometry rather than feature matches?
If one were to somehow calculate these control points, how best to
get them into the system, and then not have them altered by the
optimizer stage, etc.?
I have placed some sample images on the web for clarification of
what I'd like to accomplish:
- http://www.avl.iu.edu/~shermanw/HuginTest/
There are 16 images in an 8x2 array shot with 400mm focal length
in that directory.
Oh, and I'm using Hugin version 2010.4.0.854952d82c8f (released
01/01/11).
Finally, if you go to my base URL, you can see some of the panorams
I've made using the GigaPan software (and some of the other issues
I hope Hugin will help with, like correcting my use of auto-exposure):
- http://www.avl.iu.edu/~shermanw
Thank you for your anticipated help,
Bill
--
Bill Sherman
Sr. Technology Advisor
Advanced Visualization Lab
Pervasive Technology Inst
Indiana University
sher...@indiana.edu
Bruno Postle
>
>So for the areas where there are good features, Hugin worked well
>in finding control points. But since the relationship between all
>my images is known from the mechanized capture process, I would
>like to be able to resort to this knowledge when no feature matches
>can be found.
The answer at the moment is that you need to manually place the
featureless photos, either by dragging them in the Fast Preview
window or by typing in the estimated positions in the Images tab.
We have discussed writing some tool that automatically
interpolates/extrapolates the positions of unconnected photos, but
this doesn't exist yet.
Alternatively if your panorama robot writes a log file then this
could potentially be used to automate the creation of a .pto
project.
Slightly related: papywizard <http://www.papywizard.org/> is nice
control software for robot heads - If somebody is using this and
sends me a papywizard log file, I'll try and write a converter to
generate a Hugin .pto project file.
--
Bruno
kfj
On 17 Feb., 19:57, Bill Sherman <sherm...@indiana.edu> wrote:
> Has anyone successfully created a workflow that adds control points
> based on geometry rather than feature matches?
>
> If one were to somehow calculate these control points, how best to
> get them into the system, and then not have them altered by the
> optimizer stage, etc.?
What you need for your images is correct yaw, pitch and roll values -
control points are only a means to get them. As Bruno has pointed out,
these values might be derived from the capture process, be it from
it's parameters as passed to the robotic head, be it as a log file of
sorts. If the 'featureless' images are placed (roughly) by using the
capture process y, p and r values, and if they're not connected by
control points with other images, they'll be left where they are by
the optimizer. Putting the y, p and r values into the pto script,
which is a mere text file, isn't difficult.
On the other hand, where you have images with features, you can do the
last bit of nudging via CPs and the optimizer. In corner cases, when
you have only little usable content somewhere in an image, you need to
manually intervene if the stitch isn't perfect.
Another method to deal with featureless areas is to have some more
wide-angle material to use as backdrop, in which case you can omit
those images that don't yield (usable) CPs.
Setting CPs following the shooting pattern is not a good idea, because
CPs should reflect matching content, as determined by the feature
detector in the CP generator - or your eyes, if you set them manually.
Kay
Bill Sherman
> Topic: How to use geometry vs. feature based control points?
> <http://groups.google.com/group/hugin-ptx/t/28d5e777cace3039>
> Bruno Postle <br...@postle.net> Feb 17 08:46PM ^ <#digest_top>
>
> On Thu 17-Feb-2011 at 13:57 -0500, Bill Sherman wrote:
> >my images is known from the mechanized capture process, I would
> >like to be able to resort to this knowledge when no feature matches
> >can be found.
>
> The answer at the moment is that you need to manually place the
> featureless photos, either by dragging them in the Fast Preview
> window or by typing in the estimated positions in the Images tab.
Okay, well any solution at the moment will be a good place for me to
start. So let me ask a couple followup questions:
1) I'm having trouble with the Move/Drag portion of the Preview (actually,
to tell the truth, much of how to use the Preview window is a
mystery to me). I just reran Hugin, using my sample images, and
then go to the Move/Drag operation. At this point, I can move
some of the images independently, but some stay stuck together -- and
I want to move one relative to the others.
2) What units are the X, Y, & Z values under the "Image Orientation"
label in the Image tab? Are they percentages? And is there a way
to "lock" them? (or is the way to lock them simply to delete any
control points that get generated for that image?)
3) I noticed an image referred to in a post from yesterday an "Overview"
sub window for the preview, but I do not see that in my version. Will
this be in the next release? Will there be a new release soon? Here's
the picture:
- http://www.flickr.com/photos/36383814@N00/5451289889
> We have discussed writing some tool that automatically
> interpolates/extrapolates the positions of unconnected photos, but
> this doesn't exist yet.
>
> Alternatively if your panorama robot writes a log file then this
> could potentially be used to automate the creation of a .pto
> project.
The robot device does not directly communicate with any computer, so
there is no log. However, it reports on the screen the angle between
shots, so I could easily provide command line arguments of the MxN
array, with both horizontal and vertical movement angles.
And I'd be happy to help figure out the best procedure for this.
> Slightly related: papywizard <http://www.papywizard.org/> is nice
> control software for robot heads - If somebody is using this and
> sends me a papywizard log file, I'll try and write a converter to
> generate a Hugin .pto project file.
If you can set me on the right course, I could potentially write a .pto
generator myself -- though a template would be helpful. A first question:
would I create control points, or are the image locations stored in
the .pto file? What documentation should I read first?
> Bruno
Thank you for the help,
Bill
kfj
> 1) I'm having trouble with the Move/Drag portion of the Preview (actually,
> to tell the truth, much of how to use the Preview window is a
> mystery to me). I just reran Hugin, using my sample images, and
> then go to the Move/Drag operation. At this point, I can move
> some of the images independently, but some stay stuck together -- and
> I want to move one relative to the others.
way of previewing, offereing some features some people won't do
without. Most things are easier to do in the openGL preview, that's
the icon next to it with GL on it.
> 2) What units are the X, Y, & Z values under the "Image Orientation"
> label in the Image tab? Are they percentages? And is there a way
> to "lock" them? (or is the way to lock them simply to delete any
> control points that get generated for that image?)
your project isn't a mosaic, best leave them at 0. The relevant
parameters for panoramas are yaw, pitch and roll (y, p and r)
> 3) I noticed an image referred to in a post from yesterday an "Overview"
> sub window for the preview, but I do not see that in my version. Will
> this be in the next release? Will there be a new release soon? Here's
> the picture:
> The robot device does not directly communicate with any computer, so
> there is no log. However, it reports on the screen the angle between
> shots, so I could easily provide command line arguments of the MxN
> array, with both horizontal and vertical movement angles.
>
> generator myself -- though a template would be helpful. A first question:
> would I create control points, or are the image locations stored in
> the .pto file? What documentation should I read first?
http://wiki.panotools.org/PTStitcher
The Wiki is a treasuretrove of information, but at times it's not so
easy to find what you are looking for. If you want to look at
scripting panotools, have a look at
http://wiki.panotools.org/Panorama_scripting_in_a_nutshell
Bruno has many perl scripts at
http://search.cpan.org/dist/Panotools-Script/
and if you're into Python, there is a bit of code at my Bazaar repo at
http://bazaar.launchpad.net/~kfj/+junk/script/files/head:/main/
Kay
Bruno Postle
>
>1) I'm having trouble with the Move/Drag portion of the Preview (actually,
>to tell the truth, much of how to use the Preview window is a
>mystery to me). I just reran Hugin, using my sample images, and
>then go to the Move/Drag operation. At this point, I can move
>some of the images independently, but some stay stuck together -- and
>I want to move one relative to the others.
The fast preview lets you drag connected image groups separately, a
connected group is defined as any set that is connected by control
points.
>3) I noticed an image referred to in a post from yesterday an "Overview"
>sub window for the preview, but I do not see that in my version. Will
>this be in the next release? Will there be a new release soon? Here's
>the picture:
> - http://www.flickr.com/photos/36383814@N00/5451289889
Yes the overview will be in the next release, currently you need to
build a snapshot from Mercurial to get it.
>The robot device does not directly communicate with any computer, so
>there is no log. However, it reports on the screen the angle between
>shots, so I could easily provide command line arguments of the MxN
>array, with both horizontal and vertical movement angles.
>
>And I'd be happy to help figure out the best procedure for this.
What other ways do robots allow you to specify a sequence?
Presumably you have an option to start at the top/bottom or
left/right, or to specify the total angle rather than number of
shots, or specify the total angle and angle between shots.
--
Bruno
Martin Terblanche
save all the positions to a log file. I want to automate the entire
process of creating a panorama in Hugin.
So i'm basically doing exactly the same thing as Bruno. I want to use
a log file to help Hugin to place photos in areas with no control
points. It seems like Papy wizard can create such a file (xml) and APP
can then use this. But APP is expensive and I would like to use Hugin.
Would you set the y,p,r of all the photos and then run the optimizer,
all in a script? What train of thought must I follow here :)
On Feb 19, 1:58 am, Bruno Postle <br...@postle.net> wrote:
> On Fri 18-Feb-2011 at 10:54 -0500, Bill Sherman wrote:
>
>
>
> >1) I'm having trouble with the Move/Drag portion of the Preview (actually,
> >to tell the truth, much of how to use the Preview window is a
> >mystery to me). I just reran Hugin, using my sample images, and
> >then go to the Move/Drag operation. At this point, I can move
> >some of the images independently, but some stay stuck together -- and
> >I want to move one relative to the others.
>
> The fast preview lets you drag connected image groups separately, a
> connected group is defined as any set that is connected by control
> points.
>
> >3) I noticed an image referred to in a post from yesterday an "Overview"
> >sub window for the preview, but I do not see that in my version. Will
> >this be in the next release? Will there be a new release soon? Here's
> >the picture:
Bruno Postle
> I'm doing a project where I'm building a motorized head which will
> save all the positions to a log file. I want to automate the
> entire process of creating a panorama in Hugin.
>
> So i'm basically doing exactly the same thing as Bruno. I want to
> use a log file to help Hugin to place photos in areas with no
> control points. It seems like Papy wizard can create such a file
> (xml) and APP can then use this. But APP is expensive and I would
> like to use Hugin.
Papywizard is open source, so it would be a good choice for
controlling your panoramic head.
> Would you set the y,p,r of all the photos and then run the
> optimizer, all in a script? What train of thought must I follow
> here :)
Unfortunately (as mentioned elsewhere in this thread) you can place
the photos according to a log file, but photos move around as soon
as you optimise, so you lose any benefit of aligning them
beforehand.
--
Bruno
Jeffrey Martin
will your head calculate the optimum number of shots per row? if it doesn't, the utility is very very limited in my experience. you need a motorized head only for >200 shot panos, and with that type of lens, the number of shots when your camera is pointing nearly straight up or down is so much less than when it's pointing straight ahead, that the time wasted if you don't optimize for this, is huge.
and yes, you should use papywizard.
Jeffrey
Martin Terblanche
now. But I will certainly post some pictures as I go along. At the
moment I dont want to use papywizard because I want the product to be
standalone. No need for a phone, tablet, notebook in the field.
Jeffrey Martin
I beg you to use papywizard anyway - at least to produce a Papywizard-equivalent set of XML, which can be imported into Autopano Giga (and eventually, Hugin also...)
Also, considering the head needs a "controller" of some sort, it is not standalone, strictly speaking - it has a little computer of some kind attached to it, to configure the pano routine you're shooting.
If you'd like more thoughts on the matter, feel free to contact me offlist. I've used robotic panoheads quite extensively so I might be able to offer you a few things to think about that you might not have considered already.
either way, I look forward to seeing your results. Pano robots are great!
jeffrey