Hugin operation questions from a mostly-newby

[William Sherman]

Hello again,

Now that I have Hugin 2014.0.0_rc4 compiled and running, I've got
a few questions for the list on usage. I'm doing my best to look
on the web first for answers, but of course I may not stumble upon
the nugget I need. Also, I've used Hugin before, but it's been a
couple years, so consider me a half-newby.

My first issue is that I'd like to find a way to constrain what
images are compared in the control-point search. I've taken a
range of panorams, and the largest are quite large -- over 1000
pictures. So I'd like to find a way to limit control point
analysis to say the 8 neighboring pictures of any one picture.
I figure that for an 801 image shot, that would save two orders
of magnitude in control point searching (8 vs. 800).


And in the meantime (ie. I've done a couple of experiments since
typing the above), I now have a couple of *real* problems rather
than the hypothesized problem above (which I still wonder about).

Okay, so over the past couple days I've been experimenting with
some image collections that I shot a few weeks ago. I started
with a 21 image collection of a hand-held shoot, and then I did
a 210 image collection shot with a Gigapan "robot".

In both cases, I'm using the Assistant mode, though I did venture
into the control point views and some other tabs for experimenting.
Also, I did tests with both "enblend" and "multiblend" as the
blending tool.

*** Small ***
So the 21 image collection worked pre-well. I tried it first with
multiblend, and no enfuse (because I hadn't compiled it yet), and
that went pretty fast, and with reasonable results. It cut the
head off of one person, but for a final rendering I'd go back and
mask out the entire person anyway. That took just a few minutes
to process.

I then went back and ran with enblend/enfuse (selecting the
"Exposure fused from any arrangements" option), and processing
that took an hour or two (vs. about 10 minutes in the first
attempt). I suppose most of that is in the exposure correction
step, or maybe multiblend is really *that much faster* than
enblend. With enblend, the guy has his head, but the flag
gets oddly blended (whereas multiblend took the entire flag
from one image, so it looks fine).


*** Medium ***
Okay, so now onto a 210 image shoot (42x5) using a Gigapan robot.

First, loading the images it put them horizontally, whereas they
were shot vertically, so it has a typical striation pattern from
a misalignment -- I know that at this stage that doesn't matter,
but would be nice if I could provide Hugin with the basic layout
of the images.

I then ran the "2. Align" step. And this took over 6 hours to
process. It was not fun waiting for that, and I dread what will
happen when I get to my "large" image collections! I did notice
that at times all the CPUs were going, and then other times just
a single CPU. Also, I'd be interested to know exactly what
"Optimizing Variables" were being optimized to get a sense for
where it is in the process.

Eventually, it produced a result, and the pictures are in the
proper order.

With one huge problem -- the pictures wrap around past 360 to
about 400 degrees. In reality the shoot was about 270 degrees.

I looked and looked for a way to correct this -- tried using
the "Field of View" parameter under the "Projection" tab,
and pressed the "Fit" button, but that just did some processing,
then returned the horizontal FOV to 360, but shrunk the vertical
FOV! But it left all the pictures where they were, so there is
still about 40 degrees of overlap.

I figured I might as well see what happens, so I began the
"3. Create panorama" process. Here, likely because I have
overlapping pictures, it no longer has the "Exposure corrected,
low dynamic range" option turned on (for some definition of
option, because it seems to always be greyed out). This then
forces me to choose one of the fusing options, so I chose
"exposure fused from stacks".

The process began, but a short while later (5 minutes or so),
it seg-faulted. Here's the end of the log file:
processing IMG_7374-IMG_7584_stack_ldr_0073.tif...
processing IMG_7374-IMG_7584_stack_ldr_0074.tif...
processing IMG_7374-IMG_7584_stack_ldr_0075.tif...
processing IMG_7374make: *** [IMG_7374-IMG_7584_fused.tif]
Segmentation fault (core dumped)
make: *** Deleting file `IMG_7374-IMG_7584_fused.tif'

I looked at the "stack_ldr" pictures, and as I expected the two
ends of the panoram are blended together.

I also noticed that the list of "exposure_layers" temporary
images was missing at least one in the sequence, so perhaps
this could be related to the seq-fault.

While typing this email I decided to select the other exposure
option ("Exposure fused from any arrangements"), but since it's
about 2:00am for me, I may not see then end of it before sending
this email. (It's creating the "exposure_layer" intermediate
images now.)

Okay, it failed before I finished typing the email. But this
time it complained:
"not enough memory for image channel"
while creating the "exposure_layers" images.

So I increased the memory for "multiblend" with the "-m 30000"
option.

But like a bad scientist, I then changed some other parameters.
Mostly I changed the projection back from Mercator to Cylindrical,
and this gave me new options at the processing stage, whereby
the "Exposure corrected, low dynamic range" option was not greyed
out for the first time in my recent experience. So I selected that,
and unselected the "exposure fusion" option. So now "enfuse" shouldn't
be needed. Again, if something interesting happens before I complete
the "Large" section of this email I amend things here.

One thing I notice is that for my small panoram, the Batch Processor
is informing me that a Cylindrical projection was chosen for the
output, and for the medium, a Mercator projection was chosen.


*** Large ***
I haven't done it yet, but I have some fairly large image
collections that I want to turn into a giga-pixel image, and
I was hoping to use Hugin for this. So about 1600 images
per shoot (single exposure).

Is there a method by which I can process panorams consisting
of 1600 images with Hugin? I sure hope so.

I have a license for KRpano, so I'll use that to create a pyramid
tiled image set with a web interface when I'm done.


BTW, I receive the Hugin mailing list by digest, so unless you
CC me, I'll have to wait for the next mailing to see the response.

Thank you for your help,
Bill

Still only about a third the way through nona processing the
images in my current test, so signing off for now.

--
Bill Sherman
Sr. Technology Advisor
Advanced Visualization Lab
Pervasive Technology Inst
Indiana University
sher...@indiana.edu

[Bruno Postle]

On Mon 08-Sep-2014 at 02:08 -0400, William Sherman wrote:
>
> My first issue is that I'd like to find a way to constrain what
> images are compared in the control-point search. I've taken a
> range of panorams, and the largest are quite large -- over 1000
> pictures. So I'd like to find a way to limit control point
> analysis to say the 8 neighboring pictures of any one picture.

You can try different control point strategies in File ->
Preferences. The default CPFind detector tries to minimise the
number of comparisons by assuming you have taken photos in rows.
Alternatively if you can approximately arrange the photos using a
template, there is a 'prealigned' detector that only compares photos
that are nearby.

> Okay, so now onto a 210 image shoot (42x5) using a Gigapan robot.
>
> First, loading the images it put them horizontally, whereas they
> were shot vertically, so it has a typical striation pattern from a
> misalignment -- I know that at this stage that doesn't matter, but
> would be nice if I could provide Hugin with the basic layout of
> the images.

Hugin will read any EXIF orientation data from the camera to try and
get this right. If your photos don't have this data then you need
to used the advanced/expert interface to set the 'roll' of the
photos before alignment.

> I then ran the "2. Align" step. And this took over 6 hours to
> process. It was not fun waiting for that, and I dread what will
> happen when I get to my "large" image collections! I did notice
> that at times all the CPUs were going, and then other times just a
> single CPU. Also, I'd be interested to know exactly what
> "Optimizing Variables" were being optimized to get a sense for
> where it is in the process.

Definitely you need to stop using the Assistant for such a big image
set. The Assistant will optimise all sorts of lens parameters by
default, whereas if you have already characterised your lens there
is only any need to optimise roll, pitch and yaw for the photos.

The optimiser is much quicker when optimising fewer parameters.

> Eventually, it produced a result, and the pictures are in the
> proper order.
>
> With one huge problem -- the pictures wrap around past 360 to
> about 400 degrees. In reality the shoot was about 270 degrees.

The angle of view parameter for your _photos_ is too high. Normally
the Assistant optimises this, but if your panorama doesn't cover
much of the sphere surface then there is no way of knowing how big
it is supposed to be - i.e. your panorama is too thin and incomplete
to be able to calculate lens angle of view.

You need to characterise the angle of view of your lens by aligning
a full 360°, then use this number from then on. You could
alternatively stitch a not-thin panorama, say 90x90°, this would
work for calculating lens angle of view too.

> I looked and looked for a way to correct this -- tried using the
> "Field of View" parameter under the "Projection" tab, and pressed
> the "Fit" button, but that just did some processing, then returned
> the horizontal FOV to 360, but shrunk the vertical FOV! But it
> left all the pictures where they were, so there is still about 40
> degrees of overlap.

This is adjusting the angle of view of the _output panorama_, which
can never be larger than 360°.

> I figured I might as well see what happens

> Segmentation fault (core dumped)
> make: *** Deleting file `IMG_7374-IMG_7584_fused.tif'
>
> I looked at the "stack_ldr" pictures, and as I expected the two
> ends of the panoram are blended together.

Yes this is going to be a mess because you have overlapping photos
that shouldn't be.

> I also noticed that the list of "exposure_layers" temporary images
> was missing at least one in the sequence, so perhaps this could be
> related to the seq-fault.

The segfault could be something else, possibly just running out of
memory.

> One thing I notice is that for my small panoram, the Batch
> Processor is informing me that a Cylindrical projection was chosen
> for the output, and for the medium, a Mercator projection was
> chosen.

You need to take control of the software. The Assistant will pick a
projection for you using heuristics, but you probably always want to
stitch these panoramas in equirectangular projection.

In general, Hugin has lots of options and widgets which can be
bewildering, the feature-matching, alignment and stitching process
also has some quirks which are logical but unexpected until you see
them in action - trying to learn all this with a large image set is
going to be extremely frustrating.

> Is there a method by which I can process panorams consisting of
> 1600 images with Hugin? I sure hope so.

Yes though you need lots of memory for stitching.

--
Bruno

[Rogier Wolff]

On Tue, Sep 09, 2014 at 12:01:59AM +0100, Bruno Postle wrote:
> On Mon 08-Sep-2014 at 02:08 -0400, William Sherman wrote:
> >I then ran the "2. Align" step. And this took over 6 hours to
> >process. It was not fun waiting for that, and I dread what will
> >happen when I get to my "large" image collections! I did notice
> >that at times all the CPUs were going, and then other times just a
> >single CPU. Also, I'd be interested to know exactly what
> >"Optimizing Variables" were being optimized to get a sense for
> >where it is in the process.

William,

The optimization process cannot optimize a variable at a time: they
are all interconnected.

The optimization works a bit as follows. Suppose we have a function
that we don't know. In fact it's y=x^2, but again, we don't know that.
Suppose we want the result to be "10". So we try puttin in "1" and get
"1" put in "2" and we get "4". From this we calculate that adding one
to x, results in "3" more in the output. We need 6 more output, so we
add 6/3 = 2 to the input. 2+2 = 4, so we try 4 next. We get "16",
which is way too far. now we have from 2 to 4 on the input the
difference in output is 12, and we need to be exactly in the
middle. So we probe exactly in the middle: 3. So now we get "9" and
need to go a little higher. 16-9 is 7, we need to be 1/7th on the way
from 9 to 16, 1/7th of the way from 3 to 4: 3+1/7 = 3.1428. Now we get
pretty close, and it gets annoying doing this by hand. But now that we
are close, we get closer really, really quickly. Just a few more steps
and we'd have our solution to many decimal places. (after these few
steps "by hand, we have only a 0.6% difference with the mathematical
solution).

For panoramas there are a whole lot of parameters to adjust. This
means that there no longer is a "mathematical solution". But the
"approximation technique" still works. So for hugin to optimize the
panorama, it calculates how much to change all the parameters at once,
then it changes them all and next it will start the process of finding
how much to change each one all over again.

From my example, you can see that if you start off with a "pretty bad"
estimate, it takes a few iterations to get close enough to get into
the "gets closer really really quickly" region. The number of
iterations required to get "close enough" gets more and more as the
number of parameters increases.


Roger.

--
+-- Rogier Wolff -- www.harddisk-recovery.nl -- 0800 220 20 20 --
- Datarecovery Services Nederland B.V. Delft. KVK: 30160549 -
| Files foetsie, bestanden kwijt, alle data weg?!
| Blijf kalm en neem contact op met Harddisk-recovery.nl!

[William Sherman]

On 9/8/14 7:01 PM, Bruno Postle wrote:

Hello again,

I've been doing some tests -- many with an alternate tool, so
I'm back to learn more.

First, the alternate tool I've been testing is the Gigapan Stitcher
tool, and since I'm using a Gigapan panorama head it seemed like a
natural -- except for my predilection for Linux and open-source that
is.

So the stitch that took Hugin 6 hours (and for which I hadn't found
the proper means of setting the scope of the images), took Gigapan
Stitcher 16 minutes. And that was with basically one button press:
-> "Stitch"

The downside is that it too of course makes some errors, and I don't
have the option to go in and tweak things. Well, I can tweak the colors,
but I can't mask parts from particular images, and force the inclusion
from others. I can't go in and tweak control points when it makes
mistakes.

So the two panoramas I've stitched with Gigapan stitch can be viewed
at:
- http://web.avl.indiana.edu/~shermanw/MST

the second one is from 1547 images, but if you look to the left,
you'll notice that that process went a little wonky. It's actually
a cool effect -- love that AT&T building. But for the Gateway Arch,
it really could have done a much better job.

The other downside of Gigapan stitch is that it can't handle hand-held
shoots unless they rigidly take the same number of pictures in each
column & row.

>> My first issue is that I'd like to find a way to constrain what images
>> are compared in the control-point search. I've taken a range of
>> panorams, and the largest are quite large -- over 1000 pictures. So
>> I'd like to find a way to limit control point analysis to say the 8
>> neighboring pictures of any one picture.
>
> You can try different control point strategies in File -> Preferences.
> The default CPFind detector tries to minimise the number of comparisons
> by assuming you have taken photos in rows. Alternatively if you can
> approximately arrange the photos using a template, there is a
> 'prealigned' detector that only compares photos that are nearby.

Nice, glad to learn about that. One things makes my wonder -- after
choosing "Hugin CPFind (prealigned)" and double-clicking on it, there
is a setting window, and for Type, it has "All images at once", and
not "Prealigned panorama". Is this a bug, or some orthogonal setting?

>> Okay, so now onto a 210 image shoot (42x5) using a Gigapan robot.
>>
>> First, loading the images it put them horizontally, whereas they were
>> shot vertically, so it has a typical striation pattern from a
>> misalignment -- I know that at this stage that doesn't matter, but
>> would be nice if I could provide Hugin with the basic layout of the
>> images.
>
> Hugin will read any EXIF orientation data from the camera to try and get
> this right. If your photos don't have this data then you need to used
> the advanced/expert interface to set the 'roll' of the photos before
> alignment.

Right, well that's not exactly what I meant. What I meant is that if
I shoot pictures in columns -- say 5 pictures per column, and then
import them into Hugin, it lays them out going left to right, not top
to bottom, so they are in the wrong place. Of course I wouldn't know
how to tell Hugin how many pictures per column, so I'd have to know
that too. Anyway, that matters only once I figure out how to do
the prealigned option.

>
>> I then ran the "2. Align" step. And this took over 6 hours to process.
>> It was not fun waiting for that, and I dread what will happen when I
>> get to my "large" image collections! I did notice that at times all
>> the CPUs were going, and then other times just a single CPU. Also, I'd
>> be interested to know exactly what "Optimizing Variables" were being
>> optimized to get a sense for where it is in the process.
>
> Definitely you need to stop using the Assistant for such a big image
> set. The Assistant will optimise all sorts of lens parameters by
> default, whereas if you have already characterised your lens there is
> only any need to optimise roll, pitch and yaw for the photos.
>
> The optimiser is much quicker when optimising fewer parameters.

Okay, good. And this is where my past experiences with Hugin may
hinder me (or maybe not), but when I go to the "Advanced" layout, under
Photos it has the name, and then columns for y, p & r, whereas from
the last I remember lots and lots of parameters for each image, which
could be turned on and off for optimization.

Aha, found a few more by switching to "Expert" mode. (I previously
hadn't noticed a difference between "Advanced" and "Expert".)

Hmmm, but I don't see the buttons that I thought were there to
tell the optimizer to skip certain values.

>> Eventually, it produced a result, and the pictures are in the proper
>> order.
>>
>> With one huge problem -- the pictures wrap around past 360 to about
>> 400 degrees. In reality the shoot was about 270 degrees.
>
> The angle of view parameter for your _photos_ is too high. Normally the
> Assistant optimises this, but if your panorama doesn't cover much of the
> sphere surface then there is no way of knowing how big it is supposed to
> be - i.e. your panorama is too thin and incomplete to be able to
> calculate lens angle of view.
>
> You need to characterise the angle of view of your lens by aligning a
> full 360°, then use this number from then on. You could alternatively
> stitch a not-thin panorama, say 90x90°, this would work for calculating
> lens angle of view too.

I guess this seems like a problem to me. First, I compiled the "lensfun"
tool, does that not have a database of lenses to at least get it
in the ballpark of what the parameters are -- which can then be
fine tuned by the optimizer?

I'll go back to my Gigapan Stitch experience -- I clicked "Stitch",
and if basically figured out that the sum total was about 270deg
horizontally. I suppose it may have used a lens database to figure
that out.

In hugin, under the "Photos" tab, there is a "Lens type" section,
and it has values for Focal length and Focal length multiplier.
Does this map somewhat at least to a DSLR lens type? So if I have
a 50mm prime on a full-sensor camera, can I provide a good guess
as to the parameters -- actually, for my 210 picture shoot it's
200mm on a 70-200 lens.

Ahh, just found a bunch more parameters for each photo with the
radio buttons to the right of the image list letting me view
lens parameters and photometric parameters, etc.

But I don't see how to set an anchor image.

So I guess, am I supposed to select each image and set it to
a new (smaller) value of "Hfov(v)".

Okay, so pardon the stream-of-consciousness flow of this post,
but maybe this will be helpful to anyone looking to understand
the newby user's experience.

So I did alter the "Hfov(v)" for one image, and that affected
all the images -- so obviously all the images point to the same
lens parameter entry.

BUT, lowering the value from 17.1 to 10.0 didn't affect the
fact that the panorama still things it's a 400 degree shot.
I then tried 5.0, and no change. So I don't know where I'd
put the lens parameters that would inform Hugin of how the
images should be laid out.

>> I looked and looked for a way to correct this -- tried using the
>> "Field of View" parameter under the "Projection" tab, and pressed the
>> "Fit" button, but that just did some processing, then returned the
>> horizontal FOV to 360, but shrunk the vertical FOV! But it left all
>> the pictures where they were, so there is still about 40 degrees of
>> overlap.
>
> This is adjusting the angle of view of the _output panorama_, which can
> never be larger than 360°.

Right!

>
>> I figured I might as well see what happens
>
>> Segmentation fault (core dumped)
>> make: *** Deleting file `IMG_7374-IMG_7584_fused.tif'
>>
>> I looked at the "stack_ldr" pictures, and as I expected the two ends
>> of the panoram are blended together.
>
> Yes this is going to be a mess because you have overlapping photos that
> shouldn't be.

It's not the image I put up on my page, but when I did get it
to provide an output, it was an interesting picture, and most
of it was fine.

>> I also noticed that the list of "exposure_layers" temporary images was
>> missing at least one in the sequence, so perhaps this could be related
>> to the seq-fault.
>
> The segfault could be something else, possibly just running out of memory.
>
>> One thing I notice is that for my small panoram, the Batch Processor
>> is informing me that a Cylindrical projection was chosen for the
>> output, and for the medium, a Mercator projection was chosen.
>
> You need to take control of the software. The Assistant will pick a
> projection for you using heuristics, but you probably always want to
> stitch these panoramas in equirectangular projection.
>
> In general, Hugin has lots of options and widgets which can be
> bewildering, the feature-matching, alignment and stitching process also
> has some quirks which are logical but unexpected until you see them in
> action - trying to learn all this with a large image set is going to be
> extremely frustrating.

And so I've looked at some of the tutorials, but of course with the
major change in the GUI, many of the tutorials are for the old version.
I guess one thing that would be handy is for the names of each tutorial
to include what version of Hugin they were created with. I haven't
yet found that one tutorial that I need.

I did notice that Terry has a new tutorial on stitching mosaics which
uses the new interface (with Hugin 2013). But most of the rest of
them are pre-2013.

>
>> Is there a method by which I can process panorams consisting of 1600
>> images with Hugin? I sure hope so.
>
> Yes though you need lots of memory for stitching.

I have 24 Gig of RAM on this machine -- and the machine that sticked
the 1547 image panorama had only 12 Gig. And that took about 6 hours.
Oddly, it took like 3 hours to write out the 12Giga-pixel image in PNG
format. Only took like 30 minutes to write as a Big-TIFF.


Anwyay, I'll press on. I just wanted to provide some feed back from
a newby perspective. Not trying to be unnecessarily critical, because
I love that this program is here, but I want to love it more.


Thanks again,
Bill

[Terry Duell]

Hello William,

I can add a little to a couple of the issues you raise.

On Thu, 11 Sep 2014 17:23:32 +1000, William Sherman <sher...@indiana.edu>
wrote:

[snip]

>
> Right, well that's not exactly what I meant. What I meant is that if
> I shoot pictures in columns -- say 5 pictures per column, and then
> import them into Hugin, it lays them out going left to right, not top
> to bottom, so they are in the wrong place. Of course I wouldn't know
> how to tell Hugin how many pictures per column, so I'd have to know
> that too. Anyway, that matters only once I figure out how to do
> the prealigned option.

If you have "Detect vertical lines" set in "Preferences -> Assistant' this
may help Hugin to get the arrangement correct. I haven't tested this in
many situations so I'm not sure if it will help you, but worth a try.

[snip]

>
> Aha, found a few more by switching to "Expert" mode. (I previously
> hadn't noticed a difference between "Advanced" and "Expert".)
>
> Hmmm, but I don't see the buttons that I thought were there to
> tell the optimizer to skip certain values.

In "Expert" mode, chose to optimise "Custom parameters" which will reveal
the "Optimizer" tab. In this tab you can select which parameters to
optimise. Right click on a heading to get a menu.

[snip]

>
> I guess this seems like a problem to me. First, I compiled the "lensfun"
> tool, does that not have a database of lenses to at least get it
> in the ballpark of what the parameters are -- which can then be
> fine tuned by the optimizer?
>

If your lens is in the Lensfun database, in the "Photos" tab, right click
on an image in the list and from the menu select "Lens -> load lens
from..." which should open a dialog box showing lenses hugin has found
that might be your lens.

>
> But I don't see how to set an anchor image.
>

In the "Photos" tab, righ click on an image, and in the menu you will see
"Anchor this image position" and "Anchor this image for exposure"

>
> I did notice that Terry has a new tutorial on stitching mosaics which
> uses the new interface (with Hugin 2013). But most of the rest of
> them are pre-2013.
>>

There is a tutorial "The new user interface (2013)" and "Simple lens
calibration" has recently been updated for the new interface.

Hope some of that helps.

Cheers,
--
Regards,
Terry Duell

[Brandon]

So the stitch that took Hugin 6 hours (and for which I hadn't found
the proper means of setting the scope of the images), took Gigapan
Stitcher 16 minutes.  And that was with basically one button press:
        -> "Stitch"

One thing that I have discovered that really helps speed the stitching up is on the stitching tab put "-a" with out the quotes as a command for enblend. I am generally stitching in the10-20k pixels wide range and using the -a cuts stitching time by 1/4 to 1/3. The only down side that I have found to using -a is that it is a bugger to get the masks right when there are a lot of moving objects close to the camera(think family reunion with the whole family around the camera and the people move some between shoots) For your pictures it should help stitch time a lot.