Field of view, crop and canvas size

[Merlijn B.W. Wajer]

Hi everyone,

I've been working on stitching a lattice/grid of 4x4 images together
with Hugin. I'm running into problems where parts of the final image
gets cut off and I am hoping to find some advice here to prevent that
from happening.

A few quick details: the images are made at a high magnification and
high resolution (151PM), a thumbnail of the layout can be found here
[1]. The stitching is part of an effort to digitise microfiche cards and
the stitching is a fully automated process - so people aren't supposed
to say launch Hugin and fix some problem(s). I have written a custom
control point finder using ORB which utilises the fact that is knows
which images overlap and in what regions they overlap. The output images
are about ~1.3 gigapixel grayscale images.

The Python code that I have written can be found here [2]. Note that the
custom control point finder seems to work well and shouldn't be the
cause of my issues here - and neither should multiblend, as I can see
the fov causing a bit to be cut off in Hugin too.

So my problem is that in some cases, parts of my images get "cut off" in
the final blended image, which I have come to understand is a function
of the 'field of view' that can be set using pano_modify (and in Hugin
as well).

In the past I relied on pano_modify to find the field of view by itself
(using --fov=AUTO) but this caused the same problem as above: where a
part of the final image is cut off. So currently I set the fov to a
'known good' value for my setup - but this still causes some of the
images to be cut off. An additional problem is that if I raise the fov
much further, the pano_modify tool will hang (run forever) or crash when
trying to automatically calculate the canvas and/or crop size using
--canvas=AUTO and --crop=AUTO. Additionally, --canvas=AUTO seems to
create a canvas that "feels" like it is way too large and I would be
able to fit my stitched images on it more than a few times, but I don't
know enough about how this works to understand how this could go wrong.

Summarising:

1. --fov=AUTO doesn't work for me in some cases
2. Setting a (too) large fov breaks --canvas=AUTO and --crop=AUTO
3. --canvas=AUTO seems to usually create a unwieldy and overly large canvas

Does anyone have any tips on how to deal with this? I feel like there's
something that I am missing.

Ideally I'd just tell the software to include as much from all my input
images as possible without having to specify the field of view or canvas
size, or maybe I could write some custom code to try to set the right
values?

Regards,
Merlijn

[1] https://archive.org/~merlijn/micro_IA40385005_1500_itemimage.png
[2] https://archive.org/~merlijn/microstitch-hugin.py

[T. Modes]

mer...@archive.org schrieb am Dienstag, 25. Juni 2024 um 20:00:49 UTC+2:

The Python code that I have written can be found here [2].

Sorry, but this unnecessary complicated. I had to dig deep into your script to found the Hugin commands you use. It would be easier if you would have only cited your used commands.

The main problem is that you optimize yaw of all images:

pto_var -o  project2 --opt y0,r,y,..

Because you have not other constraints like vertical control point the optimizer result is ambiguous and the pano can easily slide to left or right.

The best advice is to look at the intermediate project files in the GUI and see the result.

PS: Hugin has also possibilities to handle such use cases.

First assign rough image positions according to your layout. Either with a template (pto_template) or with numbers/expressions (pto_var).

Then run cpfind --prealigned ... on this project file. Then cpfind looks only at overlapping images. So no need to reinvent the wheel twice.

Thomas

[Merlijn B.W. Wajer]

Hi Thomas,

Thank you for the reply and thanks for Hugin! I have replied inline.

On 26/06/2024 17:06, 'T. Modes' via hugin and other free panoramic

software wrote:
> mer...@archive.org schrieb am Dienstag, 25. Juni 2024 um 20:00:49 UTC+2:
>
> The Python code that I have written can be found here [2].
>
>
> Sorry, but this unnecessary complicated. I had to dig deep into your
> script to found the Hugin commands you use. It would be easier if you
> would have only cited your used commands.

My apologies, I'll try to just include the commands next time, I thought
this was maybe some useful context. In the next few weeks/months I also
hope to put the code as a whole (including the ORB control point finder)
online some where in case it could maybe benefit others.

> The main problem is that you optimize yaw of all images:
> pto_var -o  project2 --opt y0,r,y,..
> Because you have not other constraints like vertical control point the
> optimizer result is ambiguous and the pano can easily slide to left or
> right.

I see, that makes sense then. The material that is being photographed
moves around ever so slightly, and because one pixel is about 3.76 μm I
found that I had to also adjust for the roll and yaw. Would there be a
way to tell it to not change/optimise the yaw by more than (say) a few
degrees? That could prove to be quite helpful.

Would you mind elaborating a little bit on what it is that you mean when
you mention vertical control point constraints?

> The best advice is to look at the intermediate project files in the GUI
> and see the result.

Do you mean to look at the intermediate project files to see what
variables need to be optimised for a good result? This is how I came up
with the current workflow (as well as some online resources) and why I
split the steps up into different "project" files (project.pto,
project1.pto, project2.pto, etc).

> PS: Hugin has also possibilities to handle such use cases.
> First assign rough image positions according to your layout. Either with
> a template (pto_template) or with numbers/expressions (pto_var).
> Then run cpfind --prealigned ... on this project file. Then cpfind looks
> only at overlapping images. So no need to reinvent the wheel twice.

The reasons I made a custom control pointer finder is that (1) cpfind
wouldn't find control points in cases where I really expected it to find
control points (also with different sieve1/sieve2 sizes) and (2) because
cpfind is relatively slow for the images (151 megapixel times 16) that I
am working with, even if things are prealigned (as I understand it).
After automatically stitching 10000+ images I'm very close to a 100%
success rate, but I couldn't quite get there with just cpfind.

I didn't figure out how to use pto_template make a template with
pre-aligned images. Is there any documentation available on how to do this?

I did manage to run cpfind with the --prealigned option on a project
file where I had already done some control point finding and
optimisating step, but I recall that is was still quite a bit slower -
which is why the code that I provided above falls back to cpfind
(although not with --prealigned flag currently) when it cannot find
control points on its own.

If I could figure out how to use this template option that would help a
lot and then I could also adjust my ORB control point finder to read the
orientation/layout from the project files as well - which would make the
code more generic, and as you say, prevent having to partially reinvent
the wheel.

Regards,
Merlijn

[T. Modes]

Hi

mer...@archive.org schrieb am Mittwoch, 26. Juni 2024 um 18:16:34 UTC+2:

> The main problem is that you optimize yaw of all images:
> pto_var -o  project2 --opt y0,r,y,..
> Because you have not other constraints like vertical control point the
> optimizer result is ambiguous and the pano can easily slide to left or
> right.

I see, that makes sense then. The material that is being photographed
moves around ever so slightly, and because one pixel is about 3.76 μm I
found that I had to also adjust for the roll and yaw. Would there be a
way to tell it to not change/optimise the yaw by more than (say) a few
degrees? That could prove to be quite helpful.

No, there are no such constraints possible.

Maybe drop the optimization of y0 and call pano_modify --center after the optimization to bring the pano back to the center.

 

> PS: Hugin has also possibilities to handle such use cases.
> First assign rough image positions according to your layout. Either with
> a template (pto_template) or with numbers/expressions (pto_var).
> Then run cpfind --prealigned ... on this project file. Then cpfind looks
> only at overlapping images. So no need to reinvent the wheel twice.
The reasons I made a custom control pointer finder is that (1) cpfind
wouldn't find control points in cases where I really expected it to find
control points (also with different sieve1/sieve2 sizes) and

You are using very high number for sieve1width/height. This could also slow down the processing.

 

(2) because
cpfind is relatively slow for the images (151 megapixel times 16) that I
am working with, even if things are prealigned (as I understand it).
After automatically stitching 10000+ images I'm very close to a 100%
success rate, but I couldn't quite get there with just cpfind.

I didn't figure out how to use pto_template make a template with
pre-aligned images. Is there any documentation available on how to do this?

Take a project file of a finished pano with the same layout. Then use this project file as template for another project file with the same layout (and same number of images).

And then use cpfind --prealigned on this project.

Thomas

[T. Modes]

Another issues popped up in the mean time.

mer...@archive.org schrieb am Mittwoch, 26. Juni 2024 um 18:16:34 UTC+2:

(2) because
cpfind is relatively slow for the images (151 megapixel times 16) that I
am working with, even if things are prealigned (as I understand it).
After automatically stitching 10000+ images I'm very close to a 100%
success rate, but I couldn't quite get there with just cpfind.

Another issue could be such big images. Maybe there is not enough memory to keep several images parallel in RAM.

Have a look at the number of used threads and the memory usage.

Then you could try to decrease the number of threads (cpfind --ncores=..)

Thomas

[Merlijn B.W. Wajer]

Hi Thomas,

On 26/06/2024 19:21, 'T. Modes' via hugin and other free panoramic

software wrote:
> Hi
> mer...@archive.org schrieb am Mittwoch, 26. Juni 2024 um 18:16:34 UTC+2:
>
> > The main problem is that you optimize yaw of all images:
> > pto_var -o  project2 --opt y0,r,y,..
> > Because you have not other constraints like vertical control
> point the
> > optimizer result is ambiguous and the pano can easily slide to
> left or
> > right.
>
> I see, that makes sense then. The material that is being photographed
> moves around ever so slightly, and because one pixel is about 3.76 μm I
> found that I had to also adjust for the roll and yaw. Would there be a
> way to tell it to not change/optimise the yaw by more than (say) a few
> degrees? That could prove to be quite helpful.
>
> No, there are no such constraints possible.
> Maybe drop the optimization of y0 and call pano_modify --center after
> the optimization to bring the pano back to the center.

Understood, thanks.

> > PS: Hugin has also possibilities to handle such use cases.
> > First assign rough image positions according to your layout.
> Either with
> > a template (pto_template) or with numbers/expressions (pto_var).
> > Then run cpfind --prealigned ... on this project file. Then
> cpfind looks
> > only at overlapping images. So no need to reinvent the wheel twice.
> The reasons I made a custom control pointer finder is that (1) cpfind
> wouldn't find control points in cases where I really expected it to
> find
> control points (also with different sieve1/sieve2 sizes) and
>
> You are using very high number for sieve1width/height. This could also
> slow down the processing.

That is right -- I added that at some point to try to get a higher
success rate, but given that the template approach is working for me now
I might be able to get rid of that.

>
> (2) because
> cpfind is relatively slow for the images (151 megapixel times 16)
> that I
> am working with, even if things are prealigned (as I understand it).
> After automatically stitching 10000+ images I'm very close to a 100%
> success rate, but I couldn't quite get there with just cpfind.
>
> I didn't figure out how to use pto_template make a template with
> pre-aligned images. Is there any documentation available on how to
> do this?
>
> Take a project file of a finished pano with the same layout. Then use
> this project file as template for another project file with the same
> layout (and same number of images).
> And then use cpfind --prealigned on this project.

With the above advice I got this to work. If only I understood
previously that templates were this simple. I ended up making four
templates for the four different setups that I have and now I've been
able to stitch all the remaining images by falling back to cpfind
--prealigned when my ORB stitching doesn't work.

Really appreciate the help!

Regards,
Merlijn