incomplete lens correction model - it bites

[klaus...@gmail.com]

Hello,

It is 10+ years that I have last been active on this group,and on the panotools wiki as well. On the wiki my old credentials still work ;-)

In these old days I did detail that from the lens parameter set abc only the br^3 term is useful. Non-zero parameters a and c introduce singularities at r=0, something a real lens does not have. My query then was to also introduce r^5 and r^7 parameters.

A few days ago I purchased a new camera in the form of the Parrot Anafi drone. The lens has some fisheye characteristics, similar to Gopro. I now discover that the available choice of lens types and parameters only allows bad alignment. It is not lens shift, it is not parallax error, it is that Barrel distortion by itself is not enough.

Actually, switching on a and c parameters does make things worse :-(

Where do you suggest should one further discuss this issue, where best to upload some panoramic source images?

Best regards
Klaus

[Erik Krause]

Am 23.07.2018 um 17:26 schrieb klaus...@gmail.com:
> A few days ago I purchased a new camera in the form of the Parrot
> Anafi drone. The lens has some fisheye characteristics, similar to
> Gopro. I now discover that the available choice of lens types and
> parameters only allows bad alignment. It is not lens shift, it is not
> parallax error, it is that Barrel distortion by itself is not
> enough.

Could you provide some images for download? Without any parallax if
possible. I'd be interested to give this a try...

--
Erik Krause
http://www.erik-krause.de

[Torsten Bronger]

Hallöchen!

klaus...@gmail.com writes:

> [...]

>
> In these old days I did detail that from the lens parameter set
> abc only the br^3 term is useful. Non-zero parameters a and c
> introduce singularities at r=0, something a real lens does not
> have.

Could you elaborate on this? Possibly by a link to the old
discussion?

While I don't want to assert that using even exponents makes sense,
I've never read a valid argument against them. There may be one, I
simply never read it.

In particular, the assumption that the underlying physics yield an
odd function is a) not justified to me so far and b) does not mean
that approximating its positive branch may not contain even
components.

Again: It may be true that a and c are superfluous or even harmful.
But I'm looking for proof of that.

> My query then was to also introduce r^5 and r^7 parameters.

FWIW, I once patched the Pano library to use the exponents 3, 5, and
7 for a, b, c. It did not help for the (albeit single) pano that I
used for testing. This is not a proof, it was only discouragement
for me to follow this path further.

Tschö,
Torsten.

--
Torsten Bronger

[Gunter Königsmann]

Would that help with cellphone lenses that tend to have a different scale in the top left corner than in the lower right one?

--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/87k1pl19ow.fsf%40wilson.bronger.org.
For more options, visit https://groups.google.com/d/optout.

[Torsten Bronger]

Hallöchen!

Gunter Königsmann writes:

> Would that help with cellphone lenses that tend to have a
> different scale in the top left corner than in the lower right
> one?

No. It is all about centrosymmetric aberrations here.

[klaus...@gmail.com]

Hi all,

Thanks for the replies so far. I'll upload some photos in due course. But 1st I need to fetch suitable photographic material back at home, and 2nd I want to try to stitch a 360 deg panoramic as this better constrains the FOV fit parameter.

As the JPGs have filtering artifacts, I'll supply some DNGs as well. And of course I'll look into different parameter set myself as well.

Best regards Klaus

[klaus...@gmail.com]

Hallo Torsten,

I found an old thread on the panotools wiki.

https://wiki.panotools.org/User:Klaus/Improving_Hugin

As long as there is no hard vignetting in the lens, the reasoning should work.

Best regards
Klaus

[Torsten Bronger]

Hallöchen!

klaus...@gmail.com writes:

> I found an old thread on the panotools wiki.
>
> https://wiki.panotools.org/User:Klaus/Improving_Hugin

I find the language hard to understand, sorry. Anyway … what is the
argument against even exponents?

> As long as there is no hard vignetting in the lens, the reasoning
> should work.

How does the vignetting affect distortion?

[Erik Krause]

Am 24.07.2018 um 21:01 schrieb klaus...@gmail.com:

> I found an old thread on the panotools wiki.
>
> https://wiki.panotools.org/User:Klaus/Improving_Hugin

There probably is one problem we didn't consider then: Higher order
polynomials tend to overfit. That might be the reason why professor
Dersch choose the current model (he's a mathematician after all). At
least that's the opinion of Joost, the maker of PTGui (where this
discussion pops up from time to time, too).

There's nothing gained if we have a "correct" lens distortion model that
can't be safely optimized...

[Erik Krause]

Am 23.07.2018 um 17:26 schrieb klaus...@gmail.com:

> Non-zero parameters a and c introduce singularities at r=0, something a real lens does not have.

Since the correction function itself doesn't have a singularity at r=0
and all that changes at r=0 is the slope of the curve there is a change
in magnification only in the center. Since a single mathematical point
can't be magnified there is no singularity either. The point in the
center stays in the center.

In the wiki discussion you write "sqrt() has a singularity at 0". This
is not true. sqrt(0) is 0

[klaus...@gmail.com]

Hello,

I am using a free service. Downside is that the 2.5 GB will expire there after 48 hours. Files from a Parrot Anafi drone flight are here:

https://mab.to/baXC9ntbt

Feel free to upload some or all files to more permanent storage.

Best regards
Klaus

[klaus...@gmail.com]

Hallo,

Am Dienstag, 24. Juli 2018 21:56:16 UTC+2 schrieb Torsten Bronger:

Hallöchen!

klaus...@gmail.com writes:

> I found an old thread on the panotools wiki.
>
> https://wiki.panotools.org/User:Klaus/Improving_Hugin

I find the language hard to understand, sorry.  Anyway … what is the
argument against even exponents?

They introduce a singularity at r=0 (due to the branching point of sqrt(x=0) for the two-dimensional mapping function.

This singularity causes a rather small radius of convergence.

The problem is not the presence of even exponents. If there were enough odd exponents, and I think one or two more would make a difference, the fit would choose coefficient values close to zero for the coefficients of the even exponents.

 

> As long as there is no hard vignetting in the lens, the reasoning
> should work.

How does the vignetting affect distortion?

Central to my reasoning is that the lens mapping function is holomorphic over the entire image area.

Starting with geometric optics, ray tracing through a set of lenses gives an infinitely differentiable function

for every ray specified by start point and direction. This property is conserved if one selects rays passing through an aperture

in one plane and uses the averaged arrival positions. Here it is key that integration boundary is not a function of ray angle, or

in other words, a function of arrival point on the image sensor. When mechanical vignetting occurs, the imaging function is no longer

holomorphic at the boundary of the onset, that means that (higher) derivatives will no longer be continuous there.

Sorry, that is mathematics heavy, needing a background in complex functions of two variables.

But it has practical implications.

[klaus...@gmail.com]

Am Dienstag, 24. Juli 2018 22:37:35 UTC+2 schrieb Erik Krause:

Am 23.07.2018 um 17:26 schrieb klaus...@gmail.com:

> Non-zero parameters a and c introduce singularities at r=0, something a real lens does not have.

Since the correction function itself doesn't have a singularity at r=0

I insist to disagree.

You are talking about a polynomial that covers the radial part in polar coordinates description. Polynomials do not have singularities, yes, in that you are right.

But we have to consider a two-dimensional mapping function. Using polar coordinates, it factors, this is nice, the phi mapping is identity, even nicer.

But polar corrdinates are slightly peculiar at vector-r=0, and to have them differentiable there the function in r must be odd.

(More general, odd f(r) for f(r)*sin(2n*phi+phi0) and even f(r) for f(r)*sin((2n+1)*phi+phi0)

As the two-dimensional lens function is differentiable in vector-r=0, introducing non-differentiable functions is not a good idea.

and all that changes at r=0 is the slope of the curve there is a change
in magnification only in the center. Since a single mathematical point
can't be magnified there is no singularity either. The point in the
center stays in the center.

Please compute, in cartesian coordinates, the partial derivative d^2/dxdx of the 2-dim mapping function with c=1 as parameter.

Hint: you'll encounter some x/abs(x) - like terms.

 

 

In the wiki discussion you write "sqrt() has a singularity at 0". This
is not true. sqrt(0) is 0

In simple language: singularities can be present even if the function is continuous.

Best regards

Klaus

[Erik Krause]

Am 25.07.2018 um 16:38 schrieb klaus...@gmail.com:
> Please compute, in cartesian coordinates, the partial derivative d^2/dxdx
> of the 2-dim mapping function with c=1 as parameter.
> Hint: you'll encounter some x/abs(x) - like terms.
>
>
>
>> In the wiki discussion you write "sqrt() has a singularity at 0". This
>> is not true. sqrt(0) is 0
>>
> In simple language: singularities can be present even if the function is
> continuous.

And how would this affect the image? For no correction at all the
function is linear with a slope of 45°. If there are a, b and c
parameters the d parameter is adjusted such that the curve always goes
through (1,1). For some parameter values it is not 45° at (0,0), but
this results in a different magnification in the center, like to be
expected if you want to correct pincushion or barrel distortion.

My math is too rusty to argue with you, but I would be interested
whether you can estimate how large the error would be? Will it be as
large or larger than the error f.e. introduced by parallax due to
entrance pupil shift (which is pronounced for fisheye lenses but also
present for rectilinear wide angles).

--
Erik Krause

---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus

[klaus...@gmail.com]

Hallo,

Am Dienstag, 24. Juli 2018 22:13:04 UTC+2 schrieb Erik Krause:

Am 24.07.2018 um 21:01 schrieb klaus...@gmail.com:

> I found an old thread on the panotools wiki.
>
> https://wiki.panotools.org/User:Klaus/Improving_Hugin

There probably is one problem we didn't consider then: Higher order
polynomials tend to overfit.

Yes, this is a problem in principle. But if one uses enough CPs for a given number of images, a good part more than the degrees of freedom / fitting parameters, then the fit is overconstrained and overfitting is not an issue. Of course make sure the CPs are reasonably spread over the image areas.

That might be the reason why professor
Dersch choose the current model (he's a mathematician after all).

Side informations:

as a physicist I've also been rigorously trained in mathematics.

Professorship is no vaccine against getting something wrong ;-)

 

With due respect for Helmut Dersch and his pioneering work in panoramics, devising and coding some sterling software along the way, I think he just overlooked the fact that his parametrisation has function terms that are non-holomorphic at vector-r = zero (maybe some slight prejudice towards Fachhochschule there ...) Or maybe he did not care; a and c do give some small improvement, and being out of alignment by 3 or 5 pixels in the image corners is no big deal if your image seam runs elsewhere.

It is interesting, that NASA people (do not find the reference right now, either for star or for rocket tracking) have also calibrated their lenses with multiple images. They use only odd polynomials. But in the paper they do not give a reason for that. Maybe it seemed obvious to them....

 

At least that's the opinion of Joost, the maker of PTGui (where this
discussion pops up from time to time, too).

There's nothing gained if we have a "correct" lens distortion model that
can't be safely optimized...

I do think that two more parameters, for r^5 and r^7 (if one elects to keep a and c switched on as fit parameters) do not create optimisation instability (of course start optimisation without barrel distortion) when there are typically a few dozen optimisation parameters.

Of course the proof lies in the puddin. Let's give it a try.

Best regards

Klaus

P.S. What amazed me, years ago, take a rectangular lens model r~tan phi, add r^3 and r^5 and r^7 terms, and you can get a near-perfect description of a fish-eye lens with its r~phi property.

[klaus...@gmail.com]

Hello Erik,

Am Mittwoch, 25. Juli 2018 17:15:19 UTC+2 schrieb Erik Krause:

Am 25.07.2018 um 16:38 schrieb klaus...@gmail.com:
> Please compute, in cartesian coordinates, the partial derivative d^2/dxdx
> of the 2-dim mapping function with c=1 as parameter.
> Hint: you'll encounter some x/abs(x) - like terms.
>  
>  
>
>> In the wiki discussion you write "sqrt() has a singularity at 0". This
>> is not true. sqrt(0) is 0
>>
> In simple language: singularities can be present even if the function is
> continuous.

And how would this affect the image? For no correction at all the
function is linear with a slope of 45°. If there are a, b and c
parameters the d parameter is adjusted such that the curve always goes
through (1,1). For some parameter values it is not 45° at (0,0), but
this results in a different magnification in the center, like to be
expected if you want to correct pincushion or barrel distortion.

The main point is that when you overlay remapped images, they will not register correctly.

There are areas that work well, and other areas where image content is offset.

If you do a hard blend at the line segment bisector (Mittelsenkrechte), lines across it will not have a step but will have a kink.

My math is too rusty to argue with you, but I would be interested
whether you can estimate how large the error would be?

That depends on the lens, and also on the zoom setting. My observation is that for one of my old cameras when zooming the a and c parameters changed sign. I also observed that in such a case using only the barrel b parameter gave CP errors a few 1/10th pixels in size.

What intrigued me, after optimisation with b, then switching on a and c, alignment for CPs near the image center improved, but a good (sharp, well-defined) CP near the image edge or corner worsened in alignment from 3 to 5 pixels for instance.

 

Will it be as
large or larger than the error f.e. introduced by parallax due to
entrance pupil shift (which is pronounced for fisheye lenses but also
present for rectilinear wide angles).

That depends on object distances. You tell me real world values, but let's assume 1 cm entrance pupil shift. For an object in 10 m distance that is 1 mrad.

Now I assume a lens with one rad or 57.3 degrees opening angle. Say 4000 pixels hence a shift of 4 pixels.

A fisheye is more like 3 radians. For simplicity 3000 pixels. Shift of 1 pixel.

Adjust numbers to your situation.

 

Parallax due to entrance pupil shift, you can compensate to some extend in repositioning your camera on your nodal point adapter.

Parallax errors due to objects at different distances, the seem finder in enblend does a quite good job in putting seems away from the problem zones.

Best regards

Klaus

[klaus...@gmail.com]

Hallo,

Am Mittwoch, 25. Juli 2018 15:05:30 UTC+2 schrieb klaus...@gmail.com:

Hello,

I am using a free service. Downside is that the 2.5 GB will expire there after 48 hours. Files from a Parrot Anafi drone flight are here:

https://mab.to/baXC9ntbt

A few observations:

1) On import the lens is considered a rectangular lens. When optimising with barrel, b becomes quite large, and the Fast Preview turns into triangles.

2) Normal preview or declare lens full frame fisheye: when switching on and off images the image content moves visibly.

3) Looking at the vis-*.tif files, there is a dark path in the middle, and white misaligned parts on either side. To get a reasonably good panoramic image without line steps, one has to set exclusion paths such that the blending seam runs down the middle of the black area (done with image material from a week ago).

Cheers

Klaus

[Torsten Bronger]

Hallöchen!

klaus...@gmail.com writes:

> [...]

>
> But polar corrdinates are slightly peculiar at vector-r=0, and to
> have them differentiable there the function in r must be odd.
> (More general, odd f(r) for f(r)*sin(2n*phi+phi0) and even f(r)
> for f(r)*sin((2n+1)*phi+phi0)

I agree that infinite differentiability has strong implications, and
vanishing even exponents in an approximation may be one of them.

Complex differentiability is even stronger.

However I wonder: What is the physical reason for these hefty
requirements? I can understand continuous, but why must the mapping
of rays of light be holomorphic?

Let me play the devil's advocate. Let sgn(r)*r**2 be the mapping
function of a lens. Granted, it is not infinitely differentiable at
r=0. But it is continuous, and odd. What hinders the production of
such a lens?

That said, we do observe in the Lensfun lens database that a and c
are strongly damped. b (the only odd coefficient) mostly has the
biggest absolute value.

[klaus...@gmail.com]

Why holomorphic?

Because a spherical lens surface parametrised as z(x,y) is holomorphic.
Then light refraction according to Snell's law also gives holomorphic funktions for the rays. All this if course has a finite convergence radius.

Aspherical corrections are usually even polynomials and hence keep the lens surface holomorphic.

Of course a random lens surface (Flaschenboden) is not holomorphic let alone rotationally symmetric.

Now my point is not to throw out parameters a and c. Leave them in for backward compability. My query is to add two more odd terms. My prediction is that CP errors will go down significantly.

Fine-tuned CPs are good to about 1/10th of a pixel. This is what I found for my cameras in the past.

Best regards
Klaus

[Erik Krause]

Am 25.07.2018 um 15:05 schrieb klaus...@gmail.com:
> I am using a free service. Downside is that the 2.5 GB will expire there after 48 hours. Files from a Parrot Anafi drone flight are here:
>
> https://mab.to/baXC9ntbt

It would have been easier if you included only the images that belong to
the panorama. At least 1/3 of the jpg images don't.

As was to be expected for an aerial panorama the images suffer from
severe parallax. They can't be used to say anything about lens
correction. Lens correction optimization is only possible, if the camera
and lens was rotated exactly around the entrance pupil of the lens, the
no-parallax-point.

If the images don't align it's not the lens correction model which is to
blame, it's parallax! Even worse: As soon as there is parallax the
optimizer will try to use lens correction parameters to minimize control
point distances for points that are misaligned due to parallax.

Nevertheless it should be possible to stitch a nice panorama, you have
plenty of images with large overlap and good features to choose from in
order to avoid alignment problems.

The lens should be treated as a full frame fisheye, though. Otherwise
the b parameter gets too high.

[klaus...@gmail.com]

Re your sgn(r)*r**2 mapping function. You can realise with a small aperture, then a lens with r^2 and r^3 terms which maps your incoming theta onto sgn(theta)×theta^2 and last a lens converting direction to position.

It is doable, even on a turntable. But you'll get sizeable imaging errors as soon as you move away from your pinhole aperture. Coma and astigmatism for sure.

[klaus...@gmail.com]

Hi Erik,

Thank you for looking into it. While I was sure the drone drifted barely, and as the photos are taken from 100m height I cannot prove that it is not parallax.

What I can do is to take some photos while not flying. It will not be a 360 degrees panoramic but more like 120 to 150 degs, however I can control the nodal point to a few mm and typical objects are 100+ metres away.

Best regards
Klaus

[Erik Krause]

Am 25.07.2018 um 22:47 schrieb klaus...@gmail.com:
> Thank you for looking into it. While I was sure the drone drifted
> barely, and as the photos are taken from 100m height I cannot prove
> that it is not parallax.

The parallax is clearly visible. See the roof of the house move against
the background: http://erik-krause.de/parallax-foehl.gif

> What I can do is to take some photos while not flying. It will not be
> a 360 degrees panoramic but more like 120 to 150 degs, however I can
> control the nodal point to a few mm and typical objects are 100+
> metres away.

That would certainly help.

[Erik Krause]

Am 25.07.2018 um 23:09 schrieb Erik Krause:
>> What I can do is to take some photos while not flying. It will not be
>> a 360 degrees panoramic but more like 120 to 150 degs, however I can
>> control the nodal point to a few mm and typical objects are 100+
>> metres away.
> That would certainly help.

... provided you use the entrance pupil and not the nodal point ;-)

[klaus...@gmail.com]

Am Mittwoch, 25. Juli 2018 23:12:07 UTC+2 schrieb Erik Krause:

Am 25.07.2018 um 23:09 schrieb Erik Krause:
>> What I can do is to take some photos while not flying. It will not be
>> a 360 degrees panoramic but more like 120 to 150 degs, however I can
>> control the nodal point to a few mm and typical objects are 100+
>> metres away.
> That would certainly help.

... provided you use the entrance pupil and not the nodal point ;-)

 

https://mab.to/DNe8WRQgF

Entrance pupil positions should not be more than 5 millimetres apart.

[klaus...@gmail.com]

Am Mittwoch, 25. Juli 2018 23:09:40 UTC+2 schrieb Erik Krause:

...

The parallax is clearly visible. See the roof of the house move against
the background: http://erik-krause.de/parallax-foehl.gif

Indeed. And I see that the shadow has moved as well.

I have already looked into the 5 images of https://www.myairbridge.com/en/#!/link/DNe8WRQgF that should not suffer from parallax errors. Hugin tells me it is 171 degrees hfov, so no constraint by a full 360 degrees ring of images. And because the JPGs suffer from filtering and sharpening which could cause CP position jitter, I started from DNG using dcraw. Images are less sharp and suffer from colour noise.

No CPs in the images top half where there is blue sky. Hence image misalignment in this top half would not be noticed.

Using hugin CPfind, then fine-tuning all point and keeping only points with quality >= .80; also adding a few CPs manually near the horizon.

Lens type full frame fisheye. Using v(hfov) and b only, alignment is awful. No improvement when adding d and e.

Adding parameters a and c into the fit improves things drastically. Now I remove 5 CP outliers from the ~200 CPs.

Looking at the vis images they look quite fine. But the situation is more benign here, in that the images have 60-70 percent overlap.

Looking at the CP distances I find that they are smaller for adjacent images than non-adjacent images.

This I think is some hint that the lens parametrisation is not yet optimum.

Then I removed every other image from the project. On re-aligning I find that the d and e parameter values do change. Not much but noticeably.

i ponder to think that e value of about 30 hints to some possible improvement which is possible because CPs are unevenly distributed in y.

Looking at alignment in corners one sees some images moving by several pixels. Visible in the vis files, visible in the preview window.

One also sees misalignments for horizontal features in the vis files.

My conclusion for now, as a workaround, take image series with sizeable overlap. At least 50%, better something like 60% to 70%.

Best regards

Klaus

[klaus...@gmail.com]

Hallo,

This difference image shows the misalignment near the corner:

http://www.foehl.net/tiff00010002.jpg

Best regards

Klaus

[Erik Krause]

Am 26.07.2018 um 13:32 schrieb klaus...@gmail.com:
> This difference image shows the misalignment near the corner:
> http://www.foehl.net/tiff00010002.jpg

This could as well be caused by parallax due to entrance pupil shift.
And it is hard to tell whether chromatic aberration plays a role with
this kind of red-green comparison. Better to use the green channel only.

[klaus...@gmail.com]

Am Donnerstag, 26. Juli 2018 13:42:40 UTC+2 schrieb Erik Krause:

Am 26.07.2018 um 13:32 schrieb klaus...@gmail.com:
> This difference image shows the misalignment near the corner:
> http://www.foehl.net/tiff00010002.jpg

This could as well be caused by parallax due to entrance pupil shift.

http://www.foehl.net/anafi-camera-1.jpg

Do not think so. The lens is tiny, a few mm across, and the displacement measures in dm.

 

And it is hard to tell whether chromatic aberration plays a role with
this kind of red-green comparison. Better to use the green channel only.

Sorry for not telling how the image was prepared:

Taking two images to be blended, turning them into greyscale, only then colouring them cyan and red. Adding the two images.

http://www.foehl.net/adding-cyan-and-red.jpg

This is how the whole thing looks at 8% scale.

 

Best regards

Klaus

[Erik Krause]

Am 26.07.2018 um 12:18 schrieb klaus...@gmail.com:

> https://mab.to/DNe8WRQgF
> Entrance pupil positions should not be more than 5 millimetres apart.

I managed to get this result:
http://erik-krause.de/align-foehl.gif

misalignment is in the magnitude of image blur in the corners. In the
very corners there are no control points possible. It might improve if
there where, but that's of no relevance for panorama stitching at all.
The corners should be cut anyway due to lens blur. So the panotools lens
correction model is able to align these images perfectly. BTW.: without
using a and c the result was much, much worse.

[bugbear]

I'm slightly confused by this thread; I have no reason (or qualification!)
to doubt the maths that Klaus has posted, but the examples
seem to be more like normal stitching difficulties than high-falutin'
mathematical ones.

If the fault is as fundamental as claimed, it ought to be quite glaring.

BugBear

[klaus...@gmail.com]

Hi bugbear,

As you notice, the mathematical lens model, despite its shortcoming, does a halfway decent job at about pixel or half pixel level. In a finished panoramic, you visually notice when images are misaligned at the seam line.

Hugin's sidekick enblend does a really good job in sorting out seam problems.

Barrel correction is mathematically ok. The extra freedom parameters a and c give you work if you spread your CPs out, or in problematic cases you position them nearby the seam line.

Issues do appear when you want to place your seam line away from the geometrical optimum placement. To avoid a car being cut in half for instance.

The limitations also make it necessary that hugin / panotools have to provide different lens models, as rectangular and fisheye lenses cannot be unified. With two more odd polynomials they could. It is amazing how well a Taylor series with four odd terms can approximate tan()...

I think I should do a proper write-up.
But not tonight.

And then there are real world problems like parallax errors, image noise and oversharpened source images. For the latter cases, when CPs become dodgy, hugin is good enough. It is in good cases that its limitations get apparent.

Best regards
Klaus

[Erik Krause]

Am 27.07.2018 um 22:55 schrieb klaus...@gmail.com:

> The limitations also make it necessary that hugin / panotools have to
> provide different lens models, as rectangular and fisheye lenses
> cannot be unified. With two more odd polynomials they could. It is
> amazing how well a Taylor series with four odd terms can approximate
> tan()...

The abc lens correction model is not used to correct fisheye distortion
directly. Instead it is used to correct the deviation from an ideal
fisheye. In original panotools this was the R = f * theta model only.
hugin added four other fisheye-like models, you can choose from the lens
type dropdown.

You criticized that the current formula is phenomenology only. But
that's what it's all about. Panorama stitching is about solving real
problems, not providing the best theoretical model. The current model
showed it's abilities by stitching millions of panoramas (literally) and
where it failed to do so properly it was due to other factors, not a
theoretically sub-optimal lens correction model.

Parallax is the most common problem, chromatic aberration and lens blur
are others. Not to speak from too little features for control points,
incapable control point generators and last but not least user inability.

As long as you don't provide an error estimation you only can start to
show that the lens correction model is insufficient if you ruled out
other reasons for alignment problems. First was parallax. Then I showed
that the current model can correct your images well into the corners,
provided you put enough control points there, even though there is heavy
lens blur. This shows that the model actually can correct your images,
if you feed it the right parameters.

Getting those parameters is the next problem. You probably understand
better than me how the optimizer works and why it sometimes doesn't get
the correct parameters, although there are seemingly enough control
points. A more sophisticated model would suffer from this problem
probably to a heavier extent due to overfitting.

While under ideal conditions it might be possible to place enough points
throughout the overlap to satisfy a sophisticated lens correction this
is not possible for most real world panoramas. The optimizer must be
still able to generate reliable results, even if there are insufficient
points. I guess, Helmut Dersch choose this model because it generates
stable optimizer results (you can ask him, he's still active:
https://webuser.hs-furtwangen.de/~dersch/ ) BTW.: Furtwangen is a true
university (Hochschule) now, it was a Fachhochschule years ago.

As far as I know Autopano Pro uses a lens correction model like you
propose. But Autopano is not famous for it's better stitched panoramas.
Both other big players (hugin and PTGui) use the panotools model and are
equally successful.

With all that one shot solutions (theta, gear360 etc) and drone cameras
panorama shooting has reached the main stream. The big problem there is
not lens correction, but unavoidable parallax. This is the next hurdle
panorama stitching needs to take, not lens correction.

[bugbear]

klaus...@gmail.com wrote:
> For the latter cases, when CPs become dodgy, hugin is good enough. It is in good cases that its limitations get apparent.

Have you actually encountered these limitations? - none of your examples are
anywhere near these limits; their "realworld" problems greatly
exceed the theoretical limitations you've described.

BugBear

[klaus...@gmail.com]

Hello again, after several weeks of work-related absence...

Prompted by someone mentioning LCP files and the lens model Adobe implements in their products,

http://download.macromedia.com/pub/labs/lensprofile_creator/lensprofile_creator_cameramodel.pdf

I have done some internet searches. Interestingly, after some original papers, I discovered that wikipedia has nice write-ups. A few hooks:

Findings: hugin parameters a and c have to be zero.

https://en.wikipedia.org/wiki/Optical_aberration

cited text in there: "These degrees, named by J. Petzval (Bericht uber die Ergebnisse einiger dioptrischer Untersuchungen, Buda Pesth, 1843; Akad. Sitzber., Wien, 1857, vols. xxiv. xxvi.) the numerical orders of the image, are consequently only odd powers; ..."

This is also the result of a rigorous wavefront treatment when the lens is centered ond rotationally symmetric. Seidel-Aberration.

https://fr.wikipedia.org/wiki/Distorsion_(optique)

The Brown–Conrady model (see wikipedia and references therein) is pretty much at the base of the Adobe model and its parameters.

In addition to radial distortion coefficients it also includes tangential distortion coefficients.

https://en.wikipedia.org/wiki/Distortion_(optics)

The Brown–Conrady parametrisation being also adopted by the USGS.

"The New Camera Calibration System at the U. S. Geological Survey" https://www.asprs.org/wp-content/uploads/pers/1992journal/feb/1992_feb_185-188.pdf

Interesting reading makes a paper from

de Villiers, J. P.; Leuschner, F.W.; Geldenhuys, R. (17–19 November 2008). "Centi-pixel accurate real-time inverse distortion correction"

With barrel parameter only you get about 0.3 pixel accuracy, go further and one gets 0.07 pixel accuracy. Hugin is currently at the 0.3 level only.

Hence I advocate to include more parameters from the Brown–Conrady model.

Currently only K1 is there in form of the ptx b parameter.

To include at the very least parameter K2 into the hugin lens model.

Reasonably I suggest parameters K2 and K3 plus tangential coefficients P1 and P2.

Best regards

Klaus