Aspect ratio and coordinate system

[Torsten Bronger]

We have a problem in the LensFun project that also affects Hugin.  The coordinate system used for distortion and TCA correction is poorly designed, namely, r=1 is half-height.  By contrast for vignetting, r=1 is half-diagonal.  This is much better.

Why? Because the crop factor also refers to the diagonal.  This way, the aspect ratio of the image doesn't matter.

Contrete example of the problem: Sigma produces a given lens for different mounts.  LensFun contains calibration data, which somebody created on their FT camera (Panasonic, Olympus).  So, aspect ratio of the calibration pictures was 4:3.  Now, somebody with a Nikon owns this Sigma lens, too.  Aspect ratio 3:2.  LensFun scales the coordinates by the ratio of the crop factors.  But this only works if the aspect ratios are the same!  If this is not the case, the error in the scaling is sqrt((ar1^2 + 1)/(ar2^2 + 1)).  ar1,2 are the two aspect ratios.

There are many ways to solve this in LensFun.

1. Leave it this way.  After all, the error should occur seldomly, and most cameras have 3:2.  Moreover, we would not have to crawl through all compact cameras to find out which use 4:3.  Maybe 4:3 will become exinct some day anyway.

2. Convert all crop factors to an effective crop factor of a 3:2 sensor with the same height.  However, crop factors in LensFun are also used to determine which calibrations are eligible.  For 4:3 images, this means that the part of the image circle that they cover would be over-estimated (because they would appear to be wider than they actually were).  The new crop factors would be simply wrong (although calibration would work).

3. Introduce a new <aspect-ratio> tag in LensFun.

4. Convert the a, b, c coefficients for non-3:2 data to be correct for 3:2 images.  For polynomial models, this conversion is lossless except for rounding errors.

What would this mean for Hugin?

1. Using calibration data for an image with different aspect ratio would not work.  Well, it does not work currently either.

2. Hugin would have to adjust the crop factor for non-3:2 images and pass this effective crop factor to LensFun's routines.

3. Hugin would have to make use of the AspectRatio attribute and the aspect ratio of the image to scale the r values.

4. Hugin would have to use the aspect ratio of the image to scale the r values.  For 3:2 images, the scaling factor would be 1.

What do you think, what is the way to go?  I personally prefer (4) because it has the least impact on LensFun and its using programs, while delivering correct results in all cases.  In particular for 3:2 images with 3:2 calibration data, nothing would change (the by far most frequent case).

[Terry Duell]

On Mon, 18 Nov 2013 19:22:30 +1100, Torsten Bronger
<torsten...@gmail.com> wrote:

> We have a problem in the LensFun project that also affects Hugin. The
> coordinate system used for distortion and TCA correction is poorly
> designed

[snip]

I haven't seen any response to this.
I hope that there is some discussion going on off-list.

Cheers,
--
Regards,
Terry Duell

[Torsten Bronger]

Hall�chen!

No, but I prepared a patch against LensFun for my possibility (3)
(i.e. explicit aspect ratio field). It needs more testing, though.
LensFun's bug report is here:
http://developer.berlios.de/bugs/?func=detailbug&bug_id=19092&group_id=9034

Tsch�,
Torsten.

--
Torsten Bronger Jabber ID: torsten...@jabber.rwth-aachen.de
or http://bronger-jmp.appspot.com

[T. Modes]

Am Montag, 18. November 2013 09:22:30 UTC+1 schrieb Torsten Bronger:

Contrete example of the problem: Sigma produces a given lens for different mounts.  LensFun contains calibration data, which somebody created on their FT camera (Panasonic, Olympus).  So, aspect ratio of the calibration pictures was 4:3.  Now, somebody with a Nikon owns this Sigma lens, too.  Aspect ratio 3:2.  LensFun scales the coordinates by the ratio of the crop factors.  But this only works if the aspect ratios are the same!  If this is not the case, the error in the scaling is sqrt((ar1^2 + 1)/(ar2^2 + 1)).  ar1,2 are the two aspect ratios.

There are 2 functions to search for a lens in lensfun database

const lfLens **	FindLenses (const lfCamera *camera, const char *maker, const char *model, int sflags=0) const
const lfLens **	FindLenses (const lfLens *lens, int sflags=0) const

In the documentation to both function it states
<cite>
The algorithm will look for a lens with crop factor not less than of given camera, since the mathematical models of the lens can be incorrect for sensor sizes larger than the one used for calibration.
</cite>

So the example you have given should not be possible according to the lensfun documentation.

If lensfun would do this, it will produce bogus data. When the lens was calibrated for a crop factor of say 2, it uses only the central part of the optics and the data describe the central part of the lens.
When using the same lens on a camera with a smaller crop factor, a bigger circle is used. You can not derive from the central part calibrated with one camera to the outer part used with the other camera.
So it would only be possible from calibration data from smaller crop factor cameras to deduce the calibration data for higher crop factor camera. But with libpano distortion model this is also not possible the other way around, because of the normalization. So you would need to recalculate all factors and not only to scale the factors.

Lensfun should only return values for the lens when calibrated with the same crop factor.

<snip>

3. Introduce a new <aspect-ratio> tag in LensFun.

One more point to consider: Even when you stay inside one crop factor, there are two possibilities to achieve a different aspect ratio: a) cropping height or b) a multi-aspect sensor. This have not been considered in the proposal.
 

4. Convert the a, b, c coefficients for non-3:2 data to be correct for 3:2 images.  For polynomial models, this conversion is lossless except for rounding errors.

Are you sure? Or does the normalization in case of libpano distortion would require some more calculations and introduce more errors?

Before implementing such backward-incompatible changes and move all responsibility to the calling program all aspects should be considered and correctly documented.

[Torsten Bronger]

Hall�chen!

T. Modes writes:

> Am Montag, 18. November 2013 09:22:30 UTC+1 schrieb Torsten Bronger:
>

> [...]
>
> [...]

> In the documentation to both function it states
> <cite>
> The algorithm will look for a lens with crop factor not less than of given
> camera, since the mathematical models of the lens can be incorrect for
> sensor sizes larger than the one used for calibration.
> </cite>
>
> So the example you have given should not be possible according to
> the lensfun documentation.

Well, we have two issues here.

First, my example was erroneously swapped. I meant: Calibration was
made with APS-C (crop 1.5, aspect ratio 3:2), to-be-corrected image
was taken with Four-Thirds (2.0, 4:3).

Secondly, LensFun's documentation is wrong (even self-contradictory)
in the cited sentence. It was fixed to:

"The algorithm will look for a lens with crop factor not larger

than of given camera, since the mathematical models of the lens
can be incorrect for sensor sizes larger than the one used for
calibration."

> [...]

>
> So it would only be possible from calibration data from smaller
> crop factor cameras to deduce the calibration data for higher crop
> factor camera. But with libpano distortion model this is also not
> possible the other way around, because of the normalization. So
> you would need to recalculate all factors and not only to scale
> the factors.

I would have agreed till yesterday. I know that Adobe does more
than mere re-scaling of the "r" coordinate to get the sensor size
right, and even before this discussion here, I suspected a bug in
LensFun because of this.

However, LensFun behaves perfectly for cropped sensors, if the
calibration was done with a full-frame sensor. I can tell for sure
because LensFun now contains a testimage.py program which creates
distorted pictures for testing LensFun. The images are *exactly*
corrected in all cases. Since the distorting program works in the
coordinate system of the actual calibration sensor, cropping is
really simple here.

Besides, you can take a distorted picture, crop it manually in Gimp,
and let it be undistorted by LensFun with a camera with a
correspondingly higher crop factor -- works precisely.

I still have not understood why this works, though.

> [...]

>
> <snip>
>
>> 3. Introduce a new <aspect-ratio> tag in LensFun.
>
> One more point to consider: Even when you stay inside one crop
> factor, there are two possibilities to achieve a different aspect
> ratio: a) cropping height or b) a multi-aspect sensor. This have
> not been considered in the proposal.

The pair (crop factor, aspect ratio) must be correct. For
multi-aspect sensors, a virtual camera for each aspect ratio needs
to be defined in LensFun. The same is necessary for multi-crop
cameras (e.g. Nikon FX cameras in DX mode). The calling program
must pass the correct camera then.

We have now implemented the <aspect-ratio> tag, and a corresponding
attribute field in the lfLens class.