Defishing filters

[Edgar Bonet]

Hello group!

I am a new user of MathMap and new member of this group. I have sent
some filters to Mark Probst and he suggested I should post them in this
group. So here they are, together with some explanations for those
interested in the math.

My main concern was with remapping pictures taken with a fisheye: a
Nikon 10.5 mm DX on a D300. In the past I had used Hugin to do this, but
the problem is Hugin assumes all fisheyes give an equidistant mapping,
while mine (as most modern fisheyes, I beleive) uses an equisolid angle
projection [for details on fisheye mappings, see Wikipedia:Fisheye lens,
section on Mapping function]. So with Hugin my images were somewhat
undercorrected. When I discovered MathMap, I figured out it was the
perfect tool to get the mappings right!

So here is my first filter, to convert from equisolid angle to
stereographic. Stereographic is one of my favorite fisheye mappings
because it preserves the fisheye feeling while at the same time it gets
rid of local distorsions (it is a conformal mapping of the sphere, then
small details like people's faces in a group come out undistorted):

----------------------------------------------------------------
filter fish2stereographic(image in, float zoom: 0.7-1 (0.82))
f = 10.5/23.6*W;
theta = 2*atan(r/(2*f)/zoom);
in(ra:[2*f*sin(theta/2), a])
end
----------------------------------------------------------------

Two numbers are hard-coded in this filter:

- 10.5 is the focal length of my lens in mm
- 23.6 is the width of my sensor (Nikon DX) in mm

you will have to change them to fit your particular setup. The other
stuff here is:

- f is the focal length of the lens converted into image units
- theta is the polar angle, i.e. the angle between the object in the
scene and the optical axis of the lens, theta is given by the
formula for the stereographic projection
- 2*f*sin(theta/2) is where the object got projected in the original
image, according to the equisolid angle projection my lens claims to
provide
- the zoom parameter is just for convenience, it allows me to get more
field of view while preserving the original image size

Converting into rectilinear is almost the same, but less useful: unless
you crop substantially, this projection stretches the edges of the
image too much:

----------------------------------------------------------------
filter fish2rectilinear(image in, float zoom: 0.3-1 (0.5))
f = 10.5/23.6*W;
theta = atan(r/f/zoom);
in(ra:[2*f*sin(theta/2), a])
end
----------------------------------------------------------------

The conversion to cylindrical perspective is a little bit more complex.
There may be smarter ways to do it, but the following works for me
(backwards from cylindrical to equisolid angle):

- go from cylindrical to a spherical coordinate system where y is the
polar axis, coordinates are longitude and latitude
- go from this to an Euclidean coordinate system on the unit sphere
- go from this to another spherical system having z as the polar axis,
coordinates are theta (polar angle) and phi (azimuth)
- then from this to equisolid angle

So here is the code:

----------------------------------------------------------------
filter fish2cylindrical(image in, float zoom: 0.7-1 (0.94))
f = 10.5/23.6*W;
long = x/f/zoom;
lat = atan(y/f/zoom);
xx = cos(lat) * sin(long);
yy = sin(lat);
phi = atan(yy, xx);
theta = asin(sqrt(xx*xx+yy*yy));
in(ra:[2*f*sin(theta/2), phi])
end
----------------------------------------------------------------

In the above:

- (lat, long) are the longitude and latitude of the object
- (xx, yy, zz) are it's Euclidian coordinates on the unit sphere,
but actually zz is not needed
- theta and phi are the coordinates on the second spherical system

The conversion to Mercator is almost the same, only the formula for the
latitude is different:

----------------------------------------------------------------
filter fish2mercator(image in, float zoom: 0.7-1 (0.94))
f = 10.5/23.6*W;
long = x/f/zoom;
lat = atan(sinh(y/f/zoom));
xx = cos(lat) * sin(long);
yy = sin(lat);
phi = atan(yy, xx);
theta = asin(sqrt(xx*xx+yy*yy));
in(ra:[2*f*sin(theta/2), phi])
end
----------------------------------------------------------------

I prefer Mercator over cylindrcal. They look almost the same unless the
vertical field of view gets big, in which case cylindrical stretches
vertically the objects far from the equator. Mercator does not do that
since it is a conformal mapping, just like stereographic.

Generally my choice of projection is:
- Mercator if I want some kind of panoramic view (not too big a
vertical FoV) that preserves verticals
- stereographic if I want a more isotropic view that preserves the
fisheye feeling
The other two filters are here only for completeness.

Oh, I almos forgot! You may get equirectangular if you want. Same as the
two above but for:

lat = y/f/zoom;

And if you happen to have a fiheye with an equidistant mapping? Then
replace in all of the above, in the last instruction

2*f*sin(theta/2)

by

f*theta

Regards,

Edgar.

[photo...@gmail.com]

thank Edgar for sharing

[PKHG]

I am very sorry,
on Vista and Gimp 2.6.5 there seems to be a big problem with decimal
numbers
e.g f = 10.5/23.6*W;
I have to change into
f = 105/236*W;
But the default zoom is not done but some one color pictures is a
result of a picture.
Replacing zoom in theta = 2*atan(r/(2*f)/zoom);
bij some quotient of integers works ...

My working version looks like this:
filter fish2rectilinear(image in )

zoom =8/100;
f = 105/236*W;

theta = atan(r/f/zoom);
in(ra:[2*f*sin(theta/2), a])
end

My mathmap version is mathmap-1.2.4-win32

On 5 mrt, 18:26, Edgar Bonet <goo...@edgar-bonet.org> wrote:
> Hello group!
>

<snip>

> My main concern was with remapping pictures taken with a fisheye: a
> Nikon 10.5 mm DX on a D300. In the past I had used Hugin to do this, but
> the problem is Hugin assumes all fisheyes give an equidistant mapping,
> while mine (as most modern fisheyes, I beleive) uses an equisolid angle
> projection [for details on fisheye mappings, see Wikipedia:Fisheye lens,
> section on Mapping function]. So with Hugin my images were somewhat
> undercorrected. When I discovered MathMap, I figured out it was the
> perfect tool to get the mappings right!
>
> So here is my first filter, to convert from equisolid angle to
> stereographic. Stereographic is one of my favorite fisheye mappings
> because it preserves the fisheye feeling while at the same time it gets
> rid of local distorsions (it is a conformal mapping of the sphere, then
> small details like people's faces in a group come out undistorted):
>
>     ----------------------------------------------------------------
>     filter fish2stereographic(image in, float zoom: 0.7-1 (0.82))
>         f = 10.5/23.6*W;
>         theta = 2*atan(r/(2*f)/zoom);
>         in(ra:[2*f*sin(theta/2), a])
>     end
>     ----------------------------------------------------------------

<snip>
Edgar.

[photo...@gmail.com]

Your computer language is in english?
i mean it uses English format for number as 1,000.66? ..and not
1.000,56
If not will be problem in MM for all decimal numbers used.

since your name seems English may be something else, anyway is worth
to check the format used to display numbers

[photo...@gmail.com]

OOPSS i intended American format,
and the difference there is not 66 instead of 56 that was a typos
difference is in the use of commas and points

On Mar 11, 9:27 am, "photoco...@gmail.com" <photoco...@gmail.com>
wrote:

[s2sammy]

Hey Edgar could you make one of these up for the 8mm 3.5 Bower/Samyang/Pro-Optic Nikon mount left.

I will be using it on a Nikon1 J1, D70 and D7000 so is this complicated? 

I imagine the Nikon1 J1 with fake Ft1 adapter may be. Thanks!

[Edgar Bonet]

Hello!

s2sammy wrote:
> Hey Edgar could you make one of these up for the 8mm 3.5
> Bower/Samyang/Pro-Optic Nikon mount left. I will be using it on a Nikon1 J1,
> D70 and D7000 so is this complicated?

Well, it is complicated only insofar as you need to know the size of
your sensor and the projection formula of your particular fisheye
lens... All the defishing filters I posted start with:

f = 10.5/23.6*W;

where 10.5 is the focal length of my lens (in mm), and 23.5 the width of
my sensor. You have to replace those values to fit your lens and
sensor. The filters also end with:

in(ra:[2*f*sin(theta/2), phi])

maybe with "a" instead of "phi". You have to replace the formula

2*f*sin(theta/2)

with the projection of your lens. Actually the formula above is
incorrect: it is the right formula for an equisolid-angle fisheye, but
the 2.8/10.5 Nikon DX, contrary to what Nikon says, is not quite
equisolid-angle. It is best fit by

1.476*f*sin(.713*theta)

Anyway, concerning your cameras, Wikipedia gives the sensor widths as:

J1: 13.2 mm
D70: 23.7 mm
D7000: 23.6 mm

And for your lens, I could not find any data from Samyang, but Michel
Thoby has already measured the projection [1]. According to him:

the best fit that can be found when trying to match the Samyang
measured data against theoretical graphs of all lenses appears
to be y= 3*f*tan (theta/3) in association with f= 8.7 mm

Thus, you may try to start the filters with

f = 8.7/23.6*W;

(for the D7000 and D70, replace 23.6 by 13.2 for the J1). And end with

in(ra:[3*f*tan(theta/3), phi])

maybe with "a" instead of "phi".

Regards,

Edgar.

[1] http://michel.thoby.free.fr/SAMYANG/Early%20test%20report.html

[andrew...@gmail.com]

Hi,

I'm a little late  joining this 2009 thread, but I'm most interested in defishing an image, and I need
to know what the variable "W" is Edgar's post.

    filter fish2stereographic(image in, float zoom: 0.7-1 (0.82))
        f = 10.5/23.6*W;
        theta = 2*atan(r/(2*f)/zoom);
        in(ra:[2*f*sin(theta/2), a])
    end

Any ideas, anyone?

TIA,

Andrew

[Edgar Bonet]

Hi!

Andrew wrote:
> [...] I need to know what the variable "W" is Edgar's post.
> f = 10.5/23.6*W;

In this equation:

- 10.5 is the focal length of my lens, in mm (Nikon 10.5 Fisheye DX)
- 23.6 is the width of my sensor, in mm (Nikon DX format)
- W is the width of the image, it's a standard MathMap variable
- f is the focal length converted to MathMap units.

BTW, the equation I used here for r(theta) is not very good. It's the
standard equation for an equal-area projection. The 10.5 DX claims to be
equal area but it's not quite. It seems every fisheye is different, and
few follow a "standard" projection. Thus you should use an equation that
fits your particular lens. C.f. my followup from 2012-11-12.

Regards,

Edgar.