Calibration and the OpenCV camera model

[Kai Ritterbusch]

Hello all!

Sorry for being a bit off-topic, but I know that many people here use
the opencv camera model, for example with Nicolas' RGBDemo.

The Kinect has fx and fy values around 500 to 700. What's the unit of
fx/fy?

According to the OpenCV book, fx = F*sx, and sx is something like px/mm
or px/m, depending on what the user chooses.

When using the calibration.cpp example shipped with opencv, it is
possible to set the square size of the checkerboard in a user-defined
unit. Until yesterday, I thought that this would set the unit of the
fx / fy values in the camera matrix. But fx/fy don't change with
changing square-size.

I am sure I miss something quite obvious, but still I'm stuck since
yesterday.. Thanks in advance for help!

Kai

[nomad]

hi kay,
on the end of january i have some experiments with opencv and
calibrations:
https://github.com/freenomad/freenomad_vision/downloads
are : cvCalibKinect01.tar.gz
with this code i can make some calibrations incl. the outputfiles
have a look
hope thats useful.
regards nomad

[KaiR]

Hi nomad,

thanks for your answer. I am wondering how your fx,fy can be around 20
instead of 500? If your values are correct, it probably indicates that
there is a way to modify the unit of fx/fy after all.

[nomad]

hi kay,
on german,
darueber hab ich mir noch gar keine gedanken gemacht.
ist 1:1 aus dem buch und nur an die kinect gegebenheiten angepasst.
bei mir hat das besser funkstioniert als das vorgehen bei den rgbdemos
dort dauernd -> nicht erkannt, mit meinen code passiert das nicht

on english :-)) via google translate:
over it I still no thought. 1:1 from the book is and only to kinect
the conditions adapted.
with me that has better funkstioniert than that to proceed with
rgbdemos there continuously - >
not recognized, with my code that does not pass

regards nomad

[Mourad Boufarguine]

Hi Kai,

fx and fy are expressed in px. So their values do not depend on the metric used for the chessboard corners.

To project 3d point to an image plane of a camera, you need first to apply a geometric transformation (rotation and translation) which is called the extrinsic parameters of the camera. Then, the transformed point is projected into the image plane by dividing the x and y coordinates by the z coordinates. So at this point we have unitless 2D coordinates that will be multiplied by fx and fy and added to u0 and v0 (the central point) to get image coordinates in pixels.

When changing the metric used in the 3D coordinates of the chessboard coordinates, you change the extrinsic parameters and not the intrinsic ones.

Mourad

----

[Kai Ritterbusch]

Hi Mourad,

thank you for your answer. I checked it and you are right :-) Strange,
that the OpenCV book states something different (p.373).

If I dont care about another coordinate system, it should be possible to
use a coordinate system that uses the same origin and orientation as the
camera system, isn't it? Then, all I need would be the scaling
mm <-> px.

Can you tell me, where the second coordinate system's origin is, that
the opencv calibration procedure uses and calculates R and T for? (Is it
somehow object-fixed, but to what object, then? -> p. 379)

You are really helping me with your answer, thanks again.

Kai

[Mourad Boufarguine]

Hi Kai,

Usually when using a chessboard to calibrate a camera, we choose a corner of the chessbord as the origin of the "world" coordinate system . the plane of the chessbord is usually considered as the plane z = 0. Since the purpose of this is just to compute the intrinsic parameters, you can consider any world coordinate system you want. The choice of using one fixed to the chessboard is just to ease things. You choose a corner as an origin, and knowing the square size (in mm or m) you can get the 3D coordinates of all the corners (with z = 0 for all of them).

Once you have computed this parameters, you can consider any "world" coordinate system you want. If you choose to consider the camera coordinate system to be the world one, then the geometric transformation (rotation and translation) will be equal to identity. So, to get pixel coordinates, you need, as you said, to transform metric coordinates into pixels :

u = fx * x/z + u0;

v = fy * y/z + v0;

Sorry, i dont have the opencv book, so i cant comment on it :)

Mourad

----

[Kai Ritterbusch]

Hi Mourad,

>
>
> Usually when using a chessboard to calibrate a camera, we choose a
> corner of the chessbord as the origin of the "world" coordinate
> system . the plane of the chessbord is usually considered as the plane
> z = 0. Since the purpose of this is just to compute the intrinsic
> parameters, you can consider any world coordinate system you want. The
> choice of using one fixed to the chessboard is just to ease things.

Ok, that makes sense: Opencv returns a rotation and translation vector
for every view ( every chessboard) used during calibration.

> Once you have computed this parameters, you can consider any "world"
> coordinate system you want. If you choose to consider the camera
> coordinate system to be the world one, then the geometric
> transformation (rotation and translation) will be equal to identity.
> So, to get pixel coordinates, you need, as you said, to transform
> metric coordinates into pixels :
>

If I chose that simple way

>
> u = fx * x/z + u0;
> v = fy * y/z + v0;
>
>

, I still lack the conversion factor between px and mm. Is it possible
to retrieve it from the extrinsics provided by the calibration
procedure?

Or do I need to implement a second step where I take a picture of a
chessboard whose position (and R and t) is known to me, so that I can
calculate a conversion factor from those and the returned extrinsics?

Cheers,

Kai

[Mourad Boufarguine]

Hi Kai,

> u = fx * x/z + u0;
> v = fy * y/z + v0;
>
>

, I still lack the conversion factor between px and mm. Is it possible
to retrieve it from the extrinsics provided by the calibration
procedure?

Or do I need to implement a second step where I take a picture of a
chessboard whose position (and R and t) is known to me, so that I can
calculate a conversion factor from those and the returned extrinsics?

I think you are using cvCalibrateCamera2 ( http://opencv.willowgarage.com/documentation/camera_calibration_and_3d_reconstruction.html#calibratecamera2), right ?

it will retun the intrinsec parameters of the camera that are the A matrix (see the link) and the distorsion coeffs.

The A matrix already contains fx, fy, cx (or u0) and cy (or v0) :

 u = fx * x/z + cx;
 v = fy * y/z + cy;

 

x, y and z are in mm (or m) => x/z and y/z is unitless

fx and fy are in px

cx and cy are in px 

so you will get u and v in px.

I forgot to mention that these equations are correct if you neglect distorsion.

 Other point, hopefully opencv does contain a function to project a 3D point into an image : cvProjectPoints2 (http://opencv.willowgarage.com/documentation/camera_calibration_and_3d_reconstruction.html#projectpoints2). It will need (besides the 3d points to be projected) the camera (intrinsic) matrix A, and the extrinsec parameters : rotation and translation of the camera. Since, the camera and the world have the same coordinate system, the translation is (0,0,0) and the rotation (0,0,0) (Rodrigues representation for identity rotation). You can also give it the distorsion coeffs to get more accurate results.

Mourad

[Daniel]

Hi there,
the problem is that there is no fixed conversion factor between px and
mm. Notice that in the equation (u=fx*x/z+u0), (x/z) has no units.
This is because a two adjacent pixels could be points very close in
space if their Z coordinate is small, or very far apart if their Z
coordinate is large.

The intrinsics matrix (composed of fx,fy,u0,and v0) will give you a
ray on which the 3D point lies, so you could say that a pixel in 2D
image space is equivalent to a ray in 3D space. With traditional
projective cameras the rays are not parallel so the distance between
them varies with depth.

Hope it helps,
Daniel H.