Unable to initialize camera parameters

681 views

Skip to first unread message

Cenek Albl

unread,

Sep 18, 2016, 11:25:38 AM9/18/16

to kalibr-users

Hi,

first of all, thank you very much for making this software freely available!

I am attempting to calibrate 3 cameras (Microsoft webcams) for an experiment. The cameras don't have large radial distorion.

I generated 6x10 april calibration board using your tool and captured images. I don't use ROS, I just grabbed the images using MATLAB, always all 3 cameras approximately at once (slight time delay). Below is a sample image.

I converted to grayscale and assigned timestamps as the image file names. I've already figured out that the timestamps matter even if i am not synchronizing with IMU and that they have to be in certain range otherwise the process fails.

Therefore, I generated some "meaningful" timestamps, e.g. 913375858139019392 with +1e9 for each image (1s). This allowed the software to continue and extract the corners.

However, for the first 300 images, I obtained very bad calibration (reprojections errors ~5-10 pixels) for which so I captured another 200 images being careful not to move the target during the capture to avoid desync. For this dataset I can't even get the cameras initialized. One of them is initialized to values that are extremely off and the other 2 get NaNs for everything. As far as I can see from the debug images, most of the corners are detected in most of the images and they seem to be correct.

Also, the example provided (static.bag) works fine for me, so Kalibr seems to be wroking properly and there is probably problem with my data.

Could someone please help me figure out what could be wrong?

I am using the following command to invoke the calibration:

./cde-exec kalibr_calibrate_cameras --bag take3.bag --topics /cam0/image_raw /cam1/image_raw /cam2/image_raw --models pinhole-equi pinhole-equi pinhole-equi --target april.yaml

My april.yaml file:

target_type: 'aprilgrid' #gridtype

tagCols: 10 #number of apriltags

tagRows: 6 #number of apriltags

tagSize: 0.02 #size of apriltag, edge to edge [m]

tagSpacing: 0.333 #ratio of space between tags to tagSize

And here is the console log (actually in this run, camera 3 got also initialized and with correct values!):

importing libraries

the rosdep view is empty: call 'sudo rosdep init' and 'rosdep update'

Initializing cam0:

Camera model: pinhole-equi

Dataset: take3.bag

Topic: /cam0/image_raw

Number of images: 200

Extracting calibration target corners

Extracted corners for 200 images (of 200 images)

Projection initialized to: [ 3876.43254791 3544.10329999 638.47835145 219.64099691]

Distortion initialized to: [ 2.85485513 107.99987802 -12.28638348 -0.47947806]

Initializing cam1:

Camera model: pinhole-equi

Dataset: take3.bag

Topic: /cam1/image_raw

Number of images: 200

Extracting calibration target corners

Extracted corners for 196 images (of 200 images)

Projection initialized to: [ nan nan nan nan]

Distortion initialized to: [ nan nan nan nan]

Initializing cam2:

Camera model: pinhole-equi

Dataset: take3.bag

Topic: /cam2/image_raw

Number of images: 200

Extracting calibration target corners

Extracted corners for 195 images (of 200 images)

Projection initialized to: [ 1190.42972136 1191.07583635 645.32190562 523.87285455]

Distortion initialized to: [ 0.25240163 0.11529385 -0.04565713 0.0284536 ]

initializing initial guesses

initializing camera pair (0,1)...

initializing camera pair (0,2)...

initialized baseline between cam0 and cam1 to:

[[ nan nan nan nan]

[ nan nan nan nan]

[ 0. 0. 0. 1.]]

initialized baseline between cam1 and cam2 to:

[[ nan nan nan nan]

[ nan nan nan nan]

[ 0. 0. 0. 1.]]

initialized cam0 to:

projection cam0: [ nan nan nan nan]

distortion cam0: [ nan nan nan nan]

initialized cam1 to:

projection cam1: [ nan nan nan nan]

distortion cam1: [ nan nan nan nan]

initialized cam2 to:

projection cam2: [ nan nan nan nan]

distortion cam2: [ nan nan nan nan]

initializing calibrator

starting calibration...

Progress 25 / 200 Time remaining: 9s

------------------------------------------------------------------

Processed 26 of 200 views with 1 views used

Camera-system parameters:

cam0 (/cam0/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam1 (/cam1/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam2 (/cam2/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

baseline T_1_0:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

baseline T_2_1:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

------------------------------------------------------------------

Progress 50 / 200 Time remaining: 7s

------------------------------------------------------------------

Processed 51 of 200 views with 1 views used

Camera-system parameters:

cam0 (/cam0/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam1 (/cam1/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam2 (/cam2/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

baseline T_1_0:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

baseline T_2_1:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

------------------------------------------------------------------

Progress 75 / 200 Time remaining: 5s

------------------------------------------------------------------

Processed 76 of 200 views with 1 views used

Camera-system parameters:

cam0 (/cam0/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam1 (/cam1/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam2 (/cam2/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

baseline T_1_0:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

baseline T_2_1:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

------------------------------------------------------------------

Progress 100 / 200 Time remaining: 4s

------------------------------------------------------------------

Processed 101 of 200 views with 1 views used

Camera-system parameters:

cam0 (/cam0/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam1 (/cam1/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam2 (/cam2/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

baseline T_1_0:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

baseline T_2_1:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

------------------------------------------------------------------

Progress 125 / 200 Time remaining: 3s

------------------------------------------------------------------

Processed 126 of 200 views with 1 views used

Camera-system parameters:

cam0 (/cam0/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam1 (/cam1/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam2 (/cam2/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

baseline T_1_0:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

baseline T_2_1:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

------------------------------------------------------------------

Progress 150 / 200 Time remaining: 2s

------------------------------------------------------------------

Processed 151 of 200 views with 1 views used

Camera-system parameters:

cam0 (/cam0/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam1 (/cam1/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam2 (/cam2/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

baseline T_1_0:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

baseline T_2_1:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

------------------------------------------------------------------

Progress 175 / 200 Time remaining: 1s

------------------------------------------------------------------

Processed 176 of 200 views with 1 views used

Camera-system parameters:

cam0 (/cam0/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam1 (/cam1/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam2 (/cam2/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

baseline T_1_0:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

baseline T_2_1:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

------------------------------------------------------------------

Progress 200 / 200 Time remaining:

..................................................................

Calibration complete.

[ WARN] [1474211916.487988]: Removed 0 outlier corners.

Processed 200 images with 1 images used

Camera-system parameters:

cam0 (/cam0/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam1 (/cam1/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

cam2 (/cam2/image_raw):

type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

distortion: [ nan nan nan nan] +- [ nan nan nan nan]

projection: [ nan nan nan nan] +- [ nan nan nan nan]

reprojection error: [nan, nan] +- [nan, nan]

baseline T_1_0:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

baseline T_2_1:

q: [ nan nan nan nan] +- [ nan nan nan]

t: [ nan nan nan] +- [ nan nan nan]

Results written to file: camchain-take3.yaml

Detailed results written to file: results-cam-take3.txt

Traceback (most recent call last):

File "/var/kalibr-build/devel/bin/kalibr_calibrate_cameras", line 5, in <module>

exec(fh.read())

File "<string>", line 444, in <module>

File "<string>", line 418, in main

File "/var/kalibr-build/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_camera_calibration/CameraCalibrator.py", line 748, in generateReport

plotPolarError(cself, cidx, fno=f.number, noShow=True, title=title)

File "/var/kalibr-build/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_camera_calibration/CameraCalibrator.py", line 444, in plotPolarError

pl.hist(sae[:,0])

File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 2341, in hist

ret = ax.hist(x, bins, range, normed, weights, cumulative, bottom, histtype, align, orientation, rwidth, log, color, label, **kwargs)

File "/usr/lib/pymodules/python2.7/matplotlib/axes.py", line 7736, in hist

m, bins = np.histogram(x[i], bins, weights=w[i], **hist_kwargs)

File "/usr/lib/python2.7/dist-packages/numpy/lib/function_base.py", line 158, in histogram

'max must be larger than min in range parameter.')

AttributeError: max must be larger than min in range parameter.

Error in sys.excepthook:

Traceback (most recent call last):

File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 64, in apport_excepthook

from apport.fileutils import likely_packaged, get_recent_crashes

ImportError: No module named apport.fileutils

Original exception was:

Traceback (most recent call last):

File "/var/kalibr-build/devel/bin/kalibr_calibrate_cameras", line 5, in <module>

exec(fh.read())

File "<string>", line 444, in <module>

File "<string>", line 418, in main

File "/var/kalibr-build/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_camera_calibration/CameraCalibrator.py", line 748, in generateReport

plotPolarError(cself, cidx, fno=f.number, noShow=True, title=title)

File "/var/kalibr-build/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_camera_calibration/CameraCalibrator.py", line 444, in plotPolarError

pl.hist(sae[:,0])

File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 2341, in hist

ret = ax.hist(x, bins, range, normed, weights, cumulative, bottom, histtype, align, orientation, rwidth, log, color, label, **kwargs)

File "/usr/lib/pymodules/python2.7/matplotlib/axes.py", line 7736, in hist

m, bins = np.histogram(x[i], bins, weights=w[i], **hist_kwargs)

File "/usr/lib/python2.7/dist-packages/numpy/lib/function_base.py", line 158, in histogram

'max must be larger than min in range parameter.')

AttributeError: max must be larger than min in range parameter.

joern

unread,

Sep 20, 2016, 10:51:29 AM9/20/16

to kalibr-users

There is clearly something going pretty wrong, since the framework is not even able to find a meaningful initial guess.
Whenever I worked with kalibr, initialization has not been an issue, so I suspect either spurious detections or a serious violation of the assumptions of the camera model by the device. Could you please a) try a different target (for example fewer but larger April tags or a checkerboard) and b) detail a little on the webcams? Do these have auto focus? I have never tested auto focus cameras, since estimating a constant model does not seem very sensible for them, so I lack any intuition as to whether this could cause the initialization to fail.