Problem with single imu - single cam extrinsic calibration
adam.str...@gmail.com
I am trying to run Kalibr Toolbox to calibrate the relative rotation and translation of the camera and imu in Google Pixel. I have recorded a calibration sequence and prepared all the necessary files - it can be viewed here: https://drive.google.com/drive/folders/1vgQEZtfHXTaIpiLiXLMKb_CfezB5XG6e?usp=sharing
I run the kalibr_calibrate_imu_camera cde code on virtualbox ubuntu with the following command:
../kalibr-cde/kalibr_calibrate_imu_camera --target checker.yaml --cam camchain.yaml --imu imu.yaml --bag ch_gentle.bag
And the output is:
importing libraries
Initializing IMUs:
Update rate: 400.0
Accelerometer:
Noise density: 0.003
Noise density (discrete): 0.06
Random walk: 0.0003
Gyroscope:
Noise density: 0.006
Noise density (discrete): 0.12
Random walk: 0.0006
Initializing imu rosbag dataset reader:
Dataset: /home/viki/workspace/chessboard_imu4/ch_gentle.bag
Topic: /imu0
Number of messages: 20239
Reading IMU data (/imu0)
Read 20239 imu readings over 51.5 seconds
Initializing calibration target:
Type: checkerboard
Rows
Count: 6
Distance: 0.05 [m]
Cols
Count: 9
Distance: 0.05 [m]
Initializing camera chain:
Camera chain - cam0:
Camera model: pinhole
Focal length: [642.406, 645.745]
Principal point: [315.463, 238.763]
Distortion model: radtan
Distortion coefficients: [0.27883956940720067, -2.2111427587870867, -0.0025311661150160614, -0.0010815551002964759]
baseline: no data available
Initializing camera rosbag dataset reader:
Dataset: /home/viki/workspace/chessboard_imu4/ch_gentle.bag
Topic: /cam0/image_raw
Number of images: 3057
Extracting calibration target corners
Extracted corners for 3057 images (of 3057 images)
Building the problem
Spline order: 6
Pose knots per second: 70
Do pose motion regularization: False
xddot translation variance: 1000000.000000
xddot rotation variance: 100000.000000
Bias knots per second: 50
Do bias motion regularization: True
Blake-Zisserman on reprojection errors -1
Acceleration Huber width (m/s^2): -1.000000
Gyroscope Huber width (rad/s): -1.000000
Do time calibration: False
Max iterations: 30
Time offset padding: 0.020000
Estimating imu-camera rotation prior
Initializing a pose spline with 5145 knots (100.000000 knots per second over 51.452952 seconds)
Orientation prior camera-imu found as: (T_i_c)
[[-0.00191114 -0.99996987 0.00752429]
[-0.99994817 0.00198624 0.00998568]
[-0.01000033 -0.00750482 -0.99992183]]
Gyro bias prior found as: (b_gyro)
[ 0.0001671 0.00031136 -0.00018996]
Initializing a pose spline with 5153 knots (100.000000 knots per second over 51.532952 seconds)
Initializing the bias splines with 2577 knots
Adding camera error terms (/cam0/image_raw)
Added 3057 camera error terms
Adding accelerometer error terms (/imu0)
Added 20236 of 20239 accelerometer error terms (skipped 3 out-of-bounds measurements)
Adding gyroscope error terms (/imu0)
Added 20236 of 20239 gyroscope error terms (skipped 3 out-of-bounds measurements)
Before Optimization
===================
Reprojection error squarred (cam0): mean 0.467676793977, median 0.207836145574, std: 0.797330713119
Gyro error squarred (imu0): mean 0.189767406562, median 0.0968219978587, std: 0.308269812229
Accelerometer error squarred (imu0): mean 54354.8262336, median 54130.3940099, std: 2950.85395155
Transformation T_cam0_imu0 (imu0 to cam0, T_ci): [m]
[[-0.00191114 -0.99996987 0.00752429 0. ]
[-0.99994817 0.00198624 0.00998568 0. ]
[-0.01000033 -0.00750482 -0.99992183 0. ]
[ 0. 0. 0. 1. ]]
Optimizing...
No linear system solver set in the options. Defaulting to the sparse_cholesky solver
Using the sparse_cholesky linear system solver
Using the levenberg_marquardt trust region policy
No linear system solver set in the options. Defaulting to the sparse_cholesky solver
Using the sparse_cholesky linear system solver
Using the levenberg_marquardt trust region policy
Initializing
Optimization problem initialized with 10325 design variables and 207385 error terms
The Jacobian matrix is 537706 x 46448
[0.0]: J: 1.1e+09
[pid 8460] +++ killed by SIGKILL +++PANIC: handle_group_exit: 8460 leader 8455
[1]: J: 1.02855e+08, dJ: 9.97149e+08, deltaX: 1.74884, LM - lambda:100 mu:2
[2]: J: 1.20735e+07, dJ: 9.07815e+07, deltaX: 1.28185, LM - lambda:33.3333 mu:2
[3]: J: 4.07142e+06, dJ: 8.00203e+06, deltaX: 0.889082, LM - lambda:16.7371 mu:2
[4]: J: 375400, dJ: 3.69602e+06, deltaX: 0.627697, LM - lambda:15.5062 mu:2
Last step was a regression. Reverting
[5]: J: 3.11681e+06, dJ: -2.74141e+06, deltaX: 0.719216, LM - lambda:5.16873 mu:2
Last step was a regression. Reverting
[6]: J: 3.12445e+06, dJ: -2.74905e+06, deltaX: 0.128326, LM - lambda:20.6749 mu:4
Last step was a regression. Reverting
[7]: J: 3.1491e+06, dJ: -2.7737e+06, deltaX: 0.0148867, LM - lambda:165.399 mu:8
Last step was a regression. Reverting
[8]: J: 3.14763e+06, dJ: -2.77223e+06, deltaX: 0.00119014, LM - lambda:2646.39 mu:16
Last step was a regression. Reverting
[9]: J: 3.15405e+06, dJ: -2.77865e+06, deltaX: 0.000214775, LM - lambda:84684.5 mu:32
Last step was a regression. Reverting
[10]: J: 3.16637e+06, dJ: -2.79097e+06, deltaX: 8.99753e-07, LM - lambda:5.41981e+06 mu:64
After Optimization (Results)
==================
Reprojection error squarred (cam0): mean 0.551160888183, median 0.251199784795, std: 0.855047993769
Gyro error squarred (imu0): mean 0.260294320249, median 0.142024223516, std: 0.355983314094
Accelerometer error squarred (imu0): mean 11.6355964031, median 3.8276315513, std: 20.4208009338
Transformation T_cam0_imu0 (imu0 to cam0, T_ci): [m]
[[-0.09970554 -0.98924099 -0.10705641 -0.23471977]
[-0.99416647 0.09459437 0.05181634 0.04647423]
[-0.04113191 0.11159827 -0.9929018 -0.00756453]
[ 0. 0. 0. 1. ]]
Saving calibration to file: camchain-imucam-homevikiworkspacechessboard_imu4ch_gentle.yaml
Detailed results written to file: results-imucam-homevikiworkspacechessboard_imu4ch_gentle.txt
Generating result report...
(python:8446): GLib-WARNING **: getpwuid_r(): failed due to unknown user id (1000)
Gtk-Message: Failed to load module "unity-gtk-module"
libproxy.so.1: cannot open shared object file: No such file or directory
Failed to load module: /usr/lib/x86_64-linux-gnu/gio/modules/libgiolibproxy.so
(python:8446): GVFS-RemoteVolumeMonitor-WARNING **: cannot open directory /usr/share/gvfs/remote-volume-monitors: Error opening directory '/usr/share/gvfs/remote-volume-monitors': No such file or directory
Report written to report-imucam-homevikiworkspacechessboard_imu4ch_gentle.pdf
I repeated this process several times with different test sequences. Each time I get very good rotation matrix, but completely wrong translation values (23cm is more than the dimensions of the phone!). Also, the translation results are completely different for similar sequences.
Does anyone have an idea what I am doing wrong / what can be done to get correct calibration results?
Thank you for any help!