IMU Noise Model

[Julian Schwehr]

Hi Paul,

I would like to use your toolbox for calibration. I've read the topics in the wiki, also the one about the IMU noise model. It says that:
and correspond to the values at . However, in the diagram below, the Allan Deviation is plotted. I got quite confused but I assume the values I am interested in are the ones on the Allan Deviation curve just like shown in the diagram and given in other sources?

My actual problem is however how to obtain .  I attached the plots of my measured Allan Deviation for the acceleration in x and in all three dimensions. The problem is, that I can't really fit a line with slope +1/2 to the curve of the AD. What should I do? Take the curve of the x-axis since that one has a section with a slightly higher slope? What values are to be taken as reasonable values for in such a case?

For the gyros  , the problem doesn't occur (see attachment).

Thanks a lot for your help!
Julian

[Janosch]

Hi Julian,

I will try to answer your questions on behalf of Paul.

a.) Allan variance / deviation. You are absolutely right Julian, it should say "Allan deviation". We will make the respective changes on the wiki page.

b.) You correctly pointed out that the Allan deviation of your accelerometer is "flat" for higher integration times, and does not show a clear "random walk" or "first order Gauss-Markov process" behaviour for long integration times. This is not an uncommon behaviour for inertial sensors and it indicates that in this regime, your sensor noise is dominated by flickering.

c.) There is no easy answer to your question on how to set the accelerometer_random_walk parameter correctly.
c1.) The model used in Kalibr does not (and should not, see c2) incorporate a flicker component. What I usually do is to set accelerometer_noise_density according to the wiki procedure. I then set accelerometer_random_walk s.t. the resulting model is "conservative", i.e. that the non-parametric Allan deviation that you computed from data is "below" the Allan deviation of your model.
c2.) (more importantly) If you use uncalibrated (MEMS) inertial sensors, "noise" is only one source of error in your inertial measurements. Erroneous scale factors and axis misalignment can easily be much more significant than what your static noise model says, once your IMU is actually moving! You need to take that into account when specifying the noise parameters for Kalibr, and set them "conservative enough" s.t. they allow for some of these errors to be captured as well. Depending on the sensor you are using, that can actually mean to make the noise parameters larger by a factor of two to ten. That's all I've got Julian - sorry. What type of sensor are you using?

Best,
Janosch

[Julian Schwehr]

Hi Janosch, hi Paul,

thanks for your answer! Assuming a conservative noise model reduces the errors to a reasonable value. I am using an XSens Mti-G 700 (400Hz) together with a Bumblebee2 stereo camera (20Hz). According to other threads in this forum, the results from the calibration should be ok:

Calibration results
===================
Reprojection error squarred (cam0):  mean 0.0391469662938, median 0.0212802778721, std: 0.0698499823634
Reprojection error squarred (cam1):  mean 0.0397683752791, median 0.0244933967691, std: 0.0596360389564
Gyro error squarred (imu0):          mean 0.171323444722, median 0.082274108564, std: 0.205328937958
Accelerometer error squarred (imu0): mean 0.357950788613, median 0.242382793176, std: 0.737434428464

Transformation (cam0):
-----------------------
T_ci:  (imu to cam0): [m]
[[-0.99949509 -0.01183534 -0.02948715 -0.02826914]
 [ 0.02948785  0.00011519 -0.99956513 -0.01627083]
 [ 0.01183359 -0.99992995  0.00023387 -0.01153722]
 [ 0.          0.          0.          1.        ]]

T_ic:  (cam0 to imu): [m]
[[-0.99949509  0.02948785  0.01183359 -0.02763854]
 [-0.01183534  0.00011519 -0.99992995 -0.01186911]
 [-0.02948715 -0.99956513  0.00023387 -0.01709463]
 [ 0.          0.          0.          1.        ]]

timeshift cam0 to imu0: [s] (t_imu = t_cam + shift)
0.0396549938643


Transformation (cam1):
-----------------------
T_ci:  (imu to cam1): [m]
[[-0.99949509 -0.01183534 -0.02948715 -0.14795114]
 [ 0.02948785  0.00011519 -0.99956513 -0.01627083]
 [ 0.01183359 -0.99992995  0.00023387 -0.01153722]
 [ 0.          0.          0.          1.        ]]

T_ic:  (cam1 to imu): [m]
[[-0.99949509  0.02948785  0.01183359 -0.14726012]
 [-0.01183534  0.00011519 -0.99992995 -0.01328558]
 [-0.02948715 -0.99956513  0.00023387 -0.02062371]
 [ 0.          0.          0.          1.        ]]

timeshift cam1 to imu0: [s] (t_imu = t_cam + shift)
0.0396322637769

Baselines:
----------
Baseline (cam0 to cam1): [m]
[[ 1.        0.        0.       -0.119682]
 [ 0.        1.        0.        0.      ]
 [ 0.        0.        1.        0.      ]
 [ 0.        0.        0.        1.      ]]
baseline norm:  0.119682 [m]


Gravity vector in target coords: : [m/s^2]
[ 9.79741866 -0.42528333 -0.2565573 ]


Calibration configuration
=========================

cam0
-----
  Camera model: pinhole
  Focal length: [290.77888, 290.77872]
  Principal point: [324.56128, 249.41136]
  Distortion model: radtan
  Distortion coefficients: [0, 0, 0, 0]
  Type: checkerboard
  Rows
    Count: 6
    Distance: 0.0407 [m]
  Cols
    Count: 8
    Distance: 0.0407 [m]


cam1
-----
  Camera model: pinhole
  Focal length: [290.77888, 290.77872]
  Principal point: [324.56128, 249.41136]
  Distortion model: radtan
  Distortion coefficients: [0, 0, 0, 0]
  Type: checkerboard
  Rows
    Count: 6
    Distance: 0.0407 [m]
  Cols
    Count: 8
    Distance: 0.0407 [m]



IMU configuration
=================

  Update rate: 400.0
  Accelerometer:
    Noise density: 0.006056
    Noise density (discrete): 0.12112
    Random walk: 0.0011056
  Gyroscope:
    Noise density: 0.0013248
    Noise density (discrete): 0.026496
    Random walk: 6.2176e-05


However, I don't see how the found translation between the IMU and the cameras can be correct (the rotation seems fine), since the IMU is between the cameras with the imu-x-axis aligned with the cam-x-axes but in the opposite direction. The result should therefore be something around +-6cm (at least less than the baseline) for both cameras in the first value of the T_ci/T_ic. Trying with other datasets, the translation is always different, so how do I know if I found a good estimate, if the error values are as good as the ones above? Do I need to consider anything special when exciting the system besides rotation around at least two axes? (I attached the calibration protocol to the post)
I am also trying with the --time-calibration option because there is a small time-difference of a few ms between the sensors. I receive better results with the time-calibration option, but I have the feeling that with this parameter the calibration is very sensitive to abrupt movement, is that correct? For me, it only works sometimes and if it does not, I get the error output below.
From the output I see, that the optimization is not initialized with the T_cam_imu from the camchain.yaml file but with the baseline of the camera, could this pose a problem?



Before Optimization
===================
Reprojection error squarred (cam0):  mean 0.384721468439, median 0.0638888864722, std: 1.3472417695
Reprojection error squarred (cam1):  mean 0.413390906049, median 0.107302183586, std: 1.14918075707
Gyro error squarred (imu0):          mean 92.3289277877, median 30.6993818637, std: 186.046937566
Accelerometer error squarred (imu0): mean 13248.4207943, median 13193.633743, std: 1063.90094598

Transformation T_cam0_imu0 (imu0 to cam0, T_ci): [m]
[[-0.99912566 -0.03779982 -0.01786283  0.        ]
 [ 0.01768259  0.00509722 -0.99983066  0.        ]
 [ 0.03788447 -0.99927233 -0.00442436  0.        ]
 [ 0.          0.          0.          1.        ]]

cam0 to imu0 time: [s] (t_imu = t_cam + shift)
0.142917489186

Transformation T_cam1_imu0 (imu0 to cam1, T_ci): [m]
[[-0.99912566 -0.03779982 -0.01786283 -0.119682  ]
 [ 0.01768259  0.00509722 -0.99983066  0.        ]
 [ 0.03788447 -0.99927233 -0.00442436  0.        ]
 [ 0.          0.          0.          1.        ]]

cam1 to imu0 time: [s] (t_imu = t_cam + shift)
0.142917489186

Optimizing...
No linear system solver set in the options. Defaulting to the sparse_cholesky solver
Using the sparse_cholesky linear system solver
Using the levenberg_marquardt trust region policy
No linear system solver set in the options. Defaulting to the sparse_cholesky solver
Using the sparse_cholesky linear system solver
Using the levenberg_marquardt trust region policy
Initializing
Optimization problem initialized with 10625 design variables and 148239 error terms
The Jacobian matrix is 423690 x 47794
[0.0]: J: 2.82731e+08
Exception in thread block: [aslam::Exception] /var/kalibr-build/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [2301.9 <= 2301.84 < 2301.95]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
Exception in thread block: [aslam::Exception] /var/kalibr-build/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [2302.85 <= 2302.79 < 2302.9]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
Exception in thread block: [aslam::Exception] /var/kalibr-build/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [2250.75 <= 2250.69 < 2250.8]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
[ERROR] [1425995348.595741]: Optimization failed!
Traceback (most recent call last):
  File "/var/kalibr-build/devel/bin/kalibr_calibrate_imu_camera", line 5, in <module>
    exec(fh.read())
  File "<string>", line 206, in <module>
  File "<string>", line 182, in main
  File "/var/kalibr-build/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 181, in optimize
    raise RuntimeError("Optimization failed!")
RuntimeError: Optimization failed!
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 64, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
ImportError: No module named apport.fileutils

Original exception was:
Traceback (most recent call last):
  File "/var/kalibr-build/devel/bin/kalibr_calibrate_imu_camera", line 5, in <module>
    exec(fh.read())
  File "<string>", line 206, in <module>
  File "<string>", line 182, in main
  File "/var/kalibr-build/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 181, in optimize
    raise RuntimeError("Optimization failed!")
RuntimeError: Optimization failed!

[Julian Schwehr]

I know that the sequence is only 12 sec and thus much shorter than your example. Is it possible that the result for the translation is only an unfavorable local minimum?
I will try it with a longer sequence and with smoother movement tomorrow, hope it helps.

[Janosch]

Hi Julian,

My suggestion would be the following: Try with a longer dataset (perhaps 60-100s, three datasets to cross-check), with rapid (but smooth, i.e. no shocks) motion in position and attitude, reasonably close to the calibration target. This should give good results for the camera-IMU extrinsic translation. Make sure the target is well illuminated, such that the camera exposure time is low. With such a configuration, the "sigma" on the translation should be in the order of 1cm (we run Kalibr on HW which is similar to what you use). Switching on time-sync is, in my opinion, a good idea. Even though you HW synchronize, there will be delays introduced through camera exposure time and filters inside your Xsens IMU (in the order of some ms). For best results, fix camera exposure time.

Maybe that's sufficient to bring the accuracy to an acceptable level. If it doesn't help, I will discuss your Kalibr results with my colleagues and we'll figure something out, hopefully.

Best regards,
Janosch

[Julian Schwehr]

Hi Janosch,

your suggestions worked out fine, however with sequences around 120-180s the obtained results seem more consistent than with sequences around 60-90s. I also cross-checked the origin of the IMU accelerometers with the XSens support so the accuracy in translation obtained by Kalibr should be 1cm in all directions at most. I attached the calibration result for you!
Thank you so far for your quick and informative help!

[Janosch]

Hi Julian,

Excellent, I am glad it worked out. Did I understood you correctly, you contacted XSENS regarding the position of the individual accelerometers? Hah. Reverse engineering is an other option.

Keep us updated, Julian.

[Julian Schwehr]

Hi Janosch,

yes, that is correct. From the beginning, we had some trouble with our whole setup. So unfortunately it was more reverse engineering than we anticipated. Asking for the accelerometer origin was only the last step.

[Anton Morzhakov]

Hi creators of Kalibr and its user,

First I want to say, I'm a newbie to orientational sensors and have the same problem as  Julian in obtaining IMU parameters for Kalibr (and also I got a large erros with parameters, i've found).

I have vectornav VN100T  orientational sensor and guess its noise dominated by flickering as well. I used script for Allan variance from Matlab central (  http://www.mathworks.com/matlabcentral/fileexchange/13246-allan  )

My first question is about fitting lines to Allan variance plot. For gyro, fitting line with the slope -1/2  looks good. Problems started when I tried to fit line with slope 1/2. I've just draw this line so that it intersects "flat" (zero slope) part of Allan variance graphs at the end. Graph itself is below the  1/2 slope line.
Accelerometer data looks better and I simply done the same: draw lines so that they lies above all three Allan variance graphs. 
Any ideas about what's going wrong ?

Janosch wrote about conservative model. It is not clear for me what does it mean. Could you, please, give some links or hints ?

With parameters from Allan variance plots I've got large errors.  Here is the report and graphs:


Initializing IMUs:
  Update rate: 200.0
  Accelerometer:
    Noise density: 0.002887
    Noise density (discrete): 0.0408283455457
    Random walk: 0.001228
  Gyroscope:
    Noise density: 0.0001237
    Noise density (discrete): 0.00174938217666
    Random walk: 0.0002606


...............................................................

After Optimization (Results)
==================
Reprojection error squarred (cam0):  mean 60.7278292894, median 23.2537770743, std: 107.956169081
Gyro error squarred (imu0):          mean 308.407230427, median 112.997707502, std: 579.108230911
Accelerometer error squarred (imu0): mean 453.200705642, median 204.163049823, std: 693.473661139

Transformation T_cam0_imu0 (imu0 to cam0, T_ci): [m]

[[ 0.04267603  0.99772172 -0.05225067  0.04776099]
 [ 0.7722302   0.00024192  0.63534279 -0.73251749]
 [ 0.63390794 -0.06746345 -0.77046051  0.21126227]
 [ 0.          0.          0.          1.        ]]

[Anton Morzhakov]

I've found that data  from Matlab differs from the data of my ROS sensor driver ( will rewrite it ).  By the way,  am I right, uncompensated calibrated data should be used for Kalibr ?

[janosch...@mavt.ethz.ch]

Hi Anton,

I am not an expert on this, so perhaps someone can confirm. Looking at the Kalibr pdf report you attached, the results you obtained are unreasonable: accel residuals and biases up to 1g, rate residuals up to 50dps, and reprojection errors of tens of pixels. All these should be one or two orders of magnitude lower.

Regarding the IMU noise parameters: Thank you for attaching the AD plots. Your derivations seem correct to me. You were optimistic with the bias random walk/diffusion component, I think, especially for the gyro (i.e. in reality the gyro bias will drift more rapidly at time-scales around one second, compared to what your model suggests). For your reference I attached the AD plot for one of the low-cost (MEMS chip) IMUs we are using. The parameters indicated appear correct (noise wise), but are too optimistic to capture all errors when the device is in motion.

Also, I think it could help to be more conservative (up to, perhaps, an order of magnitude for the random walk sigmas) with the noise parameters in general, to account for other sources of error in the inertial data originating from scale factor errors, etc.

Yes, I think if your IMU offers factory calibrated data, its best to use it (although, as you suggest, probably without "on-line" compensation).

It is crucial to have datasets which are long enough (e.g. 100s), smooth, and with enough excitation.

Let us know how if that helps.

[Anton Morzhakov]

Hi, Janosh !

Thank you for the answer ! You said you had attached AD plot for low cost IMUs, but I didn't find this attachment ))). About conservatism, this is not clear for me. Conservative in what sense ? How should I modify parameters to make it more conservative ? Just raise the line with 1/2 slope on AD plot ?

By the way, I did not noticed ! Kalibr uses up to 30 iteration at optimization step. In my case 30 iterations are not enough to converge. Is it reasonable to use more iterations or it should diverge in any way ?

I also noticed in "--show-extraction" mode, in 'reprojection corners' window one or two red circles flickers (at the corner of the grid). Have no idea so far why, illumination seems good. I guess this could add errors too.

Best wishes, Anton

[janosch...@mavt.ethz.ch]

Hi Anton,

attached you find the Allan deviation plots. By "more conservative" I mean increasing the noise parameters, which corresponds to "raising the lines". The reason for this is that once your camera-IMU system is in motion, additional errors are introduced into the inertial measurements (for example if the gyro scale factors in a low-cost IMU are inaccurate). Making your noise densities larger allows to cope with this.

Yes, I think things should converge with 30 iterations, but I am not an expert.

The target should be properly detected for the calibration to work out. Make sure it is illuminated well, close enough to the camera, and that motion is smooth. What are the specs of your target?

Janosch

[Anton Morzhakov]

Hi, Janosh !

I'm using Aprilgrid 6x6 0.5x0.5 m ( guess orientation of the grid does not matter ).  Another thing I've noticed with the camera:  started with 20 Hz it skews the picture (it could be seen clearly with fast motion ). So guess i should cope with this first.  The next thing is there an easy way to get camera extrinsics from Kalibr ?  I want to be  sure  camera tracking angles are  good ( e.g. simply by rotating camera around the single axis and checking relative values  using  graduated arc and values from Kalibr ).

Again thank you and best wishes !

[Anton Morzhakov]

Hi, Janosh !

Guess got better results. Just played with setting for the IMU, without any AD plot. First, I had multiplied all my params by 10 and then started to multiply them separately by 2 or 1.5. With time-calibration I've got better results.

So my results are:

  Update rate: 200.0
  Accelerometer:
    Noise density: 0.2509
    Noise density (discrete): 3.54826182799
    Random walk: 0.1026
  Gyroscope:
    Noise density: 0.02662
    Noise density (discrete): 0.376463650304
    Random walk: 0.015562


Before Optimization
===================
Reprojection error squarred (cam0):  mean 1.17583258812, median 0.620265506764, std: 1.837665471
Gyro error squarred (imu0):          mean 0.110966213282, median 0.0575282910332, std: 0.161118653435
Accelerometer error squarred (imu0): mean 29.792038341, median 29.7557528566, std: 1.8994447607

.........................................
After Optimization (Results)
==================

Reprojection error squarred (cam0):  mean 0.521857979533, median 0.270869129197, std: 0.768901540486
Gyro error squarred (imu0):          mean 0.0660310131809, median 0.0334188195155, std: 0.094370087289
Accelerometer error squarred (imu0): mean 0.153252467937, median 0.0587771274331, std: 0.263069020754

Transformation T_cam0_imu0 (imu0 to cam0, T_ci): [m]

[[ 0.0026411   0.99856802  0.05343164 -0.00035868]
 [ 0.12102033 -0.05335827  0.9912149   0.00077826]
 [ 0.99264651  0.00384841 -0.12098796  0.06036641]

 [ 0.          0.          0.          1.        ]]

cam0 to imu0 time: [s] (t_imu = t_cam + shift)

0.0130803134769

But I have a new set of dummy questions. First of all, report messages.

Comparison of predicted and measure angular velocities.
---------------------------------------------------------------

I guess system is using some  kind of Kalman filter. So it predicts angular velocity based on the gyro  parameters  and this graph is a residuals between  "model based" (predicted) values and values from the gyro.

Comparison of predicted and measure specific force
---------------------------------------------------------------
The same, but for accelerometer.

Two other bias slides are the same, but for bias.

But what about angular velocities and acceleration error ?

cam0 reproject error
-------------------------------------------------------

Is it reprojection of extracted April grid corners to the same April grid  using estimated camera rotation and translation ? Guess I'm not right.

Then, I suppose,  it should be done something with my cam to get less reprojection error. Have no ideas what to do, it is already well illuminated and close to the camera. 

[Anton Morzhakov]

Hi, Janosh !  I hope you will read my post ))

Guess have got better results (they are in attachment). My webcam  do not give me really good picture quality (image has some bluriness in some areas), so guess reprojection error of 3-4 pixels cannot be diminished.  The main question now is how to verify  matrix T_ci. Could  I use rotation matrix from orientational sensor in yaw-pitch-roll coordinate frame and then left multiply this matrix on T_ci to get camera rotation matrix ?

I've made new script on the base of the kalibr_calibrate_imu_camera script which outputs camera transformation matrices (with respect to April grid).  Then multiplied  T_ci*T_ypr and did not get right camera transformation matrix. Btw,  optimization was made with --time-calibration so may be I should take this into account .. Any hints how to check that matrix T_ci is the right one ?

[janosch...@mavt.ethz.ch]

Hi Anton!

I am glad that you obtain better results now.

One way to check if things make sense is to compare T_cam_imu with CAD data or manual measurements. T_cam_imu(1:3, 4) should give you the translation between the IMU and the camera, and T_cam_imu(1:3, 1:3) contains the 3x3 "rotation matrix".

Janosch

[riflei...@gmail.com]

Hi Anton,
Looks you are the expert for calibration now. I am new to this calibration with a low cost IMU, mpu9250, no doubt, datasheet does not provide bias instability for it. For the AD plot, is it based on the original version of matlab allan function?
http://www.mathworks.com/matlabcentral/fileexchange/13246-allan
Can you share me the configuration of your test for sampling data, like sampling rate, data count and duration?