Issues With Stereo Camera Calibration and Stereo Disparity Creation

[Devin Willis]

Hello! I am working on a project that requires a stereo disparity map that I can use to estimate the distances to objects and at some point run SLAM as well. First step is to calibration the cameras but I think I am having some issues. I wrote a utility program that highlights the eco check board and saves the frames from both cameras when the sharpness is within an acceptable range. It stores the points on the image so I can make sure I get good coverage. Here are some example images.



I have reviewed all the images and they look good no blurriness or occluded markers. I then use the Planar Stereo Calibration app to rectify the two images and do the stereo calibration from my collected images. After rectification the images look very odd. I would expect properly radial distortion correct but it looks like below instead.



Rather than get distorted it kind of just gets sheared and rotated. Below are the results of the calibration.

Metrics             left right
quality.fill_border  98%  80%
quality.fill_inner  100% 100%
quality.geometric    73%  74%

Reprojection Errors (px):
mean=1.466 max=7.834

I know the geometric quality is low on this run but I have had similar runs with 100 percent geometric quality that produced the same issues and a similar reprojection error.

Taking this stereo yaml and trying to make a stereo disparity image gave me not bad results but I was hoping for a slightly denser stereo map. Here are the results of that below.



For context the camera I am using is here https://www.amazon.com/dp/B0D9VY8JG4?ref=fed_asin_title&th=1

It is a synchronized global shutter camera that outputs grayscale images.

I would appreciate any ideas to help me improve the quality of this calibration and or get a better stereo map.

Thank you!

[Jonny March]

I started without really trying and got:

quality.fill_border 77% 75% 

quality.fill_inner 100% 98% 

quality.geometric 71% 64% 

Reprojection Errors (px): mean=52.296 max=752.248  

Then tried again mostly holding the board more stable and more consistent range and got:

quality.fill_border 90% 83%

quality.fill_inner 100% 100%

quality.geometric 43% 57%

Reprojection Errors (px): mean=17.153 max=344.208  

The above was all having the board fairly perpendicular to the cameras.

Then again with 2.5ms exposure from 10ms. Some images I kept the board fairly perpendicular but also several I tilted the calibration board a lot in two axis, one at a time, so there was more depth and got:

quality.fill_border 95% 95%

quality.fill_inner 100% 100%

quality.geometric 100% 100%

Reprojection Errors (px): mean=0.155 max=1.298

Depth of the board giving the algorithm the most range of 3d space information I believe helped the most...
I used gig ethernet cameras with software synced global shutters and about 200 images each, about 400 total

Hope it helps,

Jonny