Undistort a fisheye lens in Bonsai

[Nils]

Dear Bonsai community,

I have a fisheye lens that I need to "undistort" in order to obtain correct x,y coordinates when I track a rat in a 2m x 2m maze.

I have been using a large checkerboard pattern that I have put on my maze while playing around with the "Find Chessboard Corners", "Calibrate Camera Chessboard" and "Undistort" nodes, but I can't quite get what I want. Here is what I have tried: I have been moving around the checkerboard pattern while the "Calibrate Camera Chessboard" continuously outputs new intrinsic parameters. After having moved around the pattern all over the maze, I try out the different intrinsic parameters in the Undistort node. However, this only leads the area around the checkeboard to become undistorted (I end up with video frames that look similar to the two photos in this post). I have read that OpenCV cant handle fisheye lenses very well, and I am therefore wondering what the best way is to undistort a fisheye lens in Bonsai - perhaps this can be done with scripting. I am currently familiarising myself with the OpenCV library, so if anyone has experience with this I would appreciate the help!

I have attached the script I have been using.

Thank you!

Nils

(I removed my previous post to make my problem a bit more clear, since I couldn't edit it)

[João Frazão]

Hi Nils,

Can you send an image with the checkerboard filling all camera image? Try to make it with the chessboard orthogonal to the camera.

preferably also well lit.

Joao

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/b4ab69d5-0107-4c96-91f8-dff658106c05%40googlegroups.com.

--

Joao Frazao

Intelligent Systems Laboratory
Champalimaud Centre for the Unknown
Av. Brasília, Doca de Pedrouços
1400-038 Lisboa, Portugal
www.neuro.fchampalimaud.org

[Nils]

Dear João,

I have tried what you suggested, and it does look a lot better. However, I suppose since my camera is tilted slightly (I did put the checkerboard orthogonal to the camera) it is still not an ideal undistortion, since as you can see in the attached photo, the top and bottom right edges are cut off.

I can't really move my camera, since it is put into a hole in the ceiling in order to cover the whole maze (this hole was created before I arrived, and it is not located in the middle of the maze, which is why the camera has to be tilted slightly).

I have also tried with the different calibration flags in the CalibrateCameraChessboard node, and I get similar results with all.

is this as good as its gonna get, or is it possible to improve the undistortion further?

Best,

Nils

Den onsdag 18 december 2019 kl. 17:06:03 UTC skrev João Frazão:

Hi Nils,

Can you send an image with the checkerboard filling all camera image? Try to make it with the chessboard orthogonal to the camera.

preferably also well lit.

Joao

On Wed, Dec 18, 2019 at 12:39 PM Nils <nils.n...@gmail.com> wrote:

Dear Bonsai community,

I have a fisheye lens that I need to "undistort" in order to obtain correct x,y coordinates when I track a rat in a 2m x 2m maze.

I have been using a large checkerboard pattern that I have put on my maze while playing around with the "Find Chessboard Corners", "Calibrate Camera Chessboard" and "Undistort" nodes, but I can't quite get what I want. Here is what I have tried: I have been moving around the checkerboard pattern while the "Calibrate Camera Chessboard" continuously outputs new intrinsic parameters. After having moved around the pattern all over the maze, I try out the different intrinsic parameters in the Undistort node. However, this only leads the area around the checkeboard to become undistorted (I end up with video frames that look similar to the two photos in this post). I have read that OpenCV cant handle fisheye lenses very well, and I am therefore wondering what the best way is to undistort a fisheye lens in Bonsai - perhaps this can be done with scripting. I am currently familiarising myself with the OpenCV library, so if anyone has experience with this I would appreciate the help!

I have attached the script I have been using.

Thank you!

Nils

(I removed my previous post to make my problem a bit more clear, since I couldn't edit it)

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/b4ab69d5-0107-4c96-91f8-dff658106c05%40googlegroups.com.

[João Frazão]

Hi Nils,

sorry for the delay, i was experimenting a bit with this.

I usually do this manually, but you have an extreme case.

I need you to send me and image of the setup without any correction, use bonsai to save it, screenshot in this case are not that useful.

meanwhile can you try to set those options:

• CV_CALIB_FIX_PRINCIPAL_POINT The principal point is not changed during the global optimization. It stays at the center or at a different location specified when CV_CALIB_USE_INTRINSIC_GUESS is set too.
• CV_CALIB_FIX_ASPECT_RATIO The functions considers only fy as a free parameter. The ratio fx/fy stays the same as in the input cameraMatrix . When CV_CALIB_USE_INTRINSIC_GUESS is not set, the actual input values of fx and fy are ignored, only their ratio is computed and used further.
• CV_CALIB_ZERO_TANGENT_DIST Tangential distortion coefficients  are set to zeros and stay zero

FixPrincipalPoint:

what i want you to see is if the principal point stays in the center of the ccd, that is half of the resolution, 

This parameter is not really important to optimize. This just means that lens axis is really pointing to the center of the image. on a really bad lens it can be several pixels of. Usually is more than good.

FixAspect Ratio FxFY:

If you fix this you are just telling that the physical pixels of the sensor are squares, they always are.

Tangent Distortion

Next thing is the tangential distortion, this just means that the lens plane is parallel to the sensor plane, they are never aligned, but usually even in cheap cameras it is just an infinitesimal part of a degree.

FixFocalDistance

This one is important when you have 3d movement in the space, is just a factor to some other parameters and a way to know the real Z distance of a maker. In here is not important at all. 

I usually fix it at 1000

You can play with undistort, by fixing some of those values and use the distortion parameters stating from zero, and them changing them by small amounts one at a time. 

You gain some insight of what those parameters try to correct.

Joao

On Wed, Dec 18, 2019 at 12:39 PM Nils <nils.n...@gmail.com> wrote:

--

You received this message because you are subscribed to the Google Groups "Bonsai Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/b4ab69d5-0107-4c96-91f8-dff658106c05%40googlegroups.com.

[João Frazão]

also if you have some kind of image crop, like on camera settings make sure the crop is done in the center of the sensor.

[João Frazão]

Hi Dorian,

 you can find the pattern here: https://github.com/opencv/opencv/blob/master/doc/pattern.png

On Wed, Oct 14, 2020, 10:30 Dorian Battivelli <dorian.b...@gmail.com> wrote:

Hi Nils, 

Would you have the picture of the check board to send me please? The link to download it on the OpenCV post you put in your first post does not work anymore. 

Thanks, 

Best; 

Dorian 

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/c6d8ec16-a4d3-4f9c-8814-4281a604f407n%40googlegroups.com.

[Dorian Battivelli]

Thank you João, I actually tried to proceed using both openCV script and a workflow similar to the one used by Nils, but I can't correct the distrotion, here the post I just wrote: https://groups.google.com/d/msgid/bonsai-users/6a2bf810-34a9-44e3-954d-d3c9b0c24cfen%40googlegroups.com.

If you have any suggestion to help me figure out how to fix the issue,

thank you!