Unable to use xml file, skew not supported

[Lewis]

Hello all,

I am cmurrently setting up my system with Basler cameras and calibrating braid. I followed the tutorial from the official user's guide. However, after getting the xml file with flydra_analysis and multicamselfcal, there is an error shows skew not supported.

I looked into the source code of Braid and found that the expected alpha c is too large after calculation. I tried to use the example xml from flydra_analysis and it is good to use. I also tried to convert the example .h5 file from flydra_analysis to xml file. However, the same error raised.

Furthermore, the format of xml file is the same as the format for water calibration, which is shown as an example from flydra_analysis.

I am using braid alpha 6 and the latest version of flydra_analysis, flydra_core and muilticamselfcal. May I ask if anyone has experience of troubleshooting of this issue?

Thank you very much,

Yue Zhu

[Andrew Straw]

Dear Yue,

Apologies for my delay. I just noticed your email now.

I presume by the “official user’s guide" you mean this: https://strawlab.github.io/strand-braid/braid_calibration.html

* Are you calibrating the intrinsic parameters using Strand Cam as described in the section "MCSC Method, Step 1: run "Checkerboard Calibration" to get the camera intrinsic parameters”? Can you send the files created by this procedure (they are saved to ~/.config/strand-cam/camera_info ) ?

* Are you using `--undistort-intrinsics-yaml` when calling `flydra_analysis_generate_recalibration`?

Best,
Andrew

> --
> You received this message because you are subscribed to the Google Groups "multi-camera software from the Straw Lab" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to multicams+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/multicams/b4368c57-1dfa-4fa3-b836-b604c21eb3b5n%40googlegroups.com.

[Lewis]

Hello Andrew,

Thank you very much for your reply and no worries for the delay.

I skipped step 1 and did not use  `--undistort-intrinsics-yaml` when calibrating. Someone informed me that "Checkerboard Calibration"  is not necessary during setting up and testing the cameras because the purpose of it is to decrease the calibration error. May I ask if this information is incorrect and if the "Checkerboard Calibration" must be done before  `flydra_analysis_generate_recalibration`?

Thank you very much,

Yue

[Andrew Straw]

Dear Yue,

If you are following the user’s guide and you find problems with the instructions there, please do indeed report these.

I can also try to help if you do not follow the user’s guide exactly, but please indicate clearly if this is the case. It is frustrating if someone writes that they followed the user’s guide but actually they deviated from it. This causes me more work as I start debugging on the basis of wrong information.

To answer your question, if using Strand Cam to implement the checkerboard calibration “must” be done: it may indeed not be necessary but I would recommend starting from a known working point and then deviating from that. You can also save images and then use the `strand-cam-offline-checkerboards` program, which does the same thing but runs offline.

Best,
Andrew

[Lewis]

Hello Andrew,

Thank you for your reply, and I apologize for not explaining everything clearly.

I performed the checkerboard calibration using Braid's URL for each camera (step 1) and repeated the entire process of step 2 today. During the checkerboard calibration, I collected at least 20 checkerboard images for each camera. However, the same error still appeared.  

I followed all the commands from the user's guide (https://strawlab.github.io/strand-braid/braid_calibration.html), except for this command: 'flydra_kalmanize ${DATAFILE} -r ${DATAFILE}.recal/result'. I had to add '--debug=0' to run the Kalmanization; otherwise, it would result in an error. The calibration error was significantly reduced after performing the checkerboard calibration.

The attached files are the calibration files I generated today. The 'camera_info.zip' file contains the checkerboard calibration, and the 'ched_data2.zip' file includes the step 2 calibration.  

Thank you very much,

Yue

[Yue]

Hello Andrew,

I hope this message finds you well.

I apologize for any inconvenience, but I want to raise the issue of "skew not supported." It seems that when validating the calibration matrices from MCSC, the expected alpha c is calculated too large.

I would greatly appreciate your suggestions on how to fix this issue.

Thank you very much for your time.

Best,

Yue

[Michael Harrap]

Dear Yue
My name is Michael Harrap (she/her) I am a postdoc in Andrew's lab and on the most recent occasions we have needed a Braid system set up I have been doing it.
I have looked at the files you sent and think I have come to the error you are describing. The below is the output of a similar error (although I may have come to it with a different command - I have been trying to see if it happens when I try the same process a different way - the below is functionally similar to the flydra_kalmanize step)

~: braid-offline-retrack --data-src '/home/strawlab/BRAID-DATA/20240723_124906_ched.braidz' --new-calibration '/home/strawlab/BRAID-DATA/chedcalibration_fromBraidZ.xml' --output '/home/strawlab/Desktop/ched/ched_data2/ched_data2/test_unbound_Imade.braidz'
Error: while reading calibration file "/home/strawlab/BRAID-DATA/chedcalibration_fromBraidZ.xml"
Caused by:
    cannot convert to or from flydra xml: skew not supported
Location:
    /builds/straw/rust-cam/braid-offline/src/lib.rs:869:79

Is this similar to what you have seen? Also can I ask if you have the images used for the checkerboard calibration? 

Thanks
Michael
(she/her)

[Yue]

Hello Michael,

Thank you very much for your reply.

Yes, this is the same error I have. 

I just checked and did not have the images during the checkerboard calibration because I used the online calibration function from Strand Cam. The checkerboard information was saved without saving the images. I can describe the method. A checkerboard was printed and moved within the FOV of a Basler's camera with the online calibration function. While using the calibration function, the number of columns and rows was set. 

This error happens with and without the checkerboard calibration when applying MCSC.

I appreciate your help!

Best,

Yue

[Michael Harrap]

Dear Yue

Ok, I understand not having the images, indeed they are not needed to be saved to get the calibration done I thought it might help identify what is going on, see below.

So the reason I am asking is that the distortion coefficients on your 'Camera Info' .xmls are very high, we usually see these values ranging from -1 to 1, some of yours are an order of magnitude bigger. We think this might be the cause, I am trying to establish why this would happen.

It might be that the lenses you have on your cameras are very heavily distorting the image (i.e. do you have really intense fish-eye lenses, ours are a quite fish-eye) - can we ask what lenses you have on your cameras?

I have discussed this with another user who has seen this issue and looked at their images to try to establish what they were doing that differed from us that may explain this. We think there might be a misunderstanding between how we do the checkerboard calibration vs what they did, that led to similar issues - I am wondering if you did something similar to them. They moved the board throughout the field of view at a bit of a distance (getting 20 or so images like I am in the example photo attached with the board elsewhere in the FOV). I tried this morning to calibrate a camera this way and this did result in higher distortion coefficients than the calibration I did on the camera before (using the way I described below).

What we do is show the board to the camera from different angles in such a way that it still fills the field of view, that is so the board gets right up in the camera. With regard to angles I mainly aim to get a straight on view and at a angle into the sides of the screen. I then try to get some similar images at a distance. I will try send a video of me doing it on a camera on our demo setup, basically I wait in the positions I want until I am able to visually confirm the board is captured (I view any extra captures as a bonus - note the green checkerboard capture doesn't always appear as it captures a frame, so I am looking at the counter that says the number of checkerboards caught).

Did your approach correspond more with the former or latter? We get that both are an understandable interpretation of the instructions, we are in the process of revising the instructions.

Thanks

Michael

[Michael Harrap]

[Yue]

Hello Michael,

Thank you very much for your reply and your example video. I followed your checkerboard calibration method from the video and did the calibration again. However, the checkerboard calibration values are still very large. The attached zip file contains the checkerboard calibration information and wand calibration data. When everything is recalibrated, the error is still there.

The lenses I use are Basler C11-2520-12M 25mm focal length. These lenses should not have heavy distortion. 

Here is another information about the checkerboard. Because of the FOV, it cannot cover a big checkerboard. The checkerboard is 5 x 5 with a 10mm length for each square block. The total size of the checkerboard is 50mm x 50mm, and there are 4 x 4 inner corners.

Best,

Yue

On Thursday, October 24, 2024 at 6:14:29 AM UTC-4 m.j.m....@gmail.com wrote: