could not initialize subgraph!

[张宝先]

Dear Bernd,

Hi, thanks for your good work.

I want to build the tag map with a single camera. The environment is simple with circle boards (tag size 0.06m) and rectangle boards (tag size 0.05m).

The tag family is tagStandard41h12. I changed the apriltag detector code to suport the tag family and works (with tag payload 6 ? , I don't know the meaning).

I have check the tag size and the camera intrinsic paramter with not error. And there were not the same two tag ID.

I still can not figure out my problem below:

could not initialize subgraph!

[ERROR] [1675949240.995866549]: /tagslam: Indeterminant linear system detected while working near variable 15 (Symbol: 15).

Thrown when a linear system is ill-posed. The most common cause for this error is having underconstrained variables. Mathematically, the system is underdetermined. See the GTSAM Doxygen documentation at http://borg.cc.gatech.edu/ on gtsam::IndeterminantLinearSystemException for more information.

/tagslam: Requested the BayesTree clique for a key that is not in the BayesTree

I read the relative group issue below but still no progress.

https://groups.google.com/g/tagslam/c/V0v7HtnaNEo

https://groups.google.com/g/tagslam/c/kmuWYoysKs0

I had send you the test bag by email. Please help me out!

Thanks!

[Bernd Pfrommer]

Your intrinsic calibration is bad. Your distortion coefficients are huge! Also your fx and fy are different. Usually they are within 1-2 pixels identical.

I tried fudging it by setting all distortion coefficients to zero and using an equidistant projection but that is apparently not the right model. So "radtan" is probably right, just the coefficients are way too big. They are usually less than 1 in magnitude.

I get better (but still terrible) results with this calib file:

cam0:
  camera_model: pinhole
  intrinsics: [2565.0, 2565.0, 1440.0, 930.0]
  distortion_model: radtan
  #distortion_model: equidistant
  distortion_coeffs: [-0.2, 0, 0, 0, 0]
  resolution: [2880, 1860]
  rostopic: /usb_cam/image_raw/compressed
  tagtopic: /detector/tags
  rig_body: rig

At least it doesn't bomb out anymore. But DON'T use my hacked-up calibration file. The tags are not in a plane etc. It's not a good calibration file.

Note: I see you are trying to use the "board" feature. This won't work. Your boards don't have the right geometry for that since your tags are staggered, not in a regular grid pattern like the standard apriltag calibration boards.

[张宝先]

After updated the camera intrinsic calibration (use standard pinhole and radtan model),  it worked. 

Thanks!

[Leo Zhang]

Dear Bernd,

Note: I see you are trying to use the "board" feature. This won't work. Your boards don't have the right geometry for that since your tags are staggered, not in a regular grid pattern like the standard apriltag calibration boards.

Yes, I want to use the "board" feature. Since my board is not a regular pattern, I just use the first row to test it. When look into the code, it seems that one body can't support the type "board" and the "simple" at the same time?  The "board" feature makes it easy to add the multiple tags, without add the constraint with other tags?     

I also found the plane.cpp part.  As in my case, I found some tags with bad poses which is not in the some plane with the other tags.  How to use the plane measurement? I really don't know the plane equation. 

What I want to do is just add the plane constraint.

[Leo Zhang]

Some tags rotate bad, how to balance  this parameter pixel_noise, max_num_incremental_opt, and minimum_viewing_angle?   

When I added distance_measurements( circle and rectangle_measurement, which set the cross tag distance as show below, the distance between tag 216 and tag 233, the distance between tag 218 and tag 231, 

error occured and trajectory was bad.  I have emailed you the test bag and config file.

[ERROR] [1676970679.569582785]: /tagslam: Requested the BayesTree clique for a key that is not in the BayesTree.

[Bernd Pfrommer]

I can't tell you why that specific tag #227 looks so bad. The only reason I can imagine is because it initialized poorly the first time it was seen.

But there are safeguards for that in place. I recommend you look at the images with the decoded tags and in particular at the image where the tag is seen the first time.

The bag you sent me (abc.bag) is a very nice data set indeed and TagSLAM should do well on this one. I switched off the measurements for now just as a sanity check and most

tags are estimated well, but tags 154, 155,  157 and 159 are off. Why is that? Your calibration, while way better, is still not good enough to capture the distortion in the corners right.

Your distortion coefficients:

distortion_coeffs: [-0.3315078575346754319, -0.2931924828871734778, -0.0002564559973218503183, -0.0004682219514811433335, 0.6552477230968362099]

look better but the last value (0.655) looks pretty big, and that's going to screw up the estimation at the edges of the image, and in particular the corners.

The attached image is the only (!) image where tags 154, 155, 157 and 159 are detected, and they are far away and at the outer edge of the image. Without  doubt that's the reason why their pose comes out poorly.

I don't know how you calibrate your camera but I strongly recommend using Kalibr. Make sure you have plenty of tags detected out *all the way in the corner*. That is so crucial. Kalibr is great in that it gives you a diagnostic output in the form of a reprojection error graph. Only if that graph looks good (should ball with about 1pix reprojection error) can you be confident your calibration is good.

In case you want to absolutely stick with whatever calibration software you have right now: you need to collect more (a lot more!) points from the very corners of the image. We are talking all-the-way out, all four corners!

I'll try to implement a "staggered board" for you. Should not take too much time. Then all the tags will be forced to be (almost) perfectly in a plane and things will look visually good. Your calibration errors will still haunt you with dropped tags due to viewing angle problems and poor board/camera pose estimation.

[Bernd Pfrommer]

I have implemented a new feature called "staggered_board" that generates the tag poses for your boards. I have pushed the changes, please pull the tagslam repository (tagslam_root has not been updated yet, so just run git pull in the tagslam folder). The attached tagslam.yaml gives visually pleasing results. It also shows you how to use the plane measurement feature.
Your feedback is very much appreciated btw. TagSLAM is really struggling on this dataset. I developed it mostly on the old MIT detector which is much less sensitive. The new UMich detector finds tags that are super small and at very shallow angle. This leads to bad initializations. I had to add a minimum size feature (see the attached tagslam yaml file) to filter out small size tags, and also toss out shallow-angle detections more aggressively.

[Leo Zhang]

As mentioned above,  the camera  intrinsic calibration is important. I looked into the tagslam code, found something confused. Please help me out!

1) In tagslam/include/gtsam_distortion/Cal3DS3.h

line 38, 

* y' = theta_d/r * y // something like equidistant in the file Cal3FS2, the comment is no need?

as line25 says, 

Calibration of a camera with fisheye (radtan) radial distortion, // it for the fisheye? as I understand, equidistant in Cal3FS2 is for fisheye

line 14, 

* @brief Calibration of a camera with radtan p1,p2, k1...k6 distortion model

As we input the distortion coeffs in the cameras.yaml like this:

 distortion_coeffs: [-0.161604, 0.093674, -0.000037, -0.000336, 0.000000]

Cause the k4, k5, k6 are zeros. Is it not precise?

as we should use the model in https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html

That is k4 = k1, k5 = k2, k6 = k3.

What is the Cal3DS3 meaning? 3 for 3D? D for ? S for ? 2 for?

What is the Cal3FS2 meaning? 3 for 3D? F for fisheye? S for ? 2 for?

2) In tagslam/src/tag_slam.cpp

line 1128:

if (sz < minTagArea_) {

    ROS_WARN_STREAM("dropping tag: " << tagPtr->getId()

<< " due to small size: " << sz);

// need add the continue statement to not use the Tag info?

}

3) In tagslam/src/factor/tag_projection.cpp, 

line 43: 

checkIfValid(vbp, "no rig pose found"); // vbp should be vrp ?

4) In tagslam/include/tagslam/staggered_board.cpp, 

line 20:

type_ = "board"; // type_ = "staggered_board";?

5 ) In tagslam/src/init_pose.cpp, 

line 395 

const cv::Mat D0 = cv::Mat::zeros(0, 4, CV_64F); // cv::Mat::zeros(1, 4, CV_64F); 0->1 ?

https://docs.opencv.org/4.x/d3/d63/classcv_1_1Mat.html

static CV_NODISCARD_STD MatExpr zeros (int rows, int cols, int type)

Why use find_pose_by_homography function to find the cv::solvePnP rvec and tvec initial values, is any benifit?

Thanks!

[Bernd Pfrommer]

Thank you for reporting a number of bugs and errors in the comments.

Of the bugs you found, the only one seriously affecting results is the check for size, your point 2). No idea why I did not catch that. Fortunately that feature (and with it, the bug) was only introduced very recently.

I will address all your remarks in an upcoming commit.

Now to answer your questions.

This question of yours I do not understand:

"As we input the distortion coeffs in the cameras.yaml like this:

 distortion_coeffs: [-0.161604, 0.093674, -0.000037, -0.000336, 0.000000]

Cause the k4, k5, k6 are zeros. Is it not precise?

as we should use the model in https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html

That is k4 = k1, k5 = k2, k6 = k3.

"

Why should k4 be equal to k1 etc? Here is the full distortion model used. TagSLAM assumes the coefficients to be in the same order and meaning as opencv. Any coefficients not specified are assumed to be zero.

https://docs.opencv.org/4.2.0/d9/d0c/group__calib3d.html

To your question 5):

The rvec, tvec computed by find_pose_by_homography are subsequently used by cv::solvePnP as a starting guess. cv::solvePnP then iteratively improves upon the starting guess (if provided with one, and if useExtrinsicGuess == true).

[Bernd Pfrommer]

I'm still not sure I get your question but I'll try to answer what I can:

1) Yes, x and y are using the same distortion parameters k1, k2, k3 (for the numerator) and k4, k5, k6 (for the denominator), see https://docs.opencv.org/4.2.0/d9/d0c/group__calib3d.html#ga549c2075fac14829ff4a58bc931c033d

2) Any parameters not specified are set to zero. So if you specify [k1, k2, p1, p2, k3], that would mean that k4 = 0, k5 = 0, and k6 = 0, i.e. only the radtan distortion in the numerator is done. For many cameras this is sufficient, however very rarely you need k4, k5, and k6 to be non-zero. Note that the more parameters you use, the harder it gets to fit a good camera model, meaning you will need a lot of images for your calibration, and many calibration points near the edges of the image, otherwise your parameters will not be stable. Typically what you see then are some very large distortion coefficients that happen to just work for your calibration training images, but then will fail out-of-sample, i.e. a classical case of overfitting. So first try your calibration with k1, k2 and p1, p2, and only allow k3 and more when your calibration results are poor.

On Thu, Apr 13, 2023 at 8:32 AM Leo Zhang <baoxi...@gmail.com> wrote:

As in the document https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html,  

radial distortion can be represented as follows: 

x_distorted = x(1 + k1*r^2 + k2*r^4 + k3*r^6)  and y_distorted use the same k1, k2, k3

y_distorted = y(1 + k1*r^2 + k2*r^4 + k3*r^6)

So I am confused that, if I use the rad-tan model, the xy distorted uses the same k1, k2, k3, or the missing k params assumed to be zero.

--
You received this message because you are subscribed to a topic in the Google Groups "tagslam" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tagslam/WT5muqg80Ww/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tagslam+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tagslam/21e53887-f06c-4bfe-b84d-fd48fc745ad0n%40googlegroups.com.

[Leo Zhang]

I foud my misunderstanding of distortion model on https://docs.opencv.org/4.2.0/d9/d0c/group__calib3d.html (both x, y are using the same distortion parameters k1-k6) which is the same with the distortion model (k1 -k3)  https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html.    

Thanks for your detailed reply, especially the practical calibration guide!    

[Leo Zhang]

As in the document https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html,  

radial distortion can be represented as follows: 

x_distorted = x(1 + k1*r^2 + k2*r^4 + k3*r^6)  and y_distorted use the same k1, k2, k3

y_distorted = y(1 + k1*r^2 + k2*r^4 + k3*r^6)

So I am confused that, if I use the rad-tan model, the xy distorted uses the same k1, k2, k3, or the missing k params assumed to be zero.

On Thu, Apr 13, 2023 at 5:57 PM Bernd Pfrommer <bernd.p...@gmail.com> wrote:

[Bernd Pfrommer]

Leo, somehow your latest email landed in the group's spam folder. But I guess it has already been answered by now.

[Leo Zhang]

Yes, thank you very much!