Re: One Camera coordinates

[Peter A]

Hi Ce,

You're definitely heading in the right direction. I would tweak some of your steps though. Instead of rotating the vector, I would convert the plane from the world frame to the camera frame. Then find the intersection of the ray with the plane. Now you know the 3D point in the camera frame. Then convert that to the world frame. You could do everything in the world frame, but you need to transform the camera center into the world frame.

When trying to debug these problems, here's what I often do. Create a unit test where you know the ground truth, then back it out to create synthetic observations. Then apply what you think is the correct formula. Since you know the ground truth for each step you can see at which point they are different. That helps a lot.

Cheers,

- Peter

On Sun, Jul 2, 2023 at 4:14 AM Ce <knappe...@gmail.com> wrote:

Hello dear community!

I have a small question about the calculation of a 3D point with only one camera. I have a plane in 3D space, it can be arbitrarily rotated in space. It is defined as PlaneNormal3D_F64 and Se3_F64 in space. The plane is defined in the camera's coordinate system.

I obtain its orientation through a previous step. This is because the camera is part of a stereo camera system. However, this is irrelevant for this task. Thus Se3_F64 defines the orientation between the camera and the plane.

Now I capture an image point in the camera image. This pixel comes from a 3D object point in space on the plane - let's just say on the table. Now I want to get the 3D coordinate of the object point from the 2D coordinate on the camera image.

In theory, a spatial ray is created that is defined by the camera centre (0,0,0) and the spatial coordinate of the image point. This can then be intersected with the plane. The geometric calculation is no problem, but I need the image point in the spatial coordinate system and not in the image coordinate system.

I can manage the normalisation and rectification, but I am only missing the coordinate of the image point in the spatial system. Here is my code so far + the missing step:


//Point on Camera in Pixel
double x = point.x;
double y = point.y;
Point2D_F64 pixelCoordinate = new Point2D_F64(x, y);

//Normalized Point
Point2D_F64 undistortedPoint = new Point2D_F64();
Point2Transform2_F64 undistorter = LensDistortionFactory.narrow(intrinsicParameters).undistort_F64(true,false);
undistorter.compute(pixelCoordinate.x,pixelCoordinate.y,undistortedPoint);

//---STEP IS MISSING---

//Space Ray
Vector3D_F64 pointing = new Vector3D_F64();
pointing.setTo(undistortedPoint.x, undistortedPoint.y, 1);
GeometryMath_F64.mult(tp.getR(), pointing, pointing);
Point3D_F64 cameraCenter = new Point3D_F64(0,0,0);
LineParametric3D_F64 spaceRay = new LineParametric3D_F64(cameraCenter, pointing);

//Ebene
PlaneGeneral3D_F64 planeGeneral = georegression.geometry.UtilPlane3D_F64.convert(planeParametric, null);

//Intersection
Point3D_F64 intersectionPoint3D = new Point3D_F64();
Intersection3D_F64.intersection(planeGeneral, spaceRay, intersectionPoint3D);





I am happy about every hint.
Many thanks and best regards
Ce

--
You received this message because you are subscribed to the Google Groups "BoofCV" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boofcv+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/boofcv/eec1d4c0-ae4e-4dcf-954c-a0e5278435d4n%40googlegroups.com.

--

"Now, now my good man, this is no time for making enemies."    — Voltaire (1694-1778), on his deathbed in response to a priest asking that he renounce Satan.

[Ce]

Now finally I had the time to test it, thank you!!