Depth to Real World XY Coordinate

[Kamarulzaman Kamarudin]

Hi, 

I am looking for a formula to convert Depth to XY coordinate. 

I found these formula in OpenKinect but when I used them, the values were off.

Do i and j correspond to horizontal and vertical index of the depth array?

And what does z means? is it Raw disparity or Actual Depth (in mm)?

x = (i - w / 2) * (z + minDistance) * scaleFactor * (w/h)
y = (j - h / 2) * (z + minDistance) * scaleFactor
z = z
Where
minDistance = -10
scaleFactor = .0021.
These values were found by hand.

Is there any other formula that is reliable to get the Real World XY coordinate?

Thanks,

Arul

[Daniel Wunderlich]

Hi Arul,

some months ago, I tried to understand this conversion, too. I detailed summed up my theoritical results at [1] and [2]. (You don't have to care about the programming aspects.)

Take a look these pages and ask me, if you have questions.

Regards,
Daniel

[1]  http://pille.iwr.uni-heidelberg.de/~kinect01/doc/reconstruction.html
[2]  http://pille.iwr.uni-heidelberg.de/~kinect01/doc/classdescription.html#kinectcloud-section

[Amirmotion]

These functions are open source in microsoft's Kinect SDK and openni. Look in NuiSkeleton.h

People love us on github.com/zigfu

[Ken Mankoff]

See also offline_registratino branch in mankoff openkinect github
fork and the kinect_register program therein.

-k.

[adam]

Hi Arul, 

Just ran into this same problem not more than two weeks ago.  Here is the code that I used: 

You may have to change some of the types. 

The conversion is to meters, and doesn't account for radial distortion, which might be important if you are using kinect for wide angle sensing.

typedef struct v { 

  float x;  

  float y;  

  float z;  

} vector3d;

float raw_depth_to_meters(int depth_value){ 

  float depth_value_f = (float) depth_value; 

  if (depth_value < 2047){ 

    float depth = 1.0 / (depth_value_f  * -0.0030711016 + 3.3309495161);

    return depth; 

  }

  return 0.0f; 

} 

vector3d get_handpos_xyz_from_cg(float cgx, float cgy, float cgz, int hand){

  double fx_d = 1.0 / 5.9421434211923247e+02;

  double fy_d = 1.0 / 5.9104053696870778e+02;

  double cx_d = 3.3930780975300314e+02;

  double cy_d = 2.4273913761751615e+02;

  float depth = raw_depth_to_meters(cgz);

  

  vector3d hand_pos = {   

    (float) (cgx - cx_d) * depth * fx_d,

    (float) (cgy - cy_d) * depth * fy_d,

    (float) depth };  

return hand_pos; 

}

[DongHyun Lee]

Hi, Arul

 

I know another formula but I am not sure it's perfectly correct.

 

However, the result is quite well. I think.

Here is a link.

 

https://sites.google.com/site/leekinectproject/project-updates/kinect-depthcaliberation

 

I hope that it would be helpful to you.

 

Regards,

DongHyun.

[Kamarulzaman Kamarudin]

Thanks very much guys.

I will try them and select the best one. Thanks again.

--

Best Regards,

 

Arul Kamarudin

CEASTech Unimap

[promach]

Hi, may I know how the values 0.0021 and -10 are derived ?

and how are the following equations found ?

x = (i - w / 2) * (z + minDistance) * scaleFactor * (w/h)
y = (j - h / 2) * (z + minDistance) * scaleFactor
z = z

[Kamarulzaman Kamarudin]

Hi Promach,

I am not using that equation since the values obtained seem not accurate.

Instead I use the equations from 

http://www.shiffman.net/2010/11/14/kinect-and-processing/

However, I am pretty sure most people here are not using this equation. Some of them are using Point CLoud Library (pointclouds.org) but I'm not entirely sure how they did that.

I use Labview for programming, so, do not need the library.

Thanks,

Arul

 

--
You received this message because you are subscribed to the Google Groups "OpenKinect" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openkinect+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Kamarulzaman Kamarudin]

Please ignore the link given. This is the function from PointCloud to calculate the 3D (real world) Depth.

PVector depthToWorld(int x, int y, int depthValue) {

final double fx_d = 1.0 / 5.9421434211923247e 02;
final double fy_d = 1.0 / 5.9104053696870778e 02;
final double cx_d = 3.3930780975300314e 02;
final double cy_d = 2.4273913761751615e 02;

PVector result = new PVector();
double depth = depthLookUp[depthValue];//rawDepthToMeters(depthValue);
result.x = (float)((x - cx_d) * depth * fx_d);
result.y = (float)((y - cy_d) * depth * fy_d);
result.z = (float)(depth);
return result;
}

You can create equation in other language or just download the PointCloud library

Thanks,

Arul

[Peter]

Those values look like they came from a camera that someone else calibrated.  If you don't mind some error that should work just fine.  If you require maximum precision then you will need to calibrate the camera yourself.  By calibrating the camera you will have the actual intrinsic parameters and be able to remove lens distortion.  After calibrating my camera I found that lens distortion moved the image by a few pixels at the image edge, which is significant in some applications.  The kinect camera is still much less distorted than your typical webcam.

Here's instructions on calibrating your camera in OpenCV.  It also does a good job of going over the distortion equations.

http://docs.opencv.org/trunk/doc/tutorials/calib3d/camera_calibration/camera_calibration.html

The one catch is that I'm not sure what the built in functions are for removing lens distortion in OpenCV.  Maybe someone else can answer that question? 

I actually used BoofCV (my own library) to calibrate the camera and then remove lens distortion and convert pixel coordinate into normalized image coordinates (e.g. pointing vector from optical center).

http://boofcv.org/index.php?title=Tutorial_Camera_Calibration
The easiest way to remove lens distortion is with the function LensDistortionOps.transformRadialToNorm_F64() which will create a class that does all the hardwork for you.

- Peter