Undistortion

[Dr BK]

I am working on a commercial project using ASUS Xtion Pro Live depth
cameras and OpenNI. For our product to be successful, we need to
measure in real-time the velocity of moving objects within a 5% error
tolerance. For this reason, we need accurate conversion from the
depth camera to spatial coordinates. We have reason to believe that
the conversion given to us in OpenNI is inaccurate. In particular,
internal tests of distance measurements show that the conversion gives
consistently low readings, as if the conversion to real world
coordinates is off by a constant multiplier. Furthermore, if we point
the camera at a flat floor, we notice significant warping,
particularly at the corners of the images.

Is there any support in OpenNI for changing the conversion from depth
camera to spatial coordinates? In particular, is there some way to
give custom undistortion coefficients or a custom focal length? If
not, do you have suggestions for how we might accomplish this
otherwise?

Finally, does anyone know if the distortion is improved by using a
Kinect? What if we were to use the Kinect for Windows SDK?

Thanks for your help

[Igor Barros Barbosa]

On this paper they talk about a better calibration for Kinect.
It might help

http://www.ee.oulu.fi/~dherrera/kinect/


Cheers
01001001011001110110111101110010

Igor Barros Barbosa

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to openn...@googlegroups.com.
To unsubscribe from this group, send email to openni-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

[Lorne Covington]

The ASUS site lists its field of view (FoV) as 58� H, 45� V, 70� D (not
sure what sense "70 D" makes...) and the Kinect is 57 H and 43 V; so
this might be a source of the error as these numbers are used in
converting the depth map from projective to world XYZ. The error would
appear greater the further from the origin (center of the depth image).

Also, since velocity is a relative measurement, actual depth is not
important as long as the error is constant. If it is a linear (or
scalar) kind of error, some quick calibration should be possible.

Good luck!

- Lorne

[Roee Shenberg]

D = diagonal. Use that and the CMOS's aspect ratio to derive the actual horizontal and vertical FoVs.

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to openn...@googlegroups.com.

To unsubscribe from this group, send email to openni-dev+unsubscribe@googlegroups.com.

[Norman Link]

If you have your own camera calibration data, you can build up your own projection
matrix and thus reproject every image point to its corresponding world coordinate.
One starting point could then be the OpenCV function ReprojectImageTo3D, which
does exactly this for stereo disparity images.

Best regards,
Norman

[Dr BK]

Just for everyone's knowledge, OpenNI reads the focal length of the
camera from the camera itself. So the difference between the FOV for
the ASUS and for the Kinect is not the culprit. However, the symptoms
_can_ be explained by a bad focal length. I am using depth images
that have been shifted to the RGB camera perspective. Therefore, the
RGB focal length should be used. I wonder if maybe the depth camera
focal length is being used instead.

Does anyone know the FOV or the focal length of the RGB camera in the
ASUS Xtion Pro Live?

-Josh Brown Kramer

On Feb 28, 10:41 am, Dr BK <jbkbusin...@gmail.com> wrote:
> I am working on a commercial project using ASUS Xtion Pro Live depth
> cameras and OpenNI.  For our product to be successful, we need to
> measure in real-time the velocity of moving objects within a 5% error
> tolerance.  For this reason, we need accurate conversion from the
> depth camera to spatial coordinates.  We have reason to believe that
> the conversion given to us in OpenNI is inaccurate.  In particular,
> internal tests of distance measurements show that the conversion gives
> consistently low readings, as if the conversion to real world
> coordinates is off by a constant multiplier.  Furthermore, if we point
> the camera at a flat floor, we notice significant warping,
> particularly at the corners of the images.
>
> Is there any support in OpenNI for changing the conversion from depth
> camera to spatial coordinates?  In particular, is there some way to

> give customundistortioncoefficients or a custom focal length?  If

[Gabriel]

ever found a solution to this?

On Mar 8, 1:15 am, Dr BK <jbkbusin...@gmail.com> wrote:
> Just for everyone's knowledge, OpenNI reads the focal length of the
> camera from the camera itself.  So the difference between the FOV for
> the ASUS and for the Kinect is not the culprit.  However, the symptoms
> _can_ be explained by a bad focal length.  I am using depth images
> that have been shifted to the RGB camera perspective.  Therefore, the
> RGB focal length should be used.  I wonder if maybe the depth camera
> focal length is being used instead.
>
> Does anyone know the FOV or the focal length of the RGB camera in the

> ASUSXtionPro Live?

>
> -Josh Brown Kramer
>
> On Feb 28, 10:41 am, Dr BK <jbkbusin...@gmail.com> wrote:
>
>
>
>
>
>
>

> > I am working on a commercial project using ASUSXtionPro Live depth

[Dr BK]

I have looked closely at the source now, and OpenNI is using the depth
camera focal length even though it should be using the RGB focal
length in the case that the depth and RGB have been registered.
Another problem is that, I believe, the registration process warps the
depth image to compensate for RGB lens distortion. So a perfect
solution would undistort, then use the RGB focal length to compute
distance.

Unfortunately, as far as I can tell there is no access to the RGB
focal length (although there is for depth). Furthermore, the
registration is being done in firmware (in the case of the Xtion Pro
Live for sure, and presumably the Kinect as well). So there is no way
to see how it is doing it.

For now, I have experimentally determined a constant (1.08275) by
which I multiply the x and y coordinates whenever I convert from
project to real coordinates.

-Josh Brown Kramer

[Rula]

The transformation table in OpenNI is calculated by Sensor in this function void BuildDepthToShiftTable(XnUInt16* pDepth2Shift, XnSensorDepthStream* m_pStream) that you can find on this file https://github.com/PrimeSense/Sensor/blob/master/Source/XnDeviceSensorV2/Registration.cpp

What you are pointing out, is probably a bug in Sensor, not OpenNI.

-Rula.

[Roee Shenberg]

You can see how registration is done at  https://github.com/PrimeSense/Sensor/blob/master/Source/XnDeviceSensorV2/Registration.cpp  - both kinect and xtion are 1080 devices. Unfortunately, that code makes it clear the parameters aren't exposed in OpenNI (they're read using XnHostProtocolAlgorithmParams and not stored in variables you an access via the OpenNI interface), so the only option seems to be to patch sensorkinect in some way to expose those parameters, or perhaps using xnusb functions query the device directly.

Roee

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to openn...@googlegroups.com.

To unsubscribe from this group, send email to openni-dev+...@googlegroups.com.

[Roee Shenberg]

Sorry, I was beaten to the punch.

[Dr BK]

Actually, I've traced execution, and it never makes it down to that
code, since the Xtion Pro Live supports firmware registration. As far
as I know, the firmware code is not open source. Is there reason to
believe that the firmware registration process is the same as the
process in
https://github.com/PrimeSense/Sensor/blob/master/Source/XnDeviceSenso...
?

I think it's a matter of perspective whether the bug is in Sensor or
OpenNI. My perspective is that xnConvertProjectiveToRealWorld (in
OpenNI) should but does not distinguish between registered and
unregistered depth images, so that is whether the error is.

BTW, I have verified that there is much less error in distance
measurements on the unregistered depth image (on the order of 1%
rather than 8%).

-DrBK

On Mar 22, 4:57 pm, Rula <rver...@gmail.com> wrote:
> The transformation table in OpenNI is calculated by Sensor in this function
> void BuildDepthToShiftTable(XnUInt16* pDepth2Shift, XnSensorDepthStream*

> m_pStream) that you can find on this filehttps://github.com/PrimeSense/Sensor/blob/master/Source/XnDeviceSenso...