accuracy / resolution of depth data?

[Denis Poussart]

Has anyone got a chance to assess the actual depth performance? From the grey level depth images I have seen, it does seem to be rather limited (but it is hard to judge from these videos)

I understand the x-y depth data is on a 320 x 240 xy grid and the original indicated 0.2 cm at 2 meters. But the MS implementation may have reduced it. The depth data stream seems to be 16 bits but the actual significant bits are certainly much less (6 or 7?). The depth resolution is also most certainly (very) distance-dependent.

I suspect the depth (z) resolution may turn out to quite seriously limiting for 3D scanning, while being quite OK for its intended gaming purpose. Of course, depth estimate can be improved (to a degree) by fusing with the RGB (intensity) data but this requires sophisticated further processing.

[Carl Kenner]

Depth is 640x480, 11 bits.

If you moved the object backwards and forwards, and did several scans
and combined the information, you might be able to use it for scanning
well.

[Dave]

Carl, the data structure is uint16_t, are we only getting 11 bits of
precision?

Dave

On Nov 13, 2:44 pm, Carl Kenner <carl.ken...@gmail.com> wrote:
> Depth is 640x480, 11 bits.
>
> If you moved the object backwards and forwards, and did several scans
> and combined the information, you might be able to use it for scanning
> well.
>

[Carl Kenner]

The real USB data structure is 11 bits, the openkinect code just
copies it into an array of uint16_t so you can use it. The values will
always be between 0 and (2^11)-1.

[Zsolt Ero]

Sorry, I am still on the "theoretical Kinect owner" level, but is the
depth resolution linear in space?

If there are 2000 possible values and it works say between 50 cm to 5
m (it's just my guess from the videos) then it would be about 3 mm in
resolution. But if the depth sensor senses in a field-of-view way,
then it would make the 5 m end have about 10x times less resolution
then the 50 cm end.

Zsolt

[SK]

No it isn't linear, you can find a data fit here:
http://www.ros.org/wiki/kinect_node

On Nov 14, 9:04 pm, Zsolt Ero <zsolt....@gmail.com> wrote:
> Sorry, I am still on the "theoretical Kinect owner" level, but is the
> depth resolution linear in space?
>
> If there are 2000 possible values and it works say between 50 cm to 5
> m (it's just my guess from the videos) then it would be about 3 mm in
> resolution. But if the depth sensor senses in a field-of-view way,
> then it would make the 5 m end have about 10x times less resolution
> then the 50 cm end.
>
> Zsolt
>
>
>
>
>
>
>
> On Sun, Nov 14, 2010 at 11:56 AM, Carl Kenner <carl.ken...@gmail.com> wrote:
> > The real USB data structure is 11 bits, the openkinect code just
> > copies it into an array of uint16_t so you can use it. The values will
> > always be between 0 and (2^11)-1.
>

[Dave]

Something doesn't seem right about that. The depth results I'm getting
from Theos OSX code are not linear. Closer to Logarithmic. It would
make sense because working with depth on a log scale would give
greater background separation. This will assist background subtraction
because a simple threshold will have a better chance to segment the
person.

Dave

[Patrick Mihelich]

My experience is with stereo cameras, but stereo and structured light
cameras are both based on triangulation - stereo computes pixel
offsets (disparities) of corresponding points in two captured images,
while the Kinect computes the offsets using the current IR image and a
known reference image from factory calibration. So the principle and
the math I believe are essentially the same.

In that case the depth resolution should decrease quadratically with
distance. E.g. if the depth Z resolution is 1cm at 2m distance (taken
from PrimeSense datasheet), it should be 4cm at 4m distance. The
formula in stereo is:

delta_Z = (Z^2 / fT) * delta_d

where
Z = depth
f = focal length of the camera
T = baseline (distance between the two cameras)
delta_d = smallest detectable offset increment - might be subpixel,
depending on how fancy the matching hardware is
delta_Z = depth range resolution

The data fit at http://www.ros.org/wiki/kinect_node is for converting
the raw 11-bit depth output from the sensor to real-world depth
values. Measurements show that these are inversely related. My
interpretation is that the raw depth output is actually the pixel
offsets, which are inversely related to depth:

Z = fT / d
where d = the disparity (pixel offset)

For the Kinect, the constant "fT" is probably something else - I guess
related to IR camera focal length, the baseline between projector and
camera, and the distance at which the reference image was captured.
But that's just a constant, it doesn't change the shape of the
relationships.

I'd assume the Kinect can provide the necessary constants to software
somehow. For now I'm also working on calibrating empirically, but
something for the reverse engineering gurus to keep in mind maybe :).

Cheers,
Patrick

[Denis Ivanov]

anyone tried more then one kinect at the same place? does they will work? what about mixed dot patterns?

Sent from my iPhone

[Radu Bogdan Rusu]

We just committed changes to our libfreenect branch that allow the usage of multiple cameras
(https://github.com/ros-pkg-git/libfreenect).

Judging from a few basic tests with two Kinects, I can't seem to get them to interfere with each other to the point
where the data is unusable.

Cheers,
Radu.

[Florian Echtler]

That's interesting, I would have expected more problems. Do you see any
noticeable interference at all?

Which brings me to another interesting question: did anybody compare the
light patterns from two different Kinects so far? Are they identical?

Florian

--
0666 - Filemode of the Beast

[Nishad Herath]

Hi Florian,

Regarding the light patterns, the following patent applications from
PrimeSense might shed some light :-)

Patent application title: DEPTH MAPPING USING PROJECTED PATTERNS
<http://www.faqs.org/patents/app/20100118123>
Patent application title: DEPTH MAPPING USING MULTI-BEAM ILLUMINATION
<http://www.faqs.org/patents/app/20100020078>

This one is interesting too:

Patent application title: INTEGRATED PROCESSOR FOR 3D MAPPING <http://
www.faqs.org/patents/app/20100007717>

Sorry about the non-USPTO links. I was too lazy to dig up the USPTO
links.

Look forward to hearing your thoughts.

Cheers
Nishad

On Nov 17, 5:41 pm, Florian Echtler <f...@butterbrot.org> wrote:
> That's interesting, I would have expected more problems. Do you see any
> noticeable interference at all?
>
> Which brings me to another interesting question: did anybody compare the
> light patterns from two different Kinects so far? Are they identical?
>
> Florian
>
>
>
>
>
>
>
>
>
> On Tue, 2010-11-16 at 23:24 -0800, Radu Bogdan Rusu wrote:
> > We just committed changes to our libfreenect branch that allow the usage of multiple cameras
> > (https://github.com/ros-pkg-git/libfreenect).
>
> > Judging from a few basic tests with two Kinects, I can't seem to get them to interfere with each other to the point
> > where the data is unusable.
>
> > Cheers,
> > Radu.
>
> > On 11/16/2010 10:14 PM, Denis Ivanov wrote:
> > > anyone tried more then one kinect at the same place? does they will work? what about mixed dot patterns?
>
> > > Sent from my iPhone
>

> > > On 17.11.2010, at 7:52, Patrick Mihelich<patrick.mihel...@gmail.com>  wrote:
>
> > >> My experience is with stereo cameras, but stereo and structured light
> > >> cameras are both based on triangulation - stereo computes pixel
> > >> offsets (disparities) of corresponding points in two captured images,
> > >> while the Kinect computes the offsets using the current IR image and a
> > >> known reference image from factory calibration. So the principle and
> > >> the math I believe are essentially the same.
>
> > >> In that case the depth resolution should decrease quadratically with
> > >> distance. E.g. if the depth Z resolution is 1cm at 2m distance (taken
> > >> from PrimeSense datasheet), it should be 4cm at 4m distance. The
> > >> formula in stereo is:
>
> > >> delta_Z = (Z^2 / fT) * delta_d
>
> > >> where
> > >> Z = depth
> > >> f = focal length of the camera
> > >> T = baseline (distance between the two cameras)
> > >> delta_d = smallest detectable offset increment - might be subpixel,
> > >> depending on how fancy the matching hardware is
> > >> delta_Z = depth range resolution
>

> > >> The data fit athttp://www.ros.org/wiki/kinect_nodeis for converting

[Simon Biggs]

Using structured light with a single camera source has been an established
technique for some years. Here's something I made before (in 1984).

http://www.littlepig.org.uk/right/CSIRO/index.htm

Best

Simon Biggs
s.b...@eca.ac.uk si...@littlepig.org.uk
Skype: simonbiggsuk
http://www.littlepig.org.uk/

Research Professor edinburgh college of art
http://www.eca.ac.uk/
Creative Interdisciplinary Research in CoLlaborative Environments
http://www.eca.ac.uk/circle/
Electronic Literature as a Model of Creativity and Innovation in Practice
http://www.elmcip.net/
Centre for Film, Performance and Media Arts
http://www.ed.ac.uk/schools-departments/film-performance-media-arts

Edinburgh College of Art (eca) is a charity registered in Scotland, number SC009201

[Henkka]

On 17 marras, 11:45, Nishad Herath <i...@nishadherath.com> wrote:
> Hi Florian,
>
> Regarding the light patterns, the following patent applications from
> PrimeSense might shed some light :-)
>
> Patent application title: DEPTH MAPPING USING PROJECTED PATTERNS
> <http://www.faqs.org/patents/app/20100118123>
> Patent application title: DEPTH MAPPING USING MULTI-BEAM ILLUMINATION
> <http://www.faqs.org/patents/app/20100020078>
>
> This one is interesting too:
>
> Patent application title: INTEGRATED PROCESSOR FOR 3D MAPPING <http://www.faqs.org/patents/app/20100007717>
>
> Sorry about the non-USPTO links. I was too lazy to dig up the USPTO
> links.
>
> Look forward to hearing your thoughts.

Hello,

It seems that there are multiple opportunities for identifying dots
between multiple beams. I wonder if anyone has taken any measurements
to measure whether the Kinetic IR-beam is somehow "uniquely"
modulated ? This could be probably done with either scope or high-
speed camera. Modulation certainly would allow multiple beams to co-
exist in same scene, as long as each modulation could be identified
uniquely. There certainly is some sort of duty-cycle in the beam, but
I wonder if it also contains some modulation data to identify each
Kinetic from other.

Just a thought, based on guess and quick look at the patents.

BR,

Henkka

[Murilo Saraiva de Queiroz]

On Wed, Nov 17, 2010 at 8:16 AM, Simon Biggs <s.b...@eca.ac.uk> wrote:

Using structured light with a single camera source has been an established
technique for some years. Here's something I made before (in 1984).

http://www.littlepig.org.uk/right/CSIRO/index.htm

  

Awesome work, kudos! 

And I loved that Apple IIe in the pictures... :-)  

--
Murilo Saraiva de Queiroz, MSc.
Senior Software Engineer 
http://www.vettalabs.com
http://www.tecnologiainteligente.com.br
http://www.acalantoemcasa.com.br

[Simon Biggs]

In the 30 years plus I've worked with computers that was the only Apple II I
ever worked with.

Best

Simon

Simon Biggs
s.b...@eca.ac.uk si...@littlepig.org.uk
Skype: simonbiggsuk
http://www.littlepig.org.uk/

Research Professor edinburgh college of art
http://www.eca.ac.uk/
Creative Interdisciplinary Research in CoLlaborative Environments
http://www.eca.ac.uk/circle/
Electronic Literature as a Model of Creativity and Innovation in Practice
http://www.elmcip.net/
Centre for Film, Performance and Media Arts
http://www.ed.ac.uk/schools-departments/film-performance-media-arts

> From: Murilo Saraiva de Queiroz <mur...@gmail.com>
> Reply-To: <openk...@googlegroups.com>
> Date: Wed, 17 Nov 2010 10:24:29 -0200
> To: <openk...@googlegroups.com>
> Subject: Re: more then one kinect
>

Edinburgh College of Art (eca) is a charity registered in Scotland, number SC009201

[linuxjacques]

I tried with two kinects last night.

At 90 to about 135 degrees (pointed at same spot and varying 7 to 10
feet apart) I saw very little interference if any.

With both in same place pointing in same direction I see black spot
interference (more small black spots than with a single kinect).

180 degrees (pointed at eachother) there are periodic black splotches
that fill the middle 50% of the depth image.
I think this is when the IR laser of one hits the sensor of the other.

jacques

On Nov 17, 1:24 am, Radu Bogdan Rusu <r...@willowgarage.com> wrote:
> We just committed changes to our libfreenect branch that allow the usage of multiple cameras
> (https://github.com/ros-pkg-git/libfreenect).
>
> Judging from a few basic tests with two Kinects, I can't seem to get them to interfere with each other to the point
> where the data is unusable.
>
> Cheers,
> Radu.
>
> On 11/16/2010 10:14 PM, Denis Ivanov wrote:
>
> > anyone tried more then one kinect at the same place? does they will work? what about mixed dot patterns?
>
> > Sent from my iPhone
>

> > On 17.11.2010, at 7:52, Patrick Mihelich<patrick.mihel...@gmail.com>  wrote:
>
> >> My experience is with stereo cameras, but stereo and structured light
> >> cameras are both based on triangulation - stereo computes pixel
> >> offsets (disparities) of corresponding points in two captured images,
> >> while the Kinect computes the offsets using the current IR image and a
> >> known reference image from factory calibration. So the principle and
> >> the math I believe are essentially the same.
>
> >> In that case the depth resolution should decrease quadratically with
> >> distance. E.g. if the depth Z resolution is 1cm at 2m distance (taken
> >> from PrimeSense datasheet), it should be 4cm at 4m distance. The
> >> formula in stereo is:
>
> >> delta_Z = (Z^2 / fT) * delta_d
>
> >> where
> >> Z = depth
> >> f = focal length of the camera
> >> T = baseline (distance between the two cameras)
> >> delta_d = smallest detectable offset increment - might be subpixel,
> >> depending on how fancy the matching hardware is
> >> delta_Z = depth range resolution
>

> >> The data fit athttp://www.ros.org/wiki/kinect_nodeis for converting