Any possibility to hack the kinect hardware?

[Steven]

Right now you have to keep a good 40 cm (nearly 3 feet) distance from
objects to scan properly.

My question is, is there anything we can do to modify the hardware so
it can scan smaller objects at a closer distance.

What I mean by modify is crack open the kinect housing, analyze the
hardware and possibly make some mods or add a modification chip (after
market) of some kind so it can scan smaller objects at a closer
distance.

THAT or is there another scanner on the market that has the ability to
scan up close?

Thanks in advance everyone.

[Christoph Heindl]

Am Donnerstag, 1. März 2012 08:35:55 UTC+1 schrieb Steven:

My question is, is there anything we can do to modify the hardware so
it can scan smaller objects at a closer distance.

I'd love to be able to do this. There have been some discussions in this forum concerning lenses.

https://groups.google.com/d/topic/reconstructme/gATy0ZxMXy4/discussion

I've also found the following interesting thread

http://openni-discussions.979934.n3.nabble.com/OpenNI-dev-modify-kinect-for-short-range-td3461781.html 

For a start :)

[Steven]

Thanks, I am currently cracking open my Kinect now to have a look
inside. Stuck on the Torx 10 security...though dremel'ing the center
security pole out now :-p

[Christoph Heindl]

Don't ruin your Kinect :)

[Steven]

> Don't ruin your Kinect :)

We have Walmarts ;-)

[GwebMa]

I was surprised to see that my Kinect works down to a distance of 45cm without any modifications:
I scanned a shoe on the floor being 45cm away from the Kinect. It was captured with the FIRST frame after hitting "p" totally and without clippings.

Now I mounted +2.5 glasses on my device: http://imageshack.us/f/846/kinectzoom.jpg/
I could now get the object at a FIRST distance of 35cm and could capture some more details (sure now the scale and depth decoding cann´t be right any more so the bigger one is the zoom result):
http://imageshack.us/f/402/zoomtest.jpg/
Also tracking worked up to a certain point and concentrating on the floor I could also get the floor:
http://imageshack.us/f/513/zoomtest2.jpg/
A rough comparison of the size of the resulting shoe shows that the zoomed result is about 1.2 times bigger. Means the resolution should be 1.2 times better. The IR beams are condensed on a 0.8x0.8x0.8 volume:
http://imageshack.us/f/10/zoomdimens.jpg/
http://imageshack.us/f/37/zoomtest3.jpg/

Ok, one can easily notice the resulting distortion, due to the added lens distortion. Think that´s why tracking is a bit harder. Comparing new distorted depth data with the already reconstructed ones (somehow interpolated) is a hard job for ReconstructMe and trying to get a 360° reconstruction, data from one side damage the result taken from another position.

Don´t know if that might work to get rid of such distortions:
Assuming one adds the same lens on cam, sensor and emitter, could it work to get the added lens distortion by comparing a standard camera calibration with(a) and without(b) lens of the Kinect RGB cam? So b-a should be the added lens distortion. Maybe the depth image stream could be corrected by these parameters?Am 01.03.2012 um 11:07 schrieb Steven:

[MagWeb]

Another comparison with and without lenses

A self-reconstruction simply moving the Kinect around my head with my
own arms.

Don´t talk about the distortions, but I wonder how much better the
result with glasses would be if there would be right data.

Never thought such a result would be possible with a totally wrong
calibrated device.

[GwebMa]

Sorry, missing link:
http://imageshack.us/f/856/selfreconstrcomp.jpg/

[Christoph Heindl]

Am Donnerstag, 1. März 2012 20:07:44 UTC+1 schrieb MagWeb:

Never thought such a result would be possible with a totally wrong
calibrated device.

That's a suprise, yes. I'm stunned. 

[Christoph Heindl]

Am Donnerstag, 1. März 2012 17:35:50 UTC+1 schrieb MagWeb:

I was surprised to see that my Kinect works down to a distance of 45cm without any modifications:
I scanned a shoe on the floor being 45cm away from the Kinect. It was captured with the FIRST frame after hitting "p" totally and without clippings.

Now I mounted +2.5 glasses on my device: http://imageshack.us/f/846/kinectzoom.jpg/

Hehe, ready for industry ;)

 

I could now get the object at a FIRST distance of 35cm and could capture some more details (sure now the scale and depth decoding cann´t be right any more so the bigger one is the zoom result):
http://imageshack.us/f/402/zoomtest.jpg/

Do you think a depth camera calibration can fix these issues? Since the pattern gets also distorted, I wonder if one needs to correct that as well.

A rough comparison of the size of the resulting shoe shows that the zoomed result is about 1.2 times bigger. Means the resolution should be 1.2 times better. The IR beams are condensed on a 0.8x0.8x0.8 volume:
http://imageshack.us/f/10/zoomdimens.jpg/
http://imageshack.us/f/37/zoomtest3.jpg

Ok, one can easily notice the resulting distortion, due to the added lens distortion. Think that´s why tracking is a bit harder. Comparing new distorted depth data with the already reconstructed ones (somehow interpolated) is a hard job for ReconstructMe and trying to get a 360° reconstruction, data from one side damage the result taken from another position.

Yes that will be definitely hard.

 

Don´t know if that might work to get rid of such distortions:
Assuming one adds the same lens on cam, sensor and emitter, could it work to get the added lens distortion by comparing a standard camera calibration with(a) and without(b) lens of the Kinect RGB cam? So b-a should be the added lens distortion. Maybe the depth image stream could be corrected by these parameters?Am 01.03.2012 um 11:07 schrieb Steven:

We already incorporate a default calibration. We could calibrate the Kinect, Asus whatever with glasses and simply allow the user to specify custom calibration values for depth sensor. Here's the catch, we don't use the RGB for anything other than displaying purposes yet, but once we map depth to color (to retrieve texture) we will need a new rgb to depth as well.

Thanks for the insights!

[geertv...@gmail.com]

Op donderdag 1 maart 2012 20:55:44 UTC+1 schreef Christoph Heindl het volgende:
> Am Donnerstag, 1. März 2012 17:35:50 UTC+1 schrieb MagWeb:<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">I was surprised to see that my Kinect works down to a distance of 45cm without any modifications:
> I scanned a shoe on the floor being 45cm away from the Kinect. It was captured with the FIRST frame after hitting &quot;p&quot; totally and without clippings.
> Now I mounted +2.5 glasses on my device: <a href="http://imageshack.us/f/846/kinectzoom.jpg/" target="_blank">http://imageshack.us/f/846/<WBR>kinectzoom.jpg/</a></p></blockquote>
> Hehe, ready for industry ;)</div>
>  
> </div><blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">

>
> I could now get the object at a FIRST distance of 35cm and could capture some more details (sure now the scale and depth decoding cann´t be right any more so the bigger one is the zoom result):

> <a href="http://imageshack.us/f/402/zoomtest.jpg/" target="_blank">http://imageshack.us/f/402/<WBR>zoomtest.jpg/</a></p></blockquote>
>
> </div>
> Do you think a depth camera calibration can fix these issues? Since the pattern gets also distorted, I wonder if one needs to correct that as well.</div>
>
> </div><blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">

>
> A rough comparison of the size of the resulting shoe shows that the zoomed result is about 1.2 times bigger. Means the resolution should be 1.2 times better. The IR beams are condensed on a 0.8x0.8x0.8 volume:

> <a href="http://imageshack.us/f/10/zoomdimens.jpg/" target="_blank">http://imageshack.us/f/10/<WBR>zoomdimens.jpg/</a>
> <a href="http://imageshack.us/f/37/zoomtest3.jpg/" target="_blank">http://imageshack.us/f/37/<WBR>zoomtest3.jpg</a></p></blockquote><blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">
> Ok, one can easily notice the resulting distortion, due to the added lens distortion. Think that´s why tracking is a bit harder. Comparing new distorted depth data with the already reconstructed ones (somehow interpolated) is a hard job for ReconstructMe and trying to get a 360° reconstruction, data from one side damage the result taken from another position.</p></blockquote>
> Yes that will be definitely hard.</div>
>  </div><blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">

> Don´t know if that might work to get rid of such distortions:

> Assuming one adds the same lens on cam, sensor and emitter, could it work to get the added lens distortion by comparing a standard camera calibration with(a) and without(b) lens of the Kinect RGB cam? So b-a should be the added lens distortion. Maybe the depth image stream could be corrected by these parameters?Am 01.03.2012 um 11:07 schrieb Steven:</p></blockquote>
> We already incorporate a default calibration. We could calibrate the Kinect, Asus whatever with glasses and simply allow the user to specify custom calibration values for depth sensor. Here&#39;s the catch, we don&#39;t use the RGB for anything other than displaying purposes yet, but once we map depth to color (to retrieve texture) we will need a new rgb to depth as well.</div>
>
> </div>
> Thanks for the insights!</div>
>
> </div><blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">
> </p>
> </p>
> </p>
> </p></blockquote>

Please keep in mind that there is a big difference in between resolution and accuracy. It isn't a fact that making your resolution higher also improves the accuracy. We thought about hardware changes almost a year ago, and indeed, these special lenses also exist already for several months now. This lens was developed for small living rooms, so that people who planned to use their kinect could be skeletically tracked at closer distances compared to the original Xbox software. Accuracy wasn't an issue and the device is not used in that case as a 3D scanning device. Even when you can see more detail, this doesn't mean that you scan more accurat. To evaluate this you need a more accurat scanner to campare the 3D surfaces. We know that, depending on the object you scan and the distance, the absolute accuracy from the Kinect is about 2-3mm when the object is within the range of 1.5m from the sensor. When you take the Artec scanner this goes down to 0.25mm and for an ATOS scanner we talk about 0.008mm =8µm! The difference is without any doubt noticable when you buy these.
We figured out that changing the lenses implies changing the IR projection pattern and this implies changes to the primesense chip. As you even don't get the possibility to interrupt the IR pattern from OpenNi (also after several requests from several people), I don't think you will be able the change the hardware and still get an accurate scan. We didn't buy the lens because Mr. Yan Cui from DFKI could show us the negative influence it had on the 3D scanning and this was about the same problem you stated Mr. Heindl with the tracking.
But trying can only make you wiser and who knows, because for every problem there is a solution ... .

[GwebMa]

Am 02.03.2012 um 10:09 schrieb geertv...@gmail.com:

Please keep in mind that there is a big difference in between resolution and accuracy. It isn't a fact that making your resolution higher also improves the accuracy. We thought about hardware changes almost a year ago, and indeed, these special lenses also exist already for several months now. This lens was developed for small living rooms, so that people who planned to use their kinect could be skeletically tracked at closer distances compared to the original Xbox software. Accuracy wasn't an issue and the device is not used in that case as a 3D scanning device.

True, but consider:

There is a difference which lenses are used:

Once again: The NYKO "zoom" isn´t a zoom lens. It actually is the opposite: a wide angle lens. It widens the field of detection (to get a whole person at a closer distance) - forget it if your aim is to get better detection.

My lenses instead were "real" zoom lenses which condense the field of detection to a smaller region. This happens also in depth! (I used another software that shows the resulting depth image) Means if your accuracy in depth was about 2-3mm  you´ve to multiply this with 0.8 (for my lenses) too.

Even when you can see more detail, this doesn't mean that you scan more accurat.

Sorry, a not detected surface detail is more accurate than a detected one? Cann´t get that argument.

To evaluate this you need a more accurat scanner to campare the 3D surfaces. We know that, depending on the object you scan and the distance, the absolute accuracy from the Kinect is about 2-3mm when the object is within the range of 1.5m from the sensor. When you take the Artec scanner this goes down to 0.25mm and for an ATOS scanner we talk about 0.008mm =8µm! The difference is without any doubt noticable when you buy these. 

I´m owning different scanning devices (down to a accuracy of 0,05 mm- depending on size and surface) so there are no problems to do mesh deviations here. True, there are more accurate, but also slower, harder to handle and more expensive solutions out there. Here we talk about using a product made for gaming. I never expected the accuracy with this device which ReconstructMe gives now. No tool is made for all jobs - but let´s find out its limits and keep them all sharp as possible.

The tracking issue is caused by a wrong calibration which result in deformed "single-depthframe-meshes" . Aligning deformed meshes with rigid registration is always a trouble causing job (and non rigid attempts should be to slow for realtime results) and I was stunned to see it work under that conditions.

Am 01.03.2012 um 20:55 schrieb Christoph Heindl:

Hehe, ready for industry ;)

Yeah, love the design - note the special curve of the tape ;-)

Do you think a depth camera calibration can fix these issues? Since the pattern gets also distorted, I wonder if one needs to correct that as well.

Hmm, the Kinect is in some parts still a riddle for me (e.g. the fact that several intersecting Kinects can fiddle out which IR pattern was send by itself) .

May  I guess ? : You process every single depth image giving each pixel some depth (depending on its color value) and direction (based on the Kinects optical parameters). This partial result is aligned to the already known geometry and integrated in the volumes' coordinates/resolution.

My test showed that the Kinect only shows a meaningful depth image stream if the same lens is added to sensor and emitter. Different (even slightly) optics fail . So my guess is, that the sensor relays on a "hardcoded" relation emitter pattern/sensor and you can not change one of them independently. So I think you should not worry about correcting the emitted pattern.

Think it is "enough" to find the right transformation for the interpretation of the depth image.

Here's the catch, we don't use the RGB for anything other than displaying purposes yet

That was just the point why I mentioned it. I thought it as a simply tool to get the added lens distortion in a separate calibration step that has to be done only once:

I thought of a calibration as if it were a standalone webcam. Comparing calibration parameters of the RGB cam with and without added lens results in "added-lens-distortion". Now forget the RGB cam, but apply the same "added-lens-distortion" onto the interpretation of the depth image stream. That was what I thought, but maybe it would be better to get the distortion via depthsensor: maybe by reconstructing a 3D calibration target with known geometry and comparing result and ideal. 

But no need to hurry with this stuff - IMO it is important first to get all out of a not modded Kinect,  just wanted to state that such a modification may work.

[realh...@googlemail.com]

Great idea with the glases!

Can someone tell me where or how I have to put the modified config paramenters into Reconstructme?

Thank you!

Am Donnerstag, 1. März 2012 08:35:55 UTC+1 schrieb Steven: