Quality of Kinect RGB Camera?

[noisecrime]

Hi,

I've been playing with kinect via openNI on winXP, through the use of
the latest C# wrapper and getting strange/poor quality from the rgb
camera. I've uploaded an example here

http://noisecrime.com/unity/images/kinect_rgb_Issue.JPG

Its a crop (no resizing) from a 640x480 image and you can clearly see
the issue on the fingers.

It looks like some wierd interlacing issue, where every other row is
offset by 1 pixel. Its not something I can remember experiencing using
previous cameras unrelated to kinect.

Is this normal? Do other developers see the same issue, or is this
unique to my set up?

In terms of installs i've used

avin2 - SensorKinect drivers
latest openNI unstable build v1.0.0.25
latest Nite unstable build v1.3.0.18

It happens both in my own project demos and using the NiViewer.exe
sample.

[gopinder verma]

Hi,
      Try changing your graphics card.May be a problem due to opengl rendering .

Regards,
Gopinder Verma.

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to openn...@googlegroups.com.
To unsubscribe from this group, send email to openni-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

[noisecrime]

On Mar 15, 4:07 am, gopinder verma <gopinderve...@gmail.com> wrote:
> Hi,
>       Try changing your graphics card.May be a problem due to opengl
> rendering .

Thanks, but its a Macbook Air so not an option on changing the card.

Oddly I noticed that when using 1280x1024x15 the image is fine, its
only when using 640x480 that it looks poor

[Nicolas Burrus]

This is probably due to the poor Bayer decoding. You could try the ROS
module (http://www.ros.org/wiki/openni_kinect) that implement Edge
Aware decoding to check the difference, it gives better results with
my Kinect, avoiding most edge artefacts.

Cheers,
Nicolas

[noisecrime]

On Mar 15, 7:25 am, Nicolas Burrus <nicolas.bur...@gmail.com> wrote:
> This is probably due to the poor Bayer decoding. You could try the ROS
> module (http://www.ros.org/wiki/openni_kinect) that implement Edge
> Aware decoding to check the difference, it gives better results with
> my Kinect, avoiding most edge artefacts.

Thanks for the info, sounds plausible, though since i've just
discovered that 1280x1024 images look perfect i'm not so sure.
Perhaps 640x480 uses a faster/poorer decoding than 1280x1024?

Anyway gives me something to look into, just suprised no-one else has
seen this.

[satori]

You're not the only one :P

I see the exact same problem on Windows 7

[suat gedikli]

Hi, yes these artifacts are from debayering. Use the code in the ROS
drivers to get RGB but edge aware. There are three implementations,
bilinear Edgeaware and EdgeawareWeighted. If you switch between those
you can see the difference. The code for debayering is in the file
"openni_image_bayer_grbg.cpp"
- Suat

[noisecrime]

On Mar 15, 8:34 pm, suat gedikli <suat.gedi...@googlemail.com> wrote:
> Hi, yes these artifacts are from debayering. Use the code in the ROS
> drivers to get RGB but edge aware. There are three implementations,
> bilinear Edgeaware and EdgeawareWeighted. If you switch between those
> you can see the difference. The code for debayering is in the file
> "openni_image_bayer_grbg.cpp"

Thanks for the link, seen ROS mentioned several times now, so will
investigate it when I can. For the moment, knowing that i'm not the
only one seeing this and that it should be a fixable debayering issue
means i wont worry about it unless I need to use a 640x480 image. As I
said 1280x1024 appears fine and its moe likely i'd use that now if I
needed a frame image.

thanks for everyones input.

[Bene]

Hi,
I suffered from the same issue, wich is a real pain if you try to
detect fingers / open hands.
Suat thanks for your great code in the ROS drivers. I was able to
modify your classes to use them in Windows with openNi and openCV. The
quality gain is great.
If you agree I would share this modified version to let others have
brighter color images too.

regards,
Bene

[Suat Gedikli]

Sure.. We are happy if others have a benefit from our code. Feel free :-)
Suat

> --
> You received this message because you are subscribed to the Google Groups
> "OpenNI" group.
> To post to this group, send email to openn...@googlegroups.com.
> To unsubscribe from this group, send email to
> openni-dev+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/openni-dev?hl=en.
>
>

--
Sent from my mobile device

*Diplom Informatiker (FH)
Dr. Suat Gedikli*

[Bene]

Fine, thanks again for this great piece of code Suat.
So here are the needed classes in a quick and perhaps dirty
modification ( so it's provided "as is" without any guarantee), but
for me it works very well.
http://ul.to/42uh5o14


regards,
Bene

On 17 Mrz., 18:09, Suat Gedikli <suat.gedi...@googlemail.com> wrote:
> Sure.. We are happy if others have  a benefit from our code. Feel free :-)
> Suat
>

[Raphael]

Hi,

Thanks Bene and Suat for sharing this useful code!
I've been trying to use it but I can't find a way to get the Bayer
image from the Kinect.

When I do:
> imageGen.SetPixelFormat(XN_PIXEL_FORMAT_GRAYSCALE_8_BIT); //imageGen == xn::ImageGenerator

It returns status code OK but then on:
> context.StartGeneratingAll();

It fails with the error status: "Bad parameter sent". I've tried
treaking my intialization sequence in all possible ways, but I can
only make it work in XN_PIXEL_FORMAT_RGB24... am I missing something?

[Dexter]

I'm having your exact same issue right now! Did you find any solution
to the bad parameter problem?

[purpledog]

I'm having the same exact problem.

1) Is it possible to get the raw bayer output from openNI ? A search
for "bayer" in all the include files in "C:\Program Files (x86)\OpenNI
\Include" fails to return anything.

2) If not, what needs to be recompiled to get this "ROS drivers" to
work ? I cannot locate this file: "openni_image_bayer_grbg.cpp"
mentioned earlier.
The closest I can find is "avin2-SensorKinect-0124bd2\Source
\XnDeviceSensorV2\Bayer.cpp", does that look like the file I should
try and fix ?

PS: I using windows7 64 bits and installed:
OpenNI-Bin-Win32-v1.0.0.25.exe
SensorKinect-Win32-5.0.0.exe

[Raphael]

No, still no solution.

I tried both on Window and OSX with the latest OpenNI (1.0.0.25).

Also, I am using Avin's modified driver from here, (at least on OSX,
have to check on windows):
https://github.com/avin2/SensorKinect
because the drivers from the Primesense website never worked for me.

I noticed that this one already has comes with
"InputFormat=6" #6 - Uncompressed 8-bit BAYER
set in the GlobalDefaults.ini, so this part should be OK...

Please tell me if you manage to make it work!

[Bene]

Hi,
sorry for the late response, I'm currently at the final phase of my
diploma thesis...

I use Win7 64bit too in combination with an quite old version of
Avin's dirver modification.
Without seeing your complete initialisation code the only thing that
came to my mind is: During initialization the pixel mode of the color
camera is set to XN_PIXEL_FORMAT_GRAYSCALE_16_BIT in my application.
Then StartGeneratingAll() is called.
Later, for edge aware debayering the mode is switched to
XN_PIXEL_FORMAT_GRAYSCALE_8_BIT

Besides, I never tried to get the bayer pattern of high resolution
images(1280x1024). Perhaps it only works in 640x480, just a guess.

Sorry can't test whether one of these could be the problem, as I don't
have a kinect at hand right now.

Hope this helps,
Bene

[purpledog]

It looks to me as a bug in the function:
void Bayer2RGB888(const XnUInt8* pBayerImage, XnUInt8* pRGBImage,
XnUInt32 nXRes, XnUInt32 nYRes, XnUInt32 nDownSampleStep, XnUInt32
nBadPixels)
SensorKinect\Source\XnDeviceSensorV2\Bayer.cpp

Can anybody confirm this ?

[Raphael]

Hi Bene,

Thank you for your answer!
I managed to get it working in the meantime and it all makes sense
now, let me explain:

I found in the driver this series of checks in the file "Source/
XnDeviceSensorV2/XnSensorImageStream.cpp" around line 180:

-------------------------
// check output format
if (nOutputFormat == XN_OUTPUT_FORMAT_GRAYSCALE8 && nInputFormat !=
XN_IO_IMAGE_FORMAT_BAYER)
{
XN_LOG_WARNING_RETURN(XN_STATUS_DEVICE_BAD_PARAM,
XN_MASK_DEVICE_SENSOR, "Grayscale8 output requires BAYER input!");
}
else if (nOutputFormat == XN_OUTPUT_FORMAT_YUV422 && nInputFormat !=
XN_IO_IMAGE_FORMAT_YUV422 && nInputFormat !=
XN_IO_IMAGE_FORMAT_UNCOMPRESSED_YUV422)
{
XN_LOG_WARNING_RETURN(XN_STATUS_DEVICE_BAD_PARAM,
XN_MASK_DEVICE_SENSOR, "YUV output requires YUV input!");
}

// check input format
if (nInputFormat == XN_IO_IMAGE_FORMAT_BAYER && nResolution !=
XN_RESOLUTION_UXGA && nResolution != XN_RESOLUTION_SXGA)
{
XN_LOG_WARNING_RETURN(XN_STATUS_DEVICE_BAD_PARAM,
XN_MASK_DEVICE_SENSOR, "BAYER input requires SXGA/UXGA resolution!");
}
else if (nInputFormat == XN_IO_IMAGE_FORMAT_JPEG && nOutputFormat !=
XN_OUTPUT_FORMAT_RGB24)
{
XN_LOG_WARNING_RETURN(XN_STATUS_DEVICE_BAD_PARAM,
XN_MASK_DEVICE_SENSOR, "Jpeg input is only supported for RGB24
output!");
}

-------------------------

Those were probably meant for Primesense's devices and do not
correspond to the possibilites of the Kinect device, as Avins' removal
of the resolution check can attest. I commented them out and
recompiled the driver, then I it worked (in 640x480 at 30 fps).
Apparently, by setting the color camera to
XN_PIXEL_FORMAT_GRAYSCALE_16_BIT in the initialization phase you were
able to bypass the security checks and then switch to
XN_PIXEL_FORMAT_GRAYSCALE_8_BIT later. A clever trick ;-)

I still have an annoying speed issue, as if the de-bayered image was
updated only every 4 frames or so... I'll try to fix that as well.

Cheers,
Raphael

[Raphael]

You're right, this is the function that does a poor debayering.
I tried different ways to perform the debayering inside my application
instead but I kept experiencing some annoying lag effects.
Eventually, I replaced this function directly inside the driver with
the code from ROS and now it works fine!

[Dexter]

Has anybody told the maintainer of the driver about that? So he can
fix the issue for the next driver build.. :-)

[purpledog]

Raphael,

Could you send us the ROS-modified source code ? :-)
Does that only affect the file SensorKinect\Source
\XnDeviceSensorV2\Bayer.cpp ??

[lamaranu]

Hi, so have you modified the Sensor or OpenNi driver with that code?

On Apr 12, 10:24 pm, Raphael <hibooo...@gmail.com> wrote:

[purpledog]

> Eventually, I replaced this function directly inside the driver with
the code from ROS and now it works fine!

Raphael, please help ! :-)


PS: it doesn't work with the new driver neither:
OpenNI-Win32-1.1.0.41-Dev.msi
SensorKinect-Win-OpenSource32-5.0.1.msi

[Raphael]

Sorry for my late reply...

Here is my Bayer.cpp file modified with the code from ROS. I provide
it without any guarantees, but this hack works well for me on both OSX
and Windows. Just replace the original file and recompile Avin's
driver following the instructions given in the README.

Hope it helps,
Raphael

https://docs.google.com/leaf?id=0B4ZFRA6Clqd_NjM1MTllOGUtM2M5NC00YzE4LWE2N2ItYmUyNWU3ZWEzMjFl&sort=name&layout=list&num=50

[CARMELO VELARDO]

Sorry, the page (or document) you have requested is not available.

Please check the address and try again.

CarmeloVELARDO

www.velardo.org

[purpledog]

Getting closer... :-)
Raphael, as Carmelo mentioned, the link doesn't work, could you give
it another go ?
Thanks for sharing !

[den ivanov]

URL dont work for me too (

Maybe you put online already compiled version of drivers?

Den

On 19 апр, 13:48, CARMELO VELARDO <carmelo.vela...@gmail.com> wrote:
> Sorry, the page (or document) you have requested is not available.
>
> Please check the address and try again.
>

>  CarmeloVELARDO*
> *
>
> *www.velardo.org*<http://www.velardo.org/>

>
>
>
>
>
>
>
> On Tue, Apr 19, 2011 at 11:41 AM, Raphael <hibooo...@gmail.com> wrote:
> > Sorry for my late reply...
>
> > Here is my Bayer.cpp file modified with the code from ROS. I provide
> > it without any guarantees, but this hack works well for me on both OSX
> > and Windows. Just replace the original file and recompile Avin's
> > driver following the instructions given in the README.
>
> > Hope it helps,
> > Raphael
>

> >https://docs.google.com/leaf?id=0B4ZFRA6Clqd_NjM1MTllOGUtM2M5NC00YzE4...

[Raphael]

Damn, this forum messes up with links... Please try again, this should
work now:

https://docs.google.com/leaf?id=0B4ZFRA6Clqd_NjM1MTllOGUtM2M5NC00YzE4LWE2N2ItYmUyNWU3ZWEzMjFl&sort=name&layout=list&num=50

On 19 avr, 11:48, CARMELO VELARDO <carmelo.vela...@gmail.com> wrote:
> Sorry, the page (or document) you have requested is not available.
>
> Please check the address and try again.
>

>  CarmeloVELARDO*
> *
>
> *www.velardo.org*<http://www.velardo.org/>
>
>
>
>
>
>
>

> On Tue, Apr 19, 2011 at 11:41 AM, Raphael <hibooo...@gmail.com> wrote:
> > Sorry for my late reply...
>
> > Here is my Bayer.cpp file modified with the code from ROS. I provide
> > it without any guarantees, but this hack works well for me on both OSX
> > and Windows. Just replace the original file and recompile Avin's
> > driver following the instructions given in the README.
>
> > Hope it helps,
> > Raphael
>

> >https://docs.google.com/leaf?id=0B4ZFRA6Clqd_NjM1MTllOGUtM2M5NC00YzE4...

[CARMELO VELARDO]

It works.

Thanks for sharing.

CarmeloVELARDO

www.velardo.org

[purpledog]

it works, thanks for sharing.

PS: I recompiled "engine.sln" and simply replace
"XnDeviceSensorV2KM.dll" with the new one.

[lamaranu]

Hi guys,

I've changed the Bayer.cpp file with the new one and recompiled all
but I don't notice any improvement in the image quality.

Is there any other thing to do?

Cheers

[Brekel]

It worked for me, thanks for sharing!

If it works it should get rid of the little horizontal teeth like
artifacts that can be seen especially on bright vertical lines.

Aim the Kinect at a window frame and it's usually quite obvious.

As a side effect the new debayering algorithm adds a small black
border around the image so it's easy to see if it's active or not.

[lukas...@arbuzz.eu]

Hi,
could you share compiled files? I have same problem, but can't compile
it by myself now.
Thansk

> >  Later, for edge awaredebayeringthe mode is switched to