Stream 70/80 and Isochronous Transfer errors.

[James_Masz]

Purchased a Kinect a few days ago and was able to successfully install
the latest unstable branch from github, applied the patch from
2010-11-16, specifically for OSX. I haven't attempted to pull the
stable branch, yet.

I am experiencing frequent packet loss, and inevitable freezing which
is intermittent (freezes immediately or sometimes after a while) when
running glview.

Here is my current system profile:

Model Name: MacBook Pro
Model Identifier: MacBookPro2,2
Processor Name: Intel Core 2 Duo
Processor Speed: 2.16 GHz
Number Of Processors: 1
Total Number Of Cores: 2
L2 Cache: 4 MB
Memory: 1 GB
Bus Speed: 667 MHz
Boot ROM Version: MBP22.00A5.B07
SMC Version (system): 1.12f5
OSX Version, 10.6.8

Here is a pastebin URL of the output GLView provided:

http://pastebin.com/s2WCZCd6

After careful scouring, all that seems to be as solid solutions is to
use Linux (better USB controller handling), reducing the FPS (which
requires a recompile) or a workaround in the software to flush the USB
controller's buffer. Both seem like band-aid solutions, and I and
others seem to be a minority to this problem. I've already tried
swapping with another Kinect from the retailer I bought it from but
this problem still persists.

Does anyone have a permanent solution? I haven't witnessed the
developers addressing the issue (or have missed this).

Thanks all,
James

[robin.dev]

Hi James,

I do not think you are the only one with this problem, as I experience
the same things as you described. I'm running openkinect on windows
Vista. When I use the SDK from microsoft, the problem is not there,
But it is very present in my most recent code, where I used directx
instead of openGL for drawing the graphics(I used directx because of
the latency opengl seemed to cause)
So I guess there is something going on with the driver, not the kinect
itself. I also noticed that the problem only seems to be there when
the video and depth image are used at the same time. So it looks to me
like a problem with handeling the 2 streams at the same time. Maybe
the 2 streams need to be handled in a specific order? now the callback
decided wich one goes first. I can imagine that the tread handling on
linux/max/windows can be quite different, and could explain the
problem only occuring on mac/windows. I would also appreciate any
pointer to where the problem could be. I will try to profile the code
one of these days.

Best regards,

Robin

[Marcos Slomp]

> I do not think you are the only one with this problem, as I experience
> the same things as you described. I'm running openkinect on windows
> Vista. When I use the SDK from microsoft, the problem is not there,
> But it is very present in my most recent code, where I used directx
> instead of openGL for drawing the graphics(I used directx because of
> the latency opengl seemed to cause)

In Windows, the libfreenect runs on top of libusb-win32, which acts as a "dummy" kernel-mode Kinect driver, while the driver logic itself sits within libfreenect (user-mode).

All current libusb-win32 releases are "debug" builds, which means that there is a lot of debug logging and security-checks going on within this "dummy" driver, which runs in Kernel mode (they don't show in the regular console,.you will need some tool like WinDbg to capture and display them).

Therefore, it is natural that this may incur into some extra performance overhead.

You can also download libusb-win32 and build it in "release" mode using the WDDK and see if this helps.

It is my conjecture that in some computer configurations, this overhead may cause packet losses, while in others, everything runs fine.

(And BTW, just because the Microsoft SDK does not echo anything, this does not mean that it is not actually dropping packets/frames! Since we have no access to the source code, it is hard to tell what is going on behind the scenes and if the official driver is silently-ignoring stuff and extrapolating data for lost frames.)

(I have also noticed that libfreenect tends to use much less CPU than the Microsoft SDK (85% vs 45%), using the same program structure.)

As for OpenGL, I am not sure if I understand what you mean by "latency".

Anyway, if there is an significant issue, it is likely to be due to the implementation of the OpenGL driver by your graphics card vendor.

You can try using multiple PixelBuffer Objects to stream the data to the GPU and subsequently to the textures.

However, if you are not aiming at portability (the strength of openkinect), then using Direct3D is actually preferred.

> So I guess there is something going on with the driver, not the kinect
> itself. I also noticed that the problem only seems to be there when
> the video and depth image are used at the same time. So it looks to me
> like a problem with handeling the 2 streams at the same time. Maybe
> the 2 streams need to be handled in a specific order? now the callback
> decided wich one goes first. I can imagine that the tread handling on
> linux/max/windows can be quite different, and could explain the
> problem only occuring on mac/windows. I would also appreciate any
> pointer to where the problem could be. I will try to profile the code
> one of these days.

The "threads" are indeed very different on Windows/Mac/Linux.

Linux/Mac uses libusb-1.0 for all the back-end USB logic (there are no libfreenect-level threads, but app-level threads doing polling).

Windows uses some sort of "emulator" for the libusb-1.0, since a Win32 version of the library is not available.

In the case of Windows, there are threads running at the libfreenect-level, and they not necessarily handle incoming packets in order.

There is reason for this madness, and it is mainly due to the restrictions that libusb-win32 (which is libusb-0.1) imposes to "emulate" libusb-1.0.

However, as for the reason of the same behavior being happening on Mac, I am clueless...

All I know is that Mac uses a special "edition" of libusb-1.0...