Win32 is in!

[Joshua Blake]

I got an interesting email this morning from a guy named Marcos Slomp. He had written libusbemu, a libusb 1.0 emulator (or wrapper) for libusb-win32 (i.e. the libusb 0.1 API). He wrote it specifically with libfreenect in mind. After some collaboration I confirmed that we can use it with the existing libfreenect codebase and the usb_libusb10 abstraction layer.

 

libusb-win32 -> libusbemu -> usb_libusb10 -> libfreenect

 

Not only does it work, it works really well. This gives us performance equivalent to Zephod's code but with the libfreenect codebase!

 

I did a bunch of work today and tonight and got all of this integrated and available in the unstable branch (https://github.com/OpenKinect/libfreenect/tree/unstable). I've also updated source files and cmake to play nicely with Visual Studio. You should be able to general Visual Studio projects (VS2010 tested here) and compile and run glview. (See http://openkinect.org/wiki/Getting_Started#Dependencies for dependencies.)

 

I've also tested lostincake's csharp_wrapper branch on top of this and with a few changes it works. I need to work with lostincake and Kyle to get the csharp_wrapper branch ready to integrate but it should be only another day or two.

 

Thanks,

Josh

---
Joshua Blake
Microsoft Surface MVP
OpenKinect Community Founder http://openkinect.org

(cell) 703-946-7176
Twitter: http://twitter.com/joshblake
Blog: http://nui.joshland.org
Multitouch on Windows book: http://manning.com/blake

[Rodrigo Ratan]

Great news!

Hope to soon play with XNA and Kinect!

On 17 dez, 09:59, Joshua Blake <joshbl...@gmail.com> wrote:
> I got an interesting email this morning from a guy named Marcos Slomp. He
> had written libusbemu, a libusb 1.0 emulator (or wrapper) for libusb-win32
> (i.e. the libusb 0.1 API). He wrote it specifically with libfreenect in
> mind. After some collaboration I confirmed that we can use it with the
> existing libfreenect codebase and the usb_libusb10 abstraction layer.
>
> libusb-win32 -> libusbemu -> usb_libusb10 -> libfreenect
>
> Not only does it work, it works really well. This gives us performance
> equivalent to Zephod's code but with the libfreenect codebase!
>
> I did a bunch of work today and tonight and got all of this integrated and
> available in the unstable branch (https://github.com/OpenKinect/libfreenect/tree/unstable). I've also updated
> source files and cmake to play nicely with Visual Studio. You should be able
> to general Visual Studio projects (VS2010 tested here) and compile and run
> glview. (Seehttp://openkinect.org/wiki/Getting_Started#Dependenciesfor
> dependencies.)
>
> I've also tested lostincake's csharp_wrapper branch on top of this and with
> a few changes it works. I need to work with lostincake and Kyle to get the
> csharp_wrapper branch ready to integrate but it should be only another day
> or two.
>
> Thanks,
> Josh
>
> ---
> Joshua Blake
> Microsoft Surface MVP

> OpenKinect Community Founderhttp://openkinect.org

[Juan Carlos del Valle]

:D nice!

I think I'll need to update the as3 wrapper too...

I'll try to do it on the weekend.

Cheers!
JC

[Ryan Gordon]

Awesome!

Can't wait to get working on the LabView wrapper now. Will this win32 branch be able to be compiled into a dll?

[Marcos Slomp]

Ryan,

Yes, a DLL build might be possible (haven't tested yet).

Josh already modified the core libfreenect headers in the unstable branch in order to account for the Win32 __declspec(dllexport) idiom.

[W.arcos]

Quick updated:

Just tested the DLL build, works fine =)

On Dec 18, 3:12 pm, Marcos Slomp <msl...@gmail.com> wrote:
> Ryan,
>
> Yes, a DLL build might be possible (haven't tested yet).
> Josh already modified the core libfreenect headers in the unstable branch in
> order to account for the Win32 __declspec(dllexport) idiom.
>
>
>
>
>
>
>
> On Sat, Dec 18, 2010 at 6:27 AM, Ryan Gordon <rygord...@gmail.com> wrote:
> > Awesome!
>
> > Can't wait to get working on the LabView wrapper now. Will this win32
> > branch be able to be compiled into a dll?
>
> > On Fri, Dec 17, 2010 at 9:49 AM, Juan Carlos del Valle <
> > jc.eki...@gmail.com> wrote:
>
> >> :D nice!
>
> >> I think I'll need to update the as3 wrapper too...
>
> >> I'll try to do it on the weekend.
>
> >> Cheers!
> >> JC
>

> >> On Fri, Dec 17, 2010 at 11:35 AM, Rodrigo Ratan <rodrigora...@gmail.com>

[Ryan Gordon]

Awesome, thanks!

[Ryan Gordon]

I was able to get it compiled and running :) The only thing to mention past the great work getting this in, is that it's pretty intensive on my Quad Core i5 (2.4GHz). On a comparable linux build, it's much less intensive.

Cheers :)

[Justin Hebert]

I was also able to get it compiled and running on an old XPS M1710 laptop.

Thanks Marcos for the driver and Josh for all the timely assistance ;)

[Marcos Slomp]

Yes, it is far from perfect... 

However, by no means the current implementation is definitive! It is actually pretty awful hehehe.

Having to deal with libusb-0.1 behind the scenes introduces some tricky issues: the main one is the need for a time-critical thread per stream to get it to work soundly, otherwise there will be a bunch of "sequence lost".

The same thing holds true for the original zephod's win32 code (if you have it, try commenting out the SetThreadPriority lines).

It seems that libusb-win32 was not designed to poll more than one stream at a time.

I am currently investigating different strategies of probing the stream data in a reliable way so that overhead can be alleviated.

Marcos

[Antonio Ospite]

On Sun, 19 Dec 2010 19:32:25 +0900
Marcos Slomp <msl...@gmail.com> wrote:

> Yes, it is far from perfect...
> However, by no means the current implementation is definitive! It is
> actually pretty awful hehehe.
>
> Having to deal with libusb-0.1 behind the scenes introduces some tricky
> issues: the main one is the need for a time-critical thread per stream to
> get it to work soundly, otherwise there will be a bunch of "sequence lost".
> The same thing holds true for the original zephod's win32 code (if you have
> it, try commenting out the SetThreadPriority lines).
> It seems that libusb-win32 was not designed to poll more than one stream at
> a time.

Can't you call libusb people directly for some help? Is there anything
blocking a proper _native_ port of libusb-1.0 to win32/win64?

The emu layer has a lot of sense as a temporary solution indeed, but on
the long run fixing libusb —if possible— should be the goal, don't you
think?

> I am currently investigating different strategies of probing the stream data
> in a reliable way so that overhead can be alleviated.
>
> Marcos

Keep up the good work.

Regards,
Antonio

--
Antonio Ospite
http://ao2.it

PGP public key ID: 0x4553B001

A: Because it messes up the order in which people normally read text.
See http://en.wikipedia.org/wiki/Posting_style
Q: Why is top-posting such a bad thing?

[LostInCake]

I am pretty sure the win32 implementation right now doesn't use
libusb. It uses the primesense driver. Primesense only uses libusb on
linux.

> Antonio Ospitehttp://ao2.it

>
> PGP public key ID: 0x4553B001
>
> A: Because it messes up the order in which people normally read text.

>    Seehttp://en.wikipedia.org/wiki/Posting_style

> Q: Why is top-posting such a bad thing?
>

>  application_pgp-signature_part
> < 1KViewDownload

[Marcos Slomp]

You are absolutely correct: the emulator is a palliative solution until the official libusb 1.0 becomes available for Windows.

According to the official libusb website:

"Windows support is coming soon in the official release (in git as of August 2010)"

Since this has not been materialized yet I decided to take some action, but my primary focus is Kinect, that is, I have focused mainly on USB isochronous transfers.

The code I originally wrote was more of a "proof-of-concept" to allow the use of the main libfreenect development branch on Windows, as well as to increase client code portability.

When I first contacted Josh regarding my idea, I confess that I was not expecting so much enthusiasm from the community.

I have never considered seriously implementing the emulation layer to a complete and sound 1.0 functionality (not alone, at least).

The idea of working/contributing directly on the official libusb was never firm on me because I am no expert in USB communication whatsoever.

Anyway, I'll follow your advice and contact the libusb staff regarding the emulator.

Certainly they can feedback us with valuable information and, perhaps, they will also get excited with the idea.

After all, having a libusb 1.0 emulator running on top of libusb 0.1 could potentially benefit other systems/platforms where the 1.0 API port is hard to achieve.

[LostInCake]

Very cool work so far. I didn't know the full details behind this, but
thanks for the hard work :)

> >   Seehttp://en.wikipedia.org/wiki/Posting_style

[GUNNM]

I've got error System.EntryPointNotFoundException: Impossible to find
entry point 'freenect_init' in DLL 'freenect' at
freenect.KinectNative.freenect_init(IntPtr& context, IntPtr
freenectUSBCont
ext) and at freenect.KinectNative.InitializeContext() in kinect
\OpenKinect
-libfreenect-a7db0d8\wrappers\csharp\src\lib\KinectNative.cs:line 153
I have compiled with the X86 option. What should I do?

With VS2008 the dell name is freenect.dll but the name is
freenectdotnet with VS2010, is it normal ?

On 17 déc, 12:59, Joshua Blake <joshbl...@gmail.com> wrote:
> I got an interesting email this morning from a guy named Marcos Slomp. He
> had written libusbemu, a libusb 1.0 emulator (or wrapper) for libusb-win32
> (i.e. the libusb 0.1 API). He wrote it specifically with libfreenect in
> mind. After some collaboration I confirmed that we can use it with the
> existing libfreenect codebase and the usb_libusb10 abstraction layer.
>
> libusb-win32 -> libusbemu -> usb_libusb10 -> libfreenect
>
> Not only does it work, it works really well. This gives us performance
> equivalent to Zephod's code but with the libfreenect codebase!
>
> I did a bunch of work today and tonight and got all of this integrated and
> available in the unstable branch (https://github.com/OpenKinect/libfreenect/tree/unstable). I've also updated
> source files and cmake to play nicely with Visual Studio. You should be able
> to general Visual Studio projects (VS2010 tested here) and compile and run
> glview. (Seehttp://openkinect.org/wiki/Getting_Started#Dependenciesfor
> dependencies.)
>
> I've also tested lostincake's csharp_wrapper branch on top of this and with
> a few changes it works. I need to work with lostincake and Kyle to get the
> csharp_wrapper branch ready to integrate but it should be only another day
> or two.
>
> Thanks,
> Josh
>
> ---
> Joshua Blake
> Microsoft Surface MVP

> OpenKinect Community Founderhttp://openkinect.org

[Joshua Blake]

The VS2010 files are correct. lostincake is supposed to check the VS2008 files with monodevelop, but if they are different than the 2010 they should be updated. (I edited the VS2008 files by hand since I don't have VS08 installed, and must have forgotten to change the output assembly name. That also explains your error since P/Invoke is attempting to load the unmanaged freenect.dll assembly, which the project is overwriting.

 

Change the output file to freenectdotnet.dll then copy over the compiled freenect.dll from the regular libfreenect output directory.

 

Josh

[GUNNM]

I 've changed the assembly name before asking.
Without changing, the dll is not found.
With the change, the dll entrypoint freenect_init is not found.

> "Change the output file to freenectdotnet.dll then copy over the compiled freenect.dll from the regular libfreenect output directory. "

English is not my language... Do you ask me to rename freenect.dll to
freenectdotnet.dll ? Without changing the assembly name? Because I'm
sure that I will have the 'dll not found' error.
Which directory is the regular one? I'm working with the wrapper\csharp
\src\test\KinectDemo\VS2010 project, and I'm not sure to understand
what you call "libfreenect output directory."

[Joshua Blake]

GUNNM,

 

The freenectdotnet.csproj project (C# wrapper) should output freenectdotnet.dll. If it isn't then you should update the settings in the project properties.

The freenectdotnet.csproj output directory is /wrappers/csharp/bin. You need to compile the core freenect project (using cmake to create Visual Studio projects, and then Visual Studio to compile freenect.dll) and then copy freenect.dll to the /wrappers/csharp/bin directory.

 

Does this make sense?

 

Thanks,

Josh

[GUNNM]

OK, thx for your reply.
I assumed that the wrapper contains everything he needs.
I'll try with Cmake, but last time I used Cmake was before last
decade.
Thanks again.

[GUNNM]

With CMAKE, I have NOTFOUND error for LIBUSB_1_INCLUDE_DIR.
Do I really have to download libusb?
Sorry, I never understood how to use cmake.

[Juan Carlos del Valle]

Hello, I made a guide to compile as3-server in windows perhaps it will
help someone to succeed in compiling OpenKinect in windows.

The guide is available here:

http://www.as3kinect.org/guides/openkinect-win32-wrapper-guide/

Please feel free to copy/modify it to the OpenKinect wiki if you want.

Hope it helps.

[Joshua Blake]

Juan,

 

Awesome that'll be very helpful to people.

 

Small typo in this line:

Set LIBUSB_1_LIBRARY to point to the pthreadVC2.lib file inside the uncompressed freenect_win_deps.
(freenect_win_deps/lib/pthreadVC2.lib)

The variable name probably should be the THREADS_PTHREADS_WIN32_LIBRARY.

 

Thanks,

Josh

[Juan Carlos del Valle]

Thanks Josh,

I fixed the error in the guide.

Greetings,
JC

[GUNNM]

Thank for your guide.
I'm just asking myself what are the freenect_win_as3server.zip and
freenect_win_deps.zip ?
Any fork of yours ?
What needs to change in your guide for use with
https://github.com/OpenKinect/libfreenect/tree/unstable ?

On 20 déc, 22:11, Juan Carlos del Valle <jc.eki...@gmail.com> wrote:
> Thanks Josh,
>
> I fixed the error in the guide.
>
> Greetings,
> JC
>
>
>
>
>
>
>
> On Mon, Dec 20, 2010 at 3:03 PM, Joshua Blake <joshbl...@gmail.com> wrote:
> > Juan,
>
> > Awesome that'll be very helpful to people.
>
> > Small typo in this line:
>
> > Set LIBUSB_1_LIBRARY to point to the pthreadVC2.lib file inside the
> > uncompressed freenect_win_deps.
> > (freenect_win_deps/lib/pthreadVC2.lib)
>
> > The variable name probably should be the THREADS_PTHREADS_WIN32_LIBRARY.
>
> > Thanks,
> > Josh
>

> > On Mon, Dec 20, 2010 at 3:53 PM, Juan Carlos del Valle <jc.eki...@gmail.com>

> > wrote:
>
> >> Hello, I made a guide to compile as3-server in windows perhaps it will
> >> help someone to succeed in compiling OpenKinect in windows.
>
> >> The guide is available here:
>
> >>http://www.as3kinect.org/guides/openkinect-win32-wrapper-guide/
>
> >> Please feel free to copy/modify it to the OpenKinect wiki if you want.
>
> >> Hope it helps.
>

[Juan Carlos del Valle]

> Thank for your guide.
>
> I'm just asking myself what are the freenect_win_as3server.zip and

This is a zip with glview and as3-server binaries, you wont need it to
compile the source
>
> freenect_win_deps.zip ?

This zip has the libs and headers that are dependencies of OpenKinect
(glut, phtrheads, libusb)

>
> Any fork of yours ?
>
> What needs to change in your guide for use with

change
git clone git://github.com/imekinox/openkinect.git

to
git clone git://github.com/OpenKinect/libfreenect.git

then cd to libfreenect instead of openkinect

Everything else is the same... You can skeep this step

Check BUILD_AS3_SERVER

> https://github.com/OpenKinect/libfreenect/tree/unstable ?

[GUNNM]

Thank you very much.
I can say that I would never have succeeded to obtain the dll without
your guide.

Next step, compile the wrapper... I'll let you know.

[dyadica]

Once you’re done could you stick a copy up somewhere.... Please :)

As far as I can tell I have managed to get the freenect.dll compiled
ok as I can get glview.exe to function ? However when I try to use
freenetdotnet.dll I receive an exception (could not load type
'freenect.”anything” ' from assembly) when trying any call via a
simple forms app?

Both are next to each other in the bin is this right? or if things
have been done correctly should I only need freenetdotnet.dll?
I have obviously missed something somewhere around the compilation of
freenetdotnet.dll but have not a clue what this may be?

> > >> > > > > > Multitouch on Windows book:http://manning.com/blake- Hide quoted text -
>
> - Show quoted text -

[Joshua Blake]

Here's the binaries for freenect.dll and freenectdotnet.dll and the two sample apps:

http://dl.dropbox.com/u/7911268/freenectdotnet.zip

[dyadica]

Hi,

Josh: thanks for the binaries. Using them I managed to verify that
my .dll builds were ok in the end. Turns out I needed to modify the
program class to reflect those of the provided example source. I knew
I missed something simple. Once done all runs fine :)

Juan: as GUNNM said thanks for the guide, its invaluable, I too would
not have been able to build the .dll without it!

Right, now thats sorted, time to build a robot to facilitate Christmas
shopping :)

Cheers d

> > > - Show quoted text -- Hide quoted text -

[GUNNM]

http://gunnm.is.free.fr/freenectdotnet.zip
It is the same as Joshua, but it contain the freenect_sync.dll who
seems to be required by the examples.

I I told, I give you news : The wrapper run without any problem. The
difficult thing was only to obtain the dll.

But, everything is for openGL stuff.
What the way to use it with DirectX? I mean, I have trouble with :
texture = new Texture(rgbHandleBack.AddrOfPinnedObject()); // I
suppose it's only a
GL trick for back buffer?
I tried also :
texture = new Texture(this.kinect.VideoCamera.DataBuffer); // Who
seems to me much understandable.
(I'm in a managed code)
I will probably have the same problem (and worse) with the depth, but
one step at
the time. (I don't want you hate me...)

[mslinn]

My machine has Windows 7 64 bit. In the first step (Installing
Drivers) I have a problem.
If I right-click on Xbox_NUI_Camera.inf in the directory
'freenect_drivers\inf\xbox nui camera' and select 'install' then I get
the message:
"The INF file you selected does not support this method of
installation."

Same for the other two driver directories. What should I do?

Mike


On Dec 20, 12:53 pm, Juan Carlos del Valle <jc.eki...@gmail.com>
wrote:

[Marcos Slomp]

Hello,

Try this:

Start >> Control Panel >> System and Security >> System >> Device Manager

Locate the "Xbox NUI Motor" there, it should appear with a warning "!" symbol.

Right-click on it, and select "Update Driver Software..."

Then pick "Browse my computer for driver software"

Then click "Browse" and select the folder where "XBox_NUI_Motor.inf" is located

Click "Next"

If Windows prompt you that "this is not a certified driver", click order it to install anyway

After that, two new devices should appear under the Device Manager:

Repeat the process for the "XBox NUI Camera" and "XBox NUI Audio"

Hope this helps,

Marcos

[Monkey]

I managed to get the libusbemu project to compile and I'm able to
interface with the kinect quite well. However, I may have discovered a
bug with libusbemu. The libusb_get_device_list() function triggers a
"vector subscript out of range" assertion error when there is no
Kinect devices attached to the machine. Looks like I managed to narrow
down the problem to line 92 in libusbemu.cpp:

memcpy(*list, &vDevices[0], vDevices.size()*sizeof(struct
usb_device*));

Doing a direct memcpy on the std::vector list seems risky to me, as
there is no guarantee that the std::vector items will be allocated in
contiguous memory internally - at least to my knowledge that is. I
have replaced the above with the following code in my local copy of
OpenKinect:

for (int I = 0; I < vDevices.size(); I++)
{
(*list)[I] = (libusb_device*)vDevices[I];
}

This seems to have fixed the problem. Can any one else confirm?

Cheers.

[drew.m...@gmail.com]

On Fri, Dec 24, 2010 at 10:47 AM, Monkey <wise....@gmail.com> wrote:
> I managed to get the libusbemu project to compile and I'm able to
> interface with the kinect quite well. However, I may have discovered a
> bug with libusbemu. The libusb_get_device_list() function triggers a
> "vector subscript out of range" assertion error when there is no
> Kinect devices attached to the machine. Looks like I managed to narrow
> down the problem to line 92 in libusbemu.cpp:
>
> memcpy(*list, &vDevices[0], vDevices.size()*sizeof(struct
> usb_device*));
>
> Doing a direct memcpy on the std::vector list seems risky to me, as
> there is no guarantee that the std::vector items will be allocated in
> contiguous memory internally - at least to my knowledge that is. I
> have replaced the above with the following code in my local copy of
> OpenKinect:

Actually, std::vector does guarantee contiguous storage; see ISO 14882
2nd ed, 23.2.4 [lib.vector].

>   for (int I = 0; I < vDevices.size(); I++)
>   {
>      (*list)[I] = (libusb_device*)vDevices[I];
>   }
>
> This seems to have fixed the problem. Can any one else confirm?
>
> Cheers.

I'd guess the real issue is that vDevices[0] isn't valid with 0
devices in the vector, so when using a range-checked vector (which you
seem to have), that'll fail an assert, even though the memcpy would
wind up copying zero bytes (which would be completely safe). My
solution would look something like:

if(vDevices.size() > 0)

memcpy(*list, &vDevices[0], vDevices.size()*sizeof(struct usb_device*));

Your solution looks sane too. :)

-Drew

[Shawn2008]

Thanks for all the great help here! I'm having a little trouble
getting the code from the unstable branch to run consistently on my
Win32 platform. (I think I followed instructions from guide on
as3kinect.org correctly.)

Whether I use the precompiled binaries, or build them myself (VS C++
2010), I get rgb issues when running glview.exe.

Issue: On some runs I get live rgb, but it is jerky. Once in a while
it seems to run correctly and rbg is smooth. Most times I get no rbg
at all, just a grey frame. In the console, on the incorrect runs, I
see a stream of messages saying
[Stream 70] Lost xxx packets
[Stream 70] Lost too many packets, resyncing

Sometimes I get the same kinds of errors, but on "[Stream 80]".

Note that there are no issues I can see with the depth stream. Other
functions (tilt, led status) work correctly.

System info:
Windows 7 64bit on AMD Phenom II X6 1090T Processor at 3.72GHz

Any ideas on what might be causing this to provide smooth rbg on some
runs and not on others?

Thx,
Shawn

[Marcos Slomp]

The empty device list bug was already solved days ago.

What happened is that my fork changes were not merged in the unstable branch yet.

https://github.com/OpenKinect/libfreenect/pulls

I believe that Shawn's problem on sequence losses was also alleviated.

You can get a copy of the most recent changes on libusbemu directly from my fork:

https://github.com/slomp/libfreenect

Marcos

[Kyle Machulis]

Ok, this is now merged in. Sorry about the wait, been a busy week. 

Marcos, One thing to note: In order to keep things linear, I rebased out your merge since it was in two parts with a merge in the middle. There were a couple of conflicts I resolved that were minor, but this is now not going to match with your unstable head anymore. You should reset your unstable to look like ours, otherwise merging our unstable to yours is gonna end up in weird results. If you have any problems, just contact me on email or IRC, we'll get it sorted.

[Marcos Slomp]

Thanks Kyle,

I'll rebase my repository with yours now.

Sorry for any merging troubles!

[Mohd Kufaisal Mohd Sidik]

Hi all,

I just read from this post.. I read that Josh have two sample application using C#.NET.. Anyone here have that source code? Coz i just found the .exe only.. If anyone have that, hope u will share.. Please:)

Regards,
Kufaisal

[Monkey]

Some more interesting issues with the libusbemu implementation.

I'm setting up an video capture and a depth capture, but I'm getting a
lot of stalling with the callback functions. Sometimes the video
callback is not getting invoked at all (no matter what setting, RGB,
Bayer, or IR), meanwhile the depth is somewhat better, but still
stalling a lot.

I have set up a dedicated worker thread for handling freenect calls,
but still no joy.

I'm usually getting a flood of messages, such as:

"[Stream 70] Lost too many packets, resyncing..."

Interestingly the performance seems to pick up a bit after a machine
reboot.

[Marcos Slomp]

Which version of libusbemu are you using?

The one that was just updated today in the unstable branch or the old one of a week ago?

If you have not updated yet, please try to do so.

As for this new update, it is possible that CMake is not accounting for the latest changes in the libusbemu yet.

However, most changes happened in the libusbemu.cpp and some new header files were added (but no new source files), so CMake should still be able to produce a reliable project.

[Monkey]

I've cloned the latest version from the repo a few hours ago before my
last post. I did a complete rebuild. Is there something unique CMake
should take into account with the latest version?

I'm getting the same issues with glview.exe as well:

[Stream 70] Lost 49 packets

[Stream 70] Lost too many packets, resyncing...

I'm sitting on a XP-x64 machine, AMD X2. My application utilises the
CPU only at 30%, complete with OpenGL rendering, and kinect streaming.
glview.exe seems to push it to nearly 80%, but i see no difference
with playback performance.

On Dec 27, 2:33 am, Marcos Slomp <msl...@gmail.com> wrote:
> Which version of libusbemu are you using?
> The one that was just updated today in the unstable branch or the old one of
> a week ago?
>
> If you have not updated yet, please try to do so.
> As for this new update, it is possible that CMake is not accounting for the
> latest changes in the libusbemu yet.
> However, most changes happened in the libusbemu.cpp and some new header
> files were added (but no new source files), so CMake should still be able to
> produce a reliable project.
>

[Marcos Slomp]

Hmmm, that's interesting...

You are building based on the unstable branch, right?

Is it being built as a DLL or as a static library?

What is the URL of your cloned repository?

I'm afraid I can not properly answer your question yet...

You could try adapting your code to the Zephod's driver and check if it gives you better performance:

http://ajaxorg.posterous.com/kinect-driver-for-windows-prototype

[Monkey]

Hi. Yes, I'm using the unstable branch. The build was based on this
repo:
https://github.com/slomp/libfreenect

I am building as a DLL, and using the external dependency pack
provided by as3kinect (it has glut, phthreads, libusb):
http://as3kinect.org/distribution/win/freenect_win_deps.zip

Re. Zephod's driver, I might eventually give it a shot, but the demo
included is not particularly stable either. Its fast, but I see a lot
of visual corruption in the streams. Now I think about it, this could
indicate there is something more fundamental going on, probably at the
driver level.

On Dec 27, 3:51 am, Marcos Slomp <msl...@gmail.com> wrote:
> Hmmm, that's interesting...
> You are building based on the unstable branch, right?
> Is it being built as a DLL or as a static library?
> What is the URL of your cloned repository?
>
> I'm afraid I can not properly answer your question yet...
> You could try adapting your code to the Zephod's driver and check if it
> gives you better performance:http://ajaxorg.posterous.com/kinect-driver-for-windows-prototype
>

[Marcos Slomp]

Alright, if the demo that comes with Zephod's driver is not running well, it must be for the same reasons libfreenect/libusbemu is not.

I guess you don't need to waste your time porting your code to Zephod's driver because it would yield to the same issues.

Here are my thoughts about it:

1) when the application loads the libfreenect DLL, the DLL is being placed at some process priority class that is lower than the normal; this would cause the thread scheduling to give less priority to the libusbemu threads, which would then cause the stream sequence losses

2) there are other processes threads running in background at high-priority which would also compromise the thread scheduling

I would go for the 2nd one. Check if there are resource-consuming applications running in your computer (anti-virus, etc).

A good way to check so is through the "Resource Monitor". You can access it within the Performance Tab of the Task Manager.

Once opened, search in the list for something called "System Interrupts" which should be indicating close to no CPU and with no internal threads.

If that is not the case, something is leeching resources from your computer and compromising libfreenect and/or Zephod's driver.

[Joshua Blake]

There could also be a simpler explanation. If there are other USB devices on the same USB bus then it will compete for bandwidth and drop packets. I've also heard some reports of some lower-end, low quality computers with USB 2.0 ports that don't actually have the full bandwidth available, perhaps due to sharing with internal hardware components.

 

Sharing with keyboard or mouse should be ok but not any high bandwidth devices like webcam, cd drive, or hard disk.

 

Josh

[Marcos Slomp]

Good point.

And be sure that freenect_process_events() has its own thread (like in the glview example) and that freenect_process_events() is being called constantly (i.e., that there are no Sleep() around the process event loop)

If everything fails, there is a dirty trick you could try:

put the following as the first statement of your program, right after the main()

SetPriorityClass(GetCurrentProcess(), REALTIME_PRIORITY_CLASS);

This will place your program at the highest possible priority and any thread that it spawns should follow this extreme scheduling policy.

Be warned that this can render other activities in your system completely idle! =P

[Monkey]

Hello all, thank you for your responses.

Out of curioustity, I have decided to add support to Zephod's driver
in my code, which is a statically linked library. The problem still
persisted with Zephod's driver as expected: It renders quicker
compared to the libusbemu version, but I see a lot of video & depth
frame corruption. Interestingly, the corruption happens more
systematically in the video stream compared to the depth stream. This
behaviour is consistent with Zephod's demo.

As for the libusbemu version, one significant problem I'm experiencing
is that my machine locks-up (i.e. needs a hard reset) quite
frequently, even with the console enabled. This is especially true
when I'm executing the Release build of my project. When libusbemu
does not freeze, the video callback returns only one or two frames
during the lifetime of the program, if I'm lucky, and the depth
callback runs around 5 FPS, but very sporadically.

As for possible USB problems, the Kinect device does not share a port
with other devices. I have used a different USB port (running on a
separete controller) just to make sure, I see no improvement.

According to ProcessExplorer, the CPU utilisation for my program is
about 60% with Zephod's version. System Interupts barely registers a
percentage, while Deferred Procerue Calls sits around 6% at most. The
system utilisation of my application process is less than 10%.

The CPU utilisation with libusbemu is around 45%. Again, System
Interupts is neglibile, Deferred Procerue Calls is at 10% at most.
System utilisation of my application about the same as earlier.

The application is getting a steady 21.6 MB/s stream from the USB
device, both for Zephod's and libusbemu.

I do not have anti-virus applications running. Typical services (apart
from OS services) I have running is Apache, MySQL, AMD CodeAnalyst,
ProcessExplorer. Applications that run atomatically per user account
is the sound driver, NetMeter, and Apache monitor. All the other stuff
is Visual Studio and its spawned processes, Google Talk, firefox,
TextPad, Total Commander. All up, about 50 processes are running on
this machine, nealrly bare bone.

I have decided to use AMD CodeAnalyst to see how different modules are
utilising my resources. Here is the top 5 most CPU intensive modules
when running Zephod's driver:

Module Name 64-bit Timer samples
kfx.exe 40.84
AmdK8.sys 1 27.13
ntoskrnl.exe 1 5.94
nvogl32.dll 3.81
USBPORT.SYS 1 2.76

AMD CodeAnalyst with libusbemu:

Module Name 64-bit Timer samples
AmdK8.sys 1 28.06
freenect.dll 18.79
ntoskrnl.exe 1 12
kfx.exe 8.12
USBPORT.SYS 1 6.08

Nothing unusual there. kfx.exe is my appliction, AmdK8.sys is my
processor driver that handles the Idle process (power management) on
my system. The differences you see in my application's CPU usage is
probably be due to the way Bayer-to-RGB decoding is handled by
Zephod's code and freenect respectively.

I'm using the Qt 4 SDK for my application. I've set up a seperate
thread class for all the freenect stuff, which enters a tight loop
that polls freenect_process_events() without delay. Something like
this:

while (!Exit && Update())
{
if (VideoUpdated()) {SignalUpdateVideo();}
if (DepthUpdated()) {SignalUpdateDepth();}
}

... where Update() is just an alias for freenect_process_events(), and
all the other things in the loop involves simple flag checking and
signalling with negligible overhead. Running the thread (or the entire
process) at different priorities makes no impact, as far as the
problem is concerned.

Btw, I'm building with Visual C++ 2010 Express, using the Windows 7.1
SDK platform toolset.

On Dec 27, 1:20 pm, Marcos Slomp <msl...@gmail.com> wrote:
> Good point.
>
> And be sure that freenect_process_events() has its own thread (like in the
> glview example) and that freenect_process_events() is being called
> constantly (i.e., that there are no Sleep() around the process event loop)
>
> If everything fails, there is a dirty trick you could try:
> put the following as the first statement of your program, right after the
> main()
> SetPriorityClass(GetCurrentProcess(), REALTIME_PRIORITY_CLASS);
> This will place your program at the highest possible priority and any thread
> that it spawns should follow this extreme scheduling policy.
> Be warned that this can render other activities in your system completely
> idle! =P
>

[Joshua Blake]

Interesting investigation. Do you have any other computer of a different model/manufacturer that you could try your application on?

 

Josh

[Marcos Slomp]

Interesting indeed...

Do you happen to have Linux available as well?

Would be interesting to check if it performs bad there too.

I am quite clueless on what could be the reason behind this behavior.

A last thing you can try, besides Linux, is to stream only one thing, say video-only or depth-only.

[Marcos Slomp]

Out of curiosity, try adding these in your code:

  const usb_version* version = usb_get_version();

  fprintf(stdout, "libusb-win32 version %d.%d.%d.%d (driver %d.%d.%d.%d)\n",

    version->dll.major, version->dll.minor, version->dll.micro, version->dll.nano,

    version->driver.major, version->driver.minor, version->driver.micro, version->driver.nano);

Maybe the libusb-win32 lib and/or libusb-driver you are using have some problem.

Mine is: libusb-win32 version 1.2.2.1 (driver 1.2.2.1)

[Antonio Ospite]

On Mon, 27 Dec 2010 18:24:50 +0900
Marcos Slomp <msl...@gmail.com> wrote:

> Interesting indeed...
>

What's interesting? Can we please limit top-posting when possible
and trim the messages removing old/unrelated parts? By doing so even a
casual reader can follow the currently interesting part of the thread
from a subset of it.

> Do you happen to have Linux available as well?
> Would be interesting to check if it performs bad there too.
>
> I am quite clueless on what could be the reason behind this behavior.
> A last thing you can try, besides Linux, is to stream only one thing, say
> video-only or depth-only.
>

Thanks,
Antonio

--
Antonio Ospite
http://ao2.it

PGP public key ID: 0x4553B001

A: Because it messes up the order in which people normally read text.
See http://en.wikipedia.org/wiki/Posting_style
Q: Why is top-posting such a bad thing?

[Monkey]

> Do you have any other computer of a different
> model/manufacturer that you could try your application on?

Unfortunately not at the moment. I have a Mac mini, but it has no
Windows partition. That said, I am planning to build on OS X in the
near future.

> Do you happen to have Linux available as well?
> Would be interesting to check if it performs bad there too.

On the same machine? Not at the moment. Well, Ubuntu guest on Windows
version of VirtualBox - which I actually tried and predictably failed.
Virtual USB controllers in a VM is probably asking too much at the
moment, I'm afraid. I might give Ubuntu live CD as shot, assuming
building under that environment is actually possible.

> Maybe the libusb-win32 lib and/or libusb-driver you are using have some
> problem.
> Mine is: libusb-win32 version 1.2.2.1 (driver 1.2.2.1)

My version is: libusb-win32 version 1.2.2.0 (driver 1.2.2.0)

Where did you get your version? The sourceforge site only has 1.2.2.0
as latest for the lib.
http://sourceforge.net/projects/libusb-win32/files/

A manual check of the NUI drivers in the Device Manager also reports
version 1.2.2.0 for all three of them. They should be the same ones I
cloned from the repo.

[Marcos Slomp]

> Where did you get your version? The sourceforge site only has 1.2.2.0
> as latest for the lib.

The 1.2.2.1 is not a libusb-win32 release, it is a snapshot; the latest one is:

http://sourceforge.net/projects/libusb-win32/files/libusb-win32-snapshots/20101117/

Once you download it, you may have to manually install (aka copy-and-paste) the driver file 'libusb0.sys' at C:\Windows\system32\drivers.

Maybe the 'libusb0.dll' as well into C:\Windows\system32.

I remember adding a note into 'libfreenect\platform\windows\libusb10emu\libusb.h' regarding this.

However, it was merely because of the names defined in the <usb.h> header, and not because of performance reasons.

But it is worth a try =)

[Monkey]

Ah yes, silly me. Thanks, I found it. What I ended up doing is
generate new .inf files for the updated libusb0.sys & libusb0.dll
using the inf-wizard.exe tool provided with libusb-win32. Then
reinstalled the driver via the Device Manager. usb_get_version() is
giving me version 1.2.2.1 now. But still no cigar though. :(

Tomorrow I will have look at other possible causes, namely power
management (CPU throttling), which is known to cause timing and
synchronisation problems in some software. I'm not sure how relevant
that would be for freenect/libusbemu, as I'm not familiar enough with
its internals.

[Marcos Slomp]

Sorry for not being able to solve your issue so far, but it is a peculiar one... hehehe

Here are my guidelines:

1) test under Linux

2) test in another machine

3) test in a 32bit Windows

Good luck and please report back.

Feel free to contact me directly so that we don't bloat this topic! =P

[Monkey]

Update: I have some encouraging results. I used the Ubuntu live CD to
build. It ran freenect effortlessly on the same machine/hardware/port.
This means the device itself is OK, and so are the USB ports on the
motherboard.

With hardware issues ruled out, I decided to manually purge all .inf
and .pif files related to libusb from the Windows directory tree. I
have reinstalled the libusb drivers once again to make sure.

Next, I disabled the dynamic frequency throttling for the CPU in the
BIOS (AMD Cool'n'Quiet), so that the machine ran at a fixed 2.3 GHz.
Looks like this improved the situation quite significantly - or at
least the streaming is more reliable now. I see the occasional image
tearing now and then, or experience a dropped frame, but for the most
part I'm getting a solid 30 FPS stream (both video and depth). Note,
I'm only testing libusbemu builds, not Zephod's verison.

I have enabled frequency throttling again, and the corruption became
more frequent, although not as bad as it was earlier. The machine
dynamically changes the clock from 1 GHz to 2.3 GHz, approximately in
100 ~ 200 MHz steps, depending on CPU load. I used wcpuclk.exe utility
to monitor the clock (part of WCPUID http://www.h-oda.com/). The video
corruption events seem to correlate with frequency change events,
generally when the frequency steps 1.8 -> 2 -> 2.3 GHz and back while
my application is running. That said, I can't claim absolute
confidence with these observations, because my clean-up efforts with
driver could be factor as well.

Not sure how useful this information is to you guys, but it might be
worthwhile to investigate other AMD rigs with CPU throttling enabled
and see if the problem can be replicated on there. Most laptops have
throttling as well; those who have one could check on those machines.

[Marcos Slomp]

Thanks for sharing this report!

It is good to hear that you could make some progress after all.

I would imagine that frequency throttling would always force the CPU to stay at maximum frequency due to the internal critical-time threads.

However, in the libusbemu.cpp, after every iteration of ReapThreaded() I call a Sleep(1) - aka, thread yield - just to give some time for other threads to run, if the system judges it necessary. Maybe this yield ends up being enough to signal the throttling monitor.

You could try commenting out that Sleep() call.

Additionally, inside ReapThreadProc(), just at the beginning of the while-loop, there is a synchronization variable being waited:

listTransfers.Head()->libusb.dev_handle->dev->ctx->processing.Wait();

You could comment out this as well.

If you do so, those critical-time threads will be always running without any external orchestration.

No guarantees that these changes will give much - if any - improvement because later on usb_reap_async_nocancel() could put the thread to sleep while waiting for USB I/O anyway.

[Monkey]

I see not much difference (re corruption) with either Sleep() and
the ...processing.Wait() lines commented out. CPU utilisation went up
though, around 70% but the frequency throttling behaviour remained
more or less the same. I'm not sure if removing thread yielding is
worth the extra CPU load.

When the problem is really bad (i.e. lots for frame losses, and seeing
only 5 FPS), the throttling behaviour is quite strange. It tends to
oscillate between 1 GHz and 1.8/2.3 GHz sporadically, almost in sync
with the depth buffer stalling. The RGB video is practically non-
existent in those situations. There is some kind of a detrimental
feedback effect with the library's CPU utilisation and the frequency
throttling. It seems by the time the CPU driver responds to the
demands of libusb, it's too late, and everything stalls. The cycle
repeats over and over. When I'm lucky, everything is "just right" and
this detrimental feedback effect does not manifest itself.

Out of curiosity, how is the incoming data handled from the device?
Does libusb actually poll for new packets? Or is there some kind of
interrupt driven callback feature, where packets can be processed
whenever they arrive? I'm not clued-in enough tell from the existing
code.

[Marcos Slomp]

> I see not much difference (re corruption) with either Sleep() and
> the ...processing.Wait() lines commented out.

Yes, as I've mentioned, significant improvements were not to be expected.

> Out of curiosity, how is the incoming data handled from the device?
> Does libusb actually poll for new packets? Or is there some kind of
> interrupt driven callback feature, where packets can be processed
> whenever they arrive?

When 'usb_reap_async_nocancel()' - this is from libusb-win32 - is called, it will poll the device-mapped file using a Win32 Event variable.

This Event variable will either return immediately with the data, if it is available, or after some timeout expires (the timeout comes from the second parameter of usb_reap_async_nocancel()).

What this translates to behind the scenes (within the Win32 I/O kernel) is that if there is nothing to get from the file, the callee will sleep until data arrives our the timeout expires, which will also put the caller thread to sleep.

[Marcos Slomp]

Monkey:

Just for the sake of it, try opening the Windows Registry Editor and go to:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Class\{36FC9E60-C465-11CF-8056-444553540000}\0000

In there, see if there is a value called "IdleEnable" set to 1.

(there are other folders there as well, 0001..., and it might be a good idea to check them all).

This may be the answer behind CPU frequency being changed by the throttling monitor even though a high-speed device is being used.

[Marcos Slomp]

BTW, if it is there, remove it and RESTART before running your Kinect app =)

[Monkey]

Hello, I have checked sub-folders 0000 to 0046 and none of them had
that registry key entry.