Kinect IMAQ Adaptor for Matlab
Gus Lott
I just put up an open source Image Acquisition Toolbox adaptor for Matlab that interface the new Microsoft Kinect USB depth camera (controller for the XBOX 360). The adaptor allows for 30Hz streaming of 11-bit depth data at 640x480 from an arbitrary number of Kinect sensors plugged into your Windows 32 based machine.
I built this primarily for the OpenKinect community but thought I'd post it here since it seemed relevant.
Here's the youtube vid demoing my adaptor.
http://www.youtube.com/watch?v=njstg3jnCg0
In my opinion, this is the most efficient way to connect the kinect data stream into Matlab. It is, however, difficult to create two simultaneous streams from a single device (i.e. the RGB and the 11-bit depth images), so I'm currently only streaming in the depth information.
A few options might be to stack the images together in one large frame that is 640x(480*5) for 8-bit RGB and 16-bit depth. But currently, the IMAQ toolbox only supports a 1 or 3 plane frame and the planes have to be a single fixed bit depth. You can have multiple "sources" for a given hardware, but seem to only be able to pull from one source at a time.
It also provides tilt motor control. I've thrown the compiled adaptor dll, installation instructions, and the entire codebase up on a quickly thrown together sourceforge site.
Hopefully this will enable some engineers/scientists who don't otherwise have C/C++ capabilities to jump into some applications with kinect. It should also scale with an arbitrary number of kinects creating multiple threaded interfaces into Matlab without burdening the command line.
The code is a Visual Studio 2008 C++ project the leverages Zephod's win32 kinect driver library downloaded off the OpenKinect site (version 1.5).
Cheers,
-Gus Lott
Paul
"Gus Lott" <lottg....@janelia.hhmi.org> wrote in message <icho38$g7g$1...@fred.mathworks.com>...
Javier
Thanks
Javier
"Gus Lott" <lottg....@janelia.hhmi.org> wrote in message <icho38$g7g$1...@fred.mathworks.com>...
Gus Lott
I'll probably just end up creating a custom getter function for a property that will hand a depth array back to matlab. It's still not easy to create array properties of an IMAQ adaptor, however, so it may end up with the grayscale depth image unraveled into a long vector that needs to be reshaped. Not ideal.
I wish mathworks would just provide a way of comprehensively integrating multithreaded code into the workspace.
"Paul " <paul.s...@uea.ac.uk> wrote in message <icobl7$emj$1...@fred.mathworks.com>...
Jeremi
Hi Gus,
Thanks so much for building this adapter. I have successfully installed the OpenKinect drivers and am able to register the adapter. Selecting the "DepthImage" format causes my Matlab to crash. Any thoughts on what is going on? The only error message tells me the program has stopped responding.
I am running:
Matlab 7.10.0.499 (R2010a) 32-bit
Image Acquisition Toolbox 3.5 (R2010a)
Windows 7 Home 32-bit
I am able to open some .exe examples from OpenKinect but am looking to capture the data from the depth cameras. Any help would be appreciated.
Jeremi
Husam Aldahiyat
I'm buying a Kinect now.
Deborah Aks
"Gus Lott" wrote in message <icho38$g7g$1...@fred.mathworks.com>...
Kalla
Gus Lott
It could work separately (either/or). I wish Matlab would setup a more generic way to pump data into the workspace from parallel threads within the matlab process. Would really kick their capacity for application development into gear. It'd be very similar to the DAQ and IMAQ toolboxes.
Kalla
> It seems like it is not going to be possible to stream both the depth and the RGB images simultaneously because of the way that the IMAQ toolbox handles connections and data sharing. A videoinput object appears to be a stand-alone entity and I don't see how I can create a connection to the kinect and then stream it into 2 videoinput objects. That's what seems to be required to pump both a 16bit grayscale image and a 24 bit RGB image into matlab via this method.
>
> It could work separately (either/or). I wish Matlab would setup a more generic way to pump data into the workspace from parallel threads within the matlab process. Would really kick their capacity for application development into gear. It'd be very similar to the DAQ and IMAQ toolboxes.
Have you checked it out?
http://www.mathworks.com/matlabcentral/linkexchange/links/2718-kinect-for-matlab
I think this can stream both depth and the RGB images simultaneously.
Gus Lott
>
> Have you checked it out?
> http://www.mathworks.com/matlabcentral/linkexchange/links/2718-kinect-for-matlab
>
> I think this can stream both depth and the RGB images simultaneously.
Streaming? Mex files can't call back into the matlab process (at least not easily or supported by mathworks), so this is a polled interface that will block the command line when called. I didn't look in detail, but that limitation is why I went with an IMAQ adaptor. Matlab has created DAQ and IMAQ toolbox interfaces in order to manage, in a thread safe manner, the passing of data from a separate thread to the workspace. The IMAQ adaptor also gives you callbacks that you can respond to asynchronously in a larger application.
But it may be that they've worked around that in the above link. Cool stuff.
khaled emam
>> imaqregister('F:\Documents\MATLAB\matkinect.dll')
??? Error using ==> imaqregister at 89
'F:\Documents\MATLAB\matkinect.dll' is not a valid adaptor. Was the shared library
created using the Image Acquisition Toolbox Adaptor Kit?
thats what i got. if you could tell me another method to add the kinect adaptor it would be highly appreciated.
thanks
am working on matlab 2010a and 32-bit processor