Matlab IMAQ Adaptor for Kinect

[Gus Lott]

Hi everyone,

This is my first post, but I've been watching and enjoying the growth
of this codebase. I've never really contributed to an open source
project before. So here it goes:

I've created a Matlab Image Acquisition Toolbox Adaptor for the Kinect
Depth Sensor. This software device allows for a parallel thread to
run within the matlab process and pump frames into the matlab
workspace at 16-bit depth and 30 frames per second. It's built based
on their included c++ adaptor kit. It requires the latest version of
Matlab (may work on older versions) and a license for the Image
Acquisition toolbox. If you're at a university, these are most likely
essentially free to you and/or your lab. I already had a skeleton
that I'd built for other projects and this was quick mod of that
code. Zephod's win32 kinect library is real easy to use.

Here's the youtube vid demoing my adaptor.
http://www.youtube.com/watch?v=njstg3jnCg0

In my opinion, this is the most efficient way to connect the kinect
data stream into Matlab. It is, however, difficult to create two
simultaneous streams from a single device (i.e. the RGB and the 11-bit
depth images), so I'm currently only streaming in the depth
information.

A few options might be to stack the images together in one large frame
that is 640x(480*5) for 8-bit RGB and 16-bit depth. But currently,
the IMAQ toolbox only supports a 1 or 3 plane frame and the planes
have to be a single fixed bit depth. You can have multiple "sources"
for a given hardware, but seem to only be able to pull from one source
at a time.

It also provides tilt motor control. I've thrown the compiled adaptor
dll, installation instructions, and the entire codebase up on a
quickly thrown together sourceforge site here:

http://matkinect.sourceforge.net/

Hopefully this will enable some engineers/scientists who don't
otherwise have C/C++ capabilities to jump into some applications with
kinect. It should also scale with an arbitrary number of kinects
creating multiple threaded interfaces into Matlab without burdening
the command line.

The code is a Visual Studio 2008 C++ project the leverages Zephod's
win32 kinect driver library downloaded off the OpenKinect site
(version 1.5).

If you have any questions, post here or shoot me an email.

Cheers,
-Gus Lott

[Jesús Torres]

This is a great idea. In my lab there are many people working with
Matlab and to have a IMAQ adaptor would be the easiest way to use
Kinect with Matlab.

If I get some time and the last version of IMAQ, I could help adding
support to Matlab for Linux.

[whatnick]

Great work,

I believe there development using the libfreenect c-api and mex files
as well. With that you will not require the Image Acq, the data should
be returned straight to registered matlab functions via callbacks.

That being said, in my lab Matlab + Windows rules and I have grown
rather fond of Zephod's bindings for working with Windows.

Cheers,

Tisham.

[Viliam Rapcan]

Excellent work! I was trying to get the Kinect output to Matlab
myself, but I didn't succeed.

I was trying to register the adaptor with the 'imaqregister', but I'm
getting this error:

imaqregister('C:\Kinect\matlab2\matkinect.dll')
??? Error using ==> imaqregister at 97
'C:\Kinect\matlab2\matkinect.dll' is not a valid adaptor. Was the
shared library created using the Image Acquisition
Toolbox Adaptor Kit?

Any idea why? I'm using 64bit Matlab, could this be the problem?

Thanks.

On 24. Nov, 00:51 h., Gus Lott <gusl...@yarcom.net> wrote:
> Hi everyone,
>
> This is my first post, but I've been watching and enjoying the growth
> of this codebase.  I've never really contributed to an open source
> project before.  So here it goes:
>
> I've created a Matlab Image Acquisition Toolbox Adaptor for the Kinect
> Depth Sensor.  This software device allows for a parallel thread to
> run within the matlab process and pump frames into the matlab
> workspace at 16-bit depth and 30 frames per second.  It's built based
> on their included c++ adaptor kit.  It requires the latest version of
> Matlab (may work on older versions) and a license for the Image
> Acquisition toolbox.  If you're at a university, these are most likely
> essentially free to you and/or your lab.  I already had a skeleton
> that I'd built for other projects and this was quick mod of that
> code.  Zephod's win32 kinect library is real easy to use.
>

> Here's the youtube vid demoing my adaptor.http://www.youtube.com/watch?v=njstg3jnCg0

[Gus Lott]

Yep, this was built versus the Matlab IMAQ adaptor kit 32-bit windows
libraries. There are 64-bit libraries, but there are only a few
supported pieces of hardware in this space. If you have a 64-bit
license, you should be able to grab the 32-bit version as well.

In theory, the IMAQ toolbox is supported on all platforms (linux, Mac,
Windows in 32/64-bit). This new broad support only came with the
latest 2010b release. I haven't played with it much.

[Viliam Rapcan]

Thanks. I work with 64bit Matlab because I work with big vectors/
matrices which 32bit version isn't able to load. Not sure if I can
have both versions of Matlab on one computer. Is there any tutorial
how to make these DLLs for the Image Acquisition toolbox? I could try
to make dll for 64bit Matlab.

> > Any idea why? I'm using 64bit Matlab, could this be the problem?- Skryť citovaný text -
>
> - Zobraziť citovaný text -

[Jesús Torres]

2010/11/24 Viliam Rapcan <viliam...@gmail.com>:

> Thanks. I work with 64bit Matlab because I work with big vectors/
> matrices which 32bit version isn't able to load. Not sure if I can
> have both versions of Matlab on one computer. Is there any tutorial
> how to make these DLLs for the Image Acquisition toolbox? I could try
> to make dll for 64bit Matlab.

The code is a Visual Studio 2008 C++ project. I don't work with Visual
Studio but I guess that you only must change the project target to x64
to make a dll for 64bit Matlab.

If you wouldn't have a full Visual Studio 2008 you could download and
install Visual Studio 2008 Express, Microsoft x64 Compilers and Tools
and Microsoft Windows Software Development Kit as is explained here:

http://www.mathworks.com/support/solutions/en/data/1-6IJJ3L/?solution=1-6IJJ3L

I never have made that so I don't know if that works.

Good luck.

[riccardo klinger]

I`m also interested in exploring data in Matlab but I´m also stucked
at the x64 border...
are their any workarounds (I'm a total newbie to programming C/C++ or
Python by the way...)

On Nov 24, 7:16 pm, Jesús Torres <aplatan...@gulic.org> wrote:
> 2010/11/24 Viliam Rapcan <viliam.rap...@gmail.com>:

>
> > Thanks. I work with 64bit Matlab because I work with big vectors/
> > matrices which 32bit version isn't able to load. Not sure if I can
> > have both versions of Matlab on one computer. Is there any tutorial
> > how to make these DLLs for the Image Acquisition toolbox? I could try
> > to make dll for 64bit Matlab.
>
> The code is a Visual Studio 2008 C++ project. I don't work with Visual
> Studio but I guess that you only must change the project target to x64
> to make a dll for 64bit Matlab.
>
> If you wouldn't have a full Visual Studio 2008 you could download and
> install Visual Studio 2008 Express, Microsoft x64 Compilers and Tools
> and Microsoft Windows Software Development Kit as is explained here:
>

> http://www.mathworks.com/support/solutions/en/data/1-6IJJ3L/?solution...

>
> I never have made that so I don't know if that works.
>
> Good luck.
>
>
>
> > On 24. Nov, 15:47 h., Gus Lott <gusl...@yarcom.net> wrote:
> >> Yep, this was built versus the Matlab IMAQ adaptor kit 32-bit windows
> >> libraries.  There are 64-bit libraries, but there are only a few
> >> supported pieces of hardware in this space.  If you have a 64-bit
> >> license, you should be able to grab the 32-bit version as well.
>
> >> In theory, the IMAQ toolbox is supported on all platforms (linux, Mac,
> >> Windows in 32/64-bit).  This new broad support only came with the
> >> latest 2010b release.  I haven't played with it much.
>
> >> On Nov 24, 8:43 am, Viliam Rapcan <viliam.rap...@gmail.com> wrote:
>
> >> > imaqregister('C:\Kinect\matlab2\matkinect.dll')
> >> > ??? Error using ==> imaqregister at 97
> >> > 'C:\Kinect\matlab2\matkinect.dll' is not a valid adaptor.  Was the
> >> > shared library created using the Image Acquisition
> >> > Toolbox Adaptor Kit?
>

> >> > Any idea why? I'm using 64bit Matlab, could this be the problem?- Skry» citovaný text -
>
> >> - Zobrazi» citovaný text -

[Jeremi]

The imaq adapter is crashing my Matlab. I am able to register the
third party adapter KinectNUI (matkinect-0). When I select the format
DepthImage (default)... Matlab crashes. Only error message is that the
program has stopped responding. I am running:

MATLAB Version 7.10.0.499 (R2010a) 32-bit
Operating System: Microsoft Windows 7 Version 6.1 (Build 7600) 32-bit
Java VM Version: Java 1.6.0_12-b04 with Sun Microsystems Inc. Java
HotSpot(TM) Client VM mixed mode
Image Acquisition Toolbox 3.5 (R2010a)

I would very much appreciate your thoughts and suggestions to resolve
this issue. My goal is access the depth sensor's signal (numbers not
just the image). Thanks again!

Jeremi

On Dec 8 2010, 7:36 am, riccardo klinger