Kinect for Java / Processing

4,603 views
Skip to first unread message

Daniel Shiffman

unread,
Nov 16, 2010, 10:55:41 AM11/16/10
to openk...@googlegroups.com
Hello all,

Just wanted to introduce myself and say hello. I'm working on a port
of OpenKinect for Java. Currently I did a really quick test with JNI
and ran the results in Processing (on Mac OSX only)

http://www.shiffman.net/2010/11/14/kinect-and-processing/

If you aren't familiar with Processing it's an open source programming
language and environment for images, animations, and interactions and
is built with Java.

http://www.processing.org/

I'm under a big deadline for another project that is happening
December 3rd, but my month of December is hopefully going to be
dedicated to Kinect testing and building out Java libraries for it.

Advice, comments, etc. are welcome!

Thanks for everyone's wonderful hard work and inspiring spirit!

Dan

Jayraj Jog

unread,
Nov 16, 2010, 10:58:11 AM11/16/10
to openk...@googlegroups.com
That's awesome!

Thanks and Regards,
Jayraj Jog
MS Digital Media Student
Georgia Institute of Technology
http://jayraj.net

Arthur Wolf

unread,
Nov 16, 2010, 11:47:34 AM11/16/10
to OpenKinect
So NOW I can buy a kinect :)

Nink

unread,
Nov 16, 2010, 11:52:05 AM11/16/10
to OpenKinect
thx Anyone get this to work on their mac ?

Florian Lier

unread,
Nov 16, 2010, 2:07:49 PM11/16/10
to openk...@googlegroups.com
Hello all,

Great - I also thought about implementing that!
Maybe I get in touch with python then ;)

> Thanks for everyone's wonderful hard work and inspiring spirit!
>
> Dan
>

Cheers,
Florian

Henkka

unread,
Nov 17, 2010, 12:29:10 AM11/17/10
to OpenKinect


On 16 marras, 17:55, Daniel Shiffman <daniel.shiff...@nyu.edu> wrote:
> Hello all,
>
> Just wanted to introduce myself and say hello.  I'm working on a port
> of OpenKinect for Java.  Currently I did a really quick test with JNI
> and ran the results in Processing (on Mac OSX only)
>
> http://www.shiffman.net/2010/11/14/kinect-and-

Hello,

I just tried it and it worked 'out of the box'! I way up and running
about 2 minutes after I had taken the Kinetic out of it's packing box!
Just downloaded your Processing library and started to use it.

Awesome work - thanks everyone involved !


I'm thinking of scanning both small parts and full-size car engine-
bays, as in my other hobby I convert cars and motorcycles to electric
drive. So this will come handy determining the size of mounting-space
available under the hood.

I'm thinking of point-cloud->mesh->CAD would be the path for me.
Smaller parts can be also printed with my Makerbot.

I'll try to first look at getting the point-cloud out of Processing.

One thing I have seen with commercial 3D scanners is that they use few
small 20mm balls in the scene as calibration & orientation points.
They just 'throw' them to the scene, here and there, and use those
points to align multiple scans, takrn from different angle. That could
be worth investigating too.

With very best regards from Finland,

Henry 'Henkka' Palonen

badoli

unread,
Nov 17, 2010, 3:13:50 AM11/17/10
to OpenKinect
Great stuff! Will there be a windows build?

jque...@gmail.com

unread,
Nov 17, 2010, 7:01:17 AM11/17/10
to openk...@googlegroups.com
Is it possible to use the .jar which comes with the extension in
conventional Java?
Thank you very much!

---------------------------------------------------------
Javier Quevedo Fernández
jque...@gmail.com
Spain +34 618 420 028
Netherlands +31 684888066
---------------------------------------------------------

Daniel Shiffman

unread,
Nov 17, 2010, 10:21:33 AM11/17/10
to openk...@googlegroups.com
> Great stuff! Will there be a windows build?

I'm hoping to dedicate the month of december to building out the Java
library for both mac and windows so stay tuned, I will announce any
new releases here. This is all happening too fast for me, darn these
other work and teaching commitments, all I want to do is kinect!

Dan

Daniel Shiffman

unread,
Nov 17, 2010, 10:22:51 AM11/17/10
to openk...@googlegroups.com
Yes, the download has two files:

shiffman.net/p5/kinect.zip
kinect.jar
kinectNative.jnilib

You can simply add the jar to your classpath when compiling and you'll
be all set! I'll publish the source, etc. to github soon, sorry for
being slow here. . .

Dan

Henkka

unread,
Nov 17, 2010, 4:09:45 PM11/17/10
to OpenKinect
Hello,

I made an small point-cloud demo, using Daniel's great library and
Kyle's great ThreePhase-application as an example of calculations
needed. This seems just about the minimum that is needed to get simple
real-time point-cloud demo up and running in Processing.

Some video at http://www.youtube.com/watch?v=4Cvqw2QS3Yk

If anyone wants to grab the code used, it can be found from
http://yty.net/h/simplekinect.pde

Thanks Daniel & Kyle for sharing these great pieces of software !

By the way - Daniel - any pointers where should I look for different
function calls that your library makes available ? I would be
interested in real depth information. For the purposes of my demo, I
just converted the color to grayscale and used it as an depth value.
Not quite the optimum solution, but as I wanted to see the results
quickly, I had to do with that for now :)

With best regards from Finland,

Henkka

Daniel Shiffman

unread,
Nov 17, 2010, 7:13:36 PM11/17/10
to openk...@googlegroups.com
This is fantastic!

Unfortunately, my library at the moment just passes back two int
arrays (one for RGB and one for depth, using grayscale colors). I
did it really quickly and just implemented that much, but plan on
exposing all of the raw data to Java in the next release. Stay
tuned!!

Dan

Alireza Sahami

unread,
Nov 18, 2010, 2:49:43 AM11/18/10
to openk...@googlegroups.com
Hi
I tried to run your project (on OSX) but get this error. However, I am
able to compile and runs Daniel Shiffman's code.
Any idea how I can solve this issue? Sry but I m newbie with Processing.


PeasyCam v0.91
processing.app.debug.RunnerException: UnsatisfiedLinkError: no Kinect in
java.library.path
at processing.app.Sketch.placeException(Sketch.java:1543)
at processing.app.debug.Runner.findException(Runner.java:583)
at processing.app.debug.Runner.reportException(Runner.java:558)
at processing.app.debug.Runner.exception(Runner.java:498)
at
processing.app.debug.EventThread.exceptionEvent(EventThread.java:367)
at processing.app.debug.EventThread.handleEvent(EventThread.java:255)
at processing.app.debug.EventThread.run(EventThread.java:89)
Exception in thread "Animation Thread" java.lang.UnsatisfiedLinkError:
no Kinect in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1755)
at java.lang.Runtime.loadLibrary0(Runtime.java:822)
at java.lang.System.loadLibrary(System.java:993)
at shiffman.kinect.NativeKinect.<clinit>(NativeKinect.java:25)
at SimpleKinect.setup(SimpleKinect.java:63)
at processing.core.PApplet.handleDraw(PApplet.java:1583)
at processing.core.PApplet.run(PApplet.java:1503)
at java.lang.Thread.run(Thread.java:655)

Thanks in advance
Ali


On 11/17/10 10:09 PM, Henkka wrote:
> Hello,
>
> I made an small point-cloud demo, using Daniel's great library and
> Kyle's great ThreePhase-application as an example of calculations
> needed. This seems just about the minimum that is needed to get simple
> real-time point-cloud demo up and running in Processing.
>
> Some video at http://www.youtube.com/watch?v=4Cvqw2QS3Yk
>
> If anyone wants to grab the code used, it can be found from
> http://yty.net/h/simplekinect.pde
>

> Thanks Daniel& Kyle for sharing these great pieces of software !

Henkka

unread,
Nov 18, 2010, 3:55:13 AM11/18/10
to OpenKinect
Hi,

To me, it seems that somehow it's not finding the kinect library? What
do you have in your Documents/Processing/libraries - directory ?
There should be "kinect"-directory that contains Daniel's example, eg.
"examples" and "library"-directories.

So

Documents/Processing
Documents/Processing/libraries/
Documents/Processing/libraries/kinect
Documents/Processing/libraries/kinect/examples
Documents/Processing/libraries/kinect/library

should all be there.

But as you are able to run Daniel's example, this may already be the
case. Then I don't know what this error is about. To me, it seems that
you are able to load the PeasyCam correctly but after that it fails.

I'll investigate this further in the evening after kids are
sleeping ... If you don't find the solution until then.

Best regards,

Henkka
> > Some video athttp://www.youtube.com/watch?v=4Cvqw2QS3Yk

Henkka

unread,
Nov 18, 2010, 2:14:58 PM11/18/10
to OpenKinect

> On 18 marras, 09:49, Alireza Sahami <alireza.sah...@gmail.com> wrote:
>
>
>
> > Hi
> > I tried to run your project (on OSX) but get this error. However, I am
> > able to compile and runs Daniel Shiffman's code.
> > Any idea how I can solve this issue? Sry but I m newbie with Processing.
>
> > PeasyCam v0.91
> > processing.app.debug.RunnerException: UnsatisfiedLinkError: no Kinect in
> > java.library.path

Hi,

I reproduced this error by temporarily moving libKinect.jnilib away
from kinect/library-directory. So it seems it doesn't find them in the
right place.
So please make sure these files exist:

/Users/your_username/Documents/Processing/libraries/kinect/library/
libKinect.jnilib
/Users/your_username/Documents/Processing/libraries/kinect/library/
kinect.jar

Best regards,

Henkka

Henkka

unread,
Nov 18, 2010, 2:25:38 PM11/18/10
to OpenKinect
Hi,

I made an small update to the "simplekinect.pde" Processing-script,
now it can output the point-cloud as .obj-file for Blender import.
Also added some grey-coloring. I had to change from points to lines
because SuperCAD libraries didn't want to output bare points as .obj-
files.

The updated script can be found from http://yty.net/h/simplekinect2.pde
and few videos at http://www.youtube.com/watch?v=Xd2HzafqxT8 and
http://www.youtube.com/watch?v=ke9r70QpqQg

Next could be enhancing the scanning of static objects by perhaps
calculating median from each pixels distance. Taking for example 10
seconds worth of measurements and calculating median from them. I
would guess it could enhance the accuracy a bit. Of course, then the
object and the kinect should be perfectly still while scanning. But it
could be good for more accurate measurements of stationary objects.
Does anyone have any opinions if this would work ? That is propably
next issue I'm trying.

And I'm still bit lost why my scans show 90 degree angle as something
bigger, like 120 degrees or similar. Will investigate that further as
well.

Best regards,

Henkka

Alireza Sahami

unread,
Nov 19, 2010, 5:00:51 AM11/19/10
to openk...@googlegroups.com
Thx for the guide, it works now.

Ali

Message has been deleted
Message has been deleted

Jean-Philippe Encausse

unread,
Nov 20, 2010, 6:15:56 AM11/20/10
to OpenKinect
Hi, Fantastic !
+1 for the Java Windows version wouldbe absolutly great :)




On Nov 17, 4:22 pm, Daniel Shiffman <daniel.shiff...@nyu.edu> wrote:
> Yes, the download has two files:
>
> shiffman.net/p5/kinect.zip
> kinect.jar
> kinectNative.jnilib
>
> You can simply add the jar to your classpath when compiling and you'll
> be all set!  I'll publish the source, etc. to github soon, sorry for
> being slow here. . .
>
> Dan
>
>
>
>
>
>
>
> On Wed, Nov 17, 2010 at 7:01 AM, jquev...@gmail.com <jquev...@gmail.com> wrote:
> > Is it possible to use the .jar which comes with the extension in
> > conventional Java?
> > Thank you very much!
>
> > ---------------------------------------------------------
> > Javier Quevedo Fernández
> > jquev...@gmail.com
> > Spain +34 618 420 028
> > Netherlands +31 684888066
> > ---------------------------------------------------------
>

AndyC

unread,
Nov 20, 2010, 9:13:06 PM11/20/10
to OpenKinect
Hi there,

Thanks for all of your work on this Processing library, Dan - it looks
really great.

I have tried installing and running the library and have got the
following error when running the KinectTest example, even though other
people here seem to be having no problem with it.

Does anyone have any suggestions for what the problem is? Is it for
Intel-based Macs only or is there something else wrong? It seems to be
finding the library ok, but having some other problem with it.

At the moment, I am using a G5 running OSX 10.5.8 and Processing 1.2.1
on Java 1.5.0 - if you need any more information, just let me know.

Thanks in advance,

Andy


-- start of error message --

processing.app.debug.RunnerException: UnsatisfiedLinkError: /Users/
mpcg/Documents/Processing/libraries/kinect/library/libKinect.jnilib:
no suitable image found. Did find: /Users/mpcg/Documents/Processing/
libraries/kinect/library/libKinect.jnilib: mach-o, but wrong
architecture
at processing.app.Sketch.placeException(Sketch.java:1543)
at processing.app.debug.Runner.findException(Runner.java:583)
at processing.app.debug.Runner.reportException(Runner.java:558)
at processing.app.debug.Runner.exception(Runner.java:498)
at processing.app.debug.EventThread.exceptionEvent(EventThread.java:
367)
at processing.app.debug.EventThread.handleEvent(EventThread.java:255)
at processing.app.debug.EventThread.run(EventThread.java:89)
Exception in thread "Animation Thread"
java.lang.UnsatisfiedLinkError: /Users/mpcg/Documents/Processing/
libraries/kinect/library/libKinect.jnilib: no suitable image found.
Did find: /Users/mpcg/Documents/Processing/libraries/kinect/library/
libKinect.jnilib: mach-o, but wrong architecture
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1822)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1739)
at java.lang.Runtime.loadLibrary0(Runtime.java:822)
at java.lang.System.loadLibrary(System.java:993)
at shiffman.kinect.NativeKinect.<clinit>(NativeKinect.java:25)
at KinectTest.setup(KinectTest.java:29)
at processing.core.PApplet.handleDraw(PApplet.java:1583)
at processing.core.PApplet.run(PApplet.java:1503)
at java.lang.Thread.run(Thread.java:613)

-- end of error message --

Daniel Shiffman

unread,
Nov 20, 2010, 10:06:12 PM11/20/10
to openk...@googlegroups.com
Ah, sorry, I think the Processing library currently supports intel mac
only. I will investigate and get back to you!

Dan

jonathan

unread,
Nov 21, 2010, 12:57:44 PM11/21/10
to OpenKinect
Thanks to Dan for this great library and to Henkka for the amazing
builds on top of it.

I'm working on OSX 10.6.4 (Java SE 6) and took a break from working in
Xcode/OpenGL to try Java/Processing (so I'm a newb here.) I am able to
compile the KinectTest without error, but the JAR won't launch at all,
and viewing it in the browser shows the following errors in the
console:
----------------------------
...basic: Starting applet…
basic: completed perf rollup
Exception in thread "Animation Thread"
java.lang.ExceptionInInitializerError
at KinectTest.setup(KinectTest.java:29)
at processing.core.PApplet.handleDraw(PApplet.java:1571)
at processing.core.PApplet.run(PApplet.java:1496)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.security.AccessControlException: access denied
(java.lang.RuntimePermission loadLibrary.Kinect)
at
java.security.AccessControlContext.checkPermission(AccessControlContext.java:
323)
at
java.security.AccessController.checkPermission(AccessController.java:
546)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:
532)
at java.lang.SecurityManager.checkLink(SecurityManager.java:818)
at java.lang.Runtime.loadLibrary0(Runtime.java:817)
at java.lang.System.loadLibrary(System.java:1045)
at shiffman.kinect.NativeKinect.<clinit>(NativeKinect.java:25)...
----------------------------
Any ideas how to fix this? I went as far as trying to add all
permissions for this codebase in the java.policy file, but that's a
stab in the dark that might have unintended consequences. Thanks for
any light you can shed!

Cheers,
Jon

Daniel Shiffman

unread,
Nov 21, 2010, 9:47:24 PM11/21/10
to openk...@googlegroups.com
This is due to applet security restrictions. You cannot access a
webcam, for example, in a java applet. Unless you sign the applet.

http://wiki.processing.org/w/Sign_an_Applet

Dan

Andy Clarke

unread,
Nov 23, 2010, 5:36:49 AM11/23/10
to openk...@googlegroups.com
Thanks for your reply. I really appreciate your work on the version
for Processing, given your other deadlines. In the meantime, I'll just
try to borrow something suitable (I usually work on a PC). If you need
people to test it, just let me know.

Regards,

Andy

On 21/11/2010 3:06 AM, Daniel Shiffman wrote:
> Ah, sorry, I think the Processing library currently supports intel mac
> only. I will investigate and get back to you!
>
>

> On Sat, Nov 20, 2010 at 9:13 PM, AndyC<an...@kinonet.com> wrote:
>> Does anyone have any suggestions for what the problem is? Is it for
>> Intel-based Macs only or is there something else wrong?
>>

Matt Gilbert

unread,
Dec 14, 2010, 12:02:44 PM12/14/10
to OpenKinect
Dan, is there a plan to release a new version of this library? Are
there plans to release the source? I've also been running into the
UnsatisfiedLinkError in Processing/Eclipse

Matt


On Nov 23, 5:36 am, Andy Clarke <a...@kinonet.com> wrote:
>   Thanks for your reply. I really appreciate your work on the version
> for Processing, given your other deadlines. In the meantime, I'll just
> try to borrow something suitable (I usually work on a PC). If you need
> people to test it, just let me know.
>
> Regards,
>
> Andy
>
> On 21/11/2010 3:06 AM, Daniel Shiffman wrote:
>
> > Ah, sorry, I think the Processing library currently supports intel mac
> > only.  I will investigate and get back to you!
>

Andy Clarke

unread,
Dec 14, 2010, 12:57:54 PM12/14/10
to openk...@googlegroups.com
The following is a good alternative Processing library:
https://github.com/nrocy/processing-openkinect

I don't know if it will help with the problem that you have, but I have
found it impressively fast.

Regards,

Andy

Dave Pentecost

unread,
Dec 14, 2010, 1:31:20 PM12/14/10
to openk...@googlegroups.com
I'm a bit confused by the Methods list on the github page, specifically these two lines:

NativeKinect.setVideoIR() - output RGB video (default)
NativeKinect.setVideoIR() - output IR video

I'm guessing there's a typo and the first line should be

NativeKinect.setVideoRGB() - output RGB video (default)

Can someone confirm this?

Thanks
Dave

--
Director, Technology
Center for Community
http://www.girlsclub.org/building

@dpentecost
Cell 646 704 2021


Daniel Shiffman

unread,
Dec 14, 2010, 3:23:17 PM12/14/10
to openk...@googlegroups.com
Hello all!

Sorry to be so out of touch about this work. I'm slowly returning to
this project and plan on implementing more over the holidays. The
library that Andy referenced is great and uses the up-to-date
libfreenect code:

https://github.com/nrocy/processing-openkinect

However, I was thinking what might be easiest / best is to piggy-back
off of the work done already to wrap libfreenect with Java:

http://openkinect.org/wiki/Java_Wrapper

My plan is to take a look at this work and make a Processing bridge to it.

If anyone has any thoughts / suggestions, etc. please let me know.

Thanks all!
Dan

Matt Gilbert

unread,
Dec 14, 2010, 3:37:04 PM12/14/10
to OpenKinect
Andy, thanks! I will try this out soon.

yoda

unread,
Dec 15, 2010, 11:21:55 AM12/15/10
to OpenKinect
It might also be worthwhile to investigate a JNA solution:
https://jna.dev.java.net/

This is apparently how the Java interface for CV was written (http://
code.google.com/p/javacv/) and it seems that a single technology for
utilizing both OpenKinect & CV from Java might be desirable. I hope
to investigate a bit over the next few weeks, but haven't had a chance
yet.

Regards.
-Brian

misan

unread,
Dec 15, 2010, 12:55:26 PM12/15/10
to OpenKinect
Hi Daniel:

Maybe I'm missing something here but I fail to see the Java folder in
the wrappers of libfreenect project (as now openkinect has merged into
it).

So following the instructions you mention is not possible at the
moment (or so I think).

I'm really interested on getting Kinect working on Processing in
Linux, but I'm a little bit disoriented (as I am not familiar with JNI
nor Processing internals).

But I'll be happy to help.

I've got it working in Linux with libfreenect, in Windows 7 with CL
NUI and on OSX with openFrameworks and in Processing too (with your
code).

Kind regards,

Miguel Sánchez
http://fightpc.blogspot.com

yoda

unread,
Dec 15, 2010, 1:42:13 PM12/15/10
to OpenKinect
Michael Nischt's JNI wrapper has not yet been merged into the main
libfreenect branch. Its available here: https://github.com/michael-nischt

On Dec 15, 11:55 am, misan <m1g...@gmail.com> wrote:
> Hi Daniel:
>
> Maybe I'm missing something here but I fail to see the Java folder in
> the wrappers of libfreenect project (as now openkinect has merged into
> it).
>
> So following the instructions you mention is not possible at the
> moment (or so I think).
>
> I'm really interested on getting Kinect working on Processing in
> Linux, but I'm a little bit disoriented (as I am not familiar with JNI
> nor Processing internals).
>
> But I'll be happy to help.
>
> I've got it working in Linux with libfreenect, in Windows 7 with CL
> NUI and on OSX with openFrameworks and in Processing too (with your
> code).
>
> Kind regards,
>
> Miguel Sánchezhttp://fightpc.blogspot.com
Message has been deleted
Message has been deleted

Florian Lier

unread,
Dec 16, 2010, 9:17:53 AM12/16/10
to openk...@googlegroups.com
Hello all,

based on Michaels git repos (Java Wrapper) I wrote a trivial
Java app which just displays the rgb frames in a JFrame.

I'm experiencing serious performance issues now... The wrapper
delivers ~6 fps (ColorImage) and ~4-5 fps (DepthImage). Do
you guys have noticed that too?

The glview example seems to be okay (fast).

I can upload my app somewhere if needed ...

Cheers,
Florian

On 12/16/2010 09:24 AM, misan wrote:
> Oops, I answer myself:
>
> Wrappers are not in the current version but going back they are there:
>
> https://github.com/michael-nischt/libfreenect/commit/8b5cf892cfbed3426bd8b4db4a5a711e3070d0c5
>
> Thanks,
>
> Miguel
>
>
> On 16 dic, 09:20, misan <m1g...@gmail.com> wrote:
>> Hi Yoda,
>>
>> I'm afraid java wrappers are missing there too (at least I failed to
>> find them).
>>
>> Thanks,
>>
>> Miguel


>>
>> On 15 dic, 19:42, yoda <y...@greatmindsworking.com> wrote:
>>
>>
>>
>>
>>
>>
>>
>>> Michael Nischt's JNI wrapper has not yet been merged into the main
>>> libfreenect branch. Its available here:https://github.com/michael-nischt
>>> On Dec 15, 11:55 am, misan <m1g...@gmail.com> wrote:
>>>> Hi Daniel:
>>>> Maybe I'm missing something here but I fail to see the Java folder in
>>>> the wrappers of libfreenect project (as now openkinect has merged into
>>>> it).
>>>> So following the instructions you mention is not possible at the
>>>> moment (or so I think).
>>>> I'm really interested on getting Kinect working on Processing in
>>>> Linux, but I'm a little bit disoriented (as I am not familiar with JNI
>>>> nor Processing internals).
>>>> But I'll be happy to help.
>>>> I've got it working in Linux with libfreenect, in Windows 7 with CL
>>>> NUI and on OSX with openFrameworks and in Processing too (with your
>>>> code).
>>>> Kind regards,

>>>> Miguel S�nchezhttp://fightpc.blogspot.com

yoda

unread,
Dec 16, 2010, 4:57:01 PM12/16/10
to OpenKinect
Appears that there is a JNA solution already in the works by tweakt
(Mark Renouf):
https://github.com/tweakt/libfreenect/tree/java-updates/wrappers/java

Felix

unread,
Dec 17, 2010, 10:49:15 AM12/17/10
to OpenKinect
I had to add -d32 to the jvm becuase the native code is 32bit only.
Everything else work nicely, although a bit slow.

Florian Lier

unread,
Dec 17, 2010, 10:52:06 AM12/17/10
to openk...@googlegroups.com
Hey,

On 12/17/2010 04:49 PM, Felix wrote:
> I had to add -d32 to the jvm becuase the native code is 32bit only.
> Everything else work nicely, although a bit slow.
>

What do you mean by slow?
The java wrapper

https://github.com/michael-nischt/libfreenect

produces approx. 5-6 fps

Cheers,
Florian

Daniel Shiffman

unread,
Dec 17, 2010, 5:28:07 PM12/17/10
to openk...@googlegroups.com
Hello all!

I adapted the existing JNI wrapper to work with Processing. Made some
changes and compiled all the native code into a JNILIB file using
xcode (yuck!).

Source is here:
https://github.com/shiffman/libfreenect/tree/master/wrappers/java/processing

Download library here:
https://github.com/shiffman/libfreenect/blob/master/wrappers/java/processing/distribution/openkinect.zip

I'm getting 30 FPS for both the depth and RGB images (independently).
If you ask for both I'm getting about 20 FPS on each.

Tons more to do on this library, help welcome!

(Also, embarrassingly, this is the first time I've ever used git. So
any help / advice is welcome. I forked Michael Nischt's fork of
libfreenect, which I hope was the appropriate thing to do.)

Thanks all!
Dan

Andy Clarke

unread,
Dec 20, 2010, 10:32:50 AM12/20/10
to openk...@googlegroups.com
Hi there,

Thanks for this library. I have downloaded it and have got the following
error when I try the example sketch. Do you have any suggestions?

"You are missing the kinectEvent() method.
java.lang.NoSuchMethodException:
RGBDepthTest.kinectEvent(org.openkinect.processing.Kinect)
Invalid memory access of location 0xe6 eip=0x921c9a74"

libfreenect is installed using Homebrew and the glview test application
works, as does the nrocy Processing library.

If you need any more info, feel free to contact me either on or off list.

Regards,

Andy

Florian Lier

unread,
Dec 20, 2010, 10:34:30 AM12/20/10
to openk...@googlegroups.com
Hey,

On 12/20/2010 04:32 PM, Andy Clarke wrote:
> Hi there,
>
> Thanks for this library. I have downloaded it and have got the following
> error when I try the example sketch. Do you have any suggestions?
>
> "You are missing the kinectEvent() method.
> java.lang.NoSuchMethodException:
> RGBDepthTest.kinectEvent(org.openkinect.processing.Kinect)
> Invalid memory access of location 0xe6 eip=0x921c9a74"
>
> libfreenect is installed using Homebrew and the glview test application
> works, as does the nrocy Processing library.
>
>

Currently it just works on MacOS, thats why there's a JNILIB ...

Andy Clarke

unread,
Dec 20, 2010, 11:02:24 AM12/20/10
to openk...@googlegroups.com
Yes - this is on a Mac (running OS X 10.6.5)

Regards,

Andy

> Currently it just works on MacOS, thats why there's a JNILIB ...

>> Thanks for this library. I have downloaded it and have got the following


>> error when I try the example sketch. Do you have any suggestions?
>>
>> "You are missing the kinectEvent() method.
>> java.lang.NoSuchMethodException:
>> RGBDepthTest.kinectEvent(org.openkinect.processing.Kinect)
>> Invalid memory access of location 0xe6 eip=0x921c9a74"

>>> Download library here:
>>> https://github.com/shiffman/libfreenect/blob/master/wrappers/java/processing/distribution/openkinect.zip


Daniel Shiffman

unread,
Dec 20, 2010, 1:54:20 PM12/20/10
to openk...@googlegroups.com
You can ignore the eventMethod() related error. That's just a
placeholder right now, i should probably remove it, but haven't
implemented a kinectEvent() method yet.

This message:

Invalid memory access of location 0xe6 eip=0x921c9a74

is more concerning. Is it possible your connect is open / in use
with another app?

Dan

Andy Clarke

unread,
Dec 22, 2010, 6:09:52 AM12/22/10
to openk...@googlegroups.com
Dan,

Thanks for the reply. I have done some more investigation into this.

The "Invalid memory access" error only occurs when I shut down the
sketch in certain ways. If I press the 'stop' button in the Processing
IDE, the sketch exits fine without error. If I press 'command Q' or
'escape' within the sketch, or choose 'quit RGBDepthTest' from the menu
in the sketch, I get the error message and a warning dialog which says
"RGBDepthTest quit unexpectedly while using the libKinect.jnlib
plug-in." Once I have dismissed the error, I can start the sketch up again.

I hope this helps. If you need any more information, just let me know
(though as you already have this issue logged (as #7), I presume you can
reproduce the error).

Aside from this, the library works fine. I was just thrown by the
kinectEvent() error and the blank screen and didn't realise that the
demo sketch starts with the RGB and depth displays both turned off as
default.

Regards,

Andy

On 20/12/2010 6:54 PM, Daniel Shiffman wrote:
> You can ignore the eventMethod() related error. That's just a
> placeholder right now, i should probably remove it, but haven't
> implemented a kinectEvent() method yet.
>
> This message:
> Invalid memory access of location 0xe6 eip=0x921c9a74
> is more concerning. Is it possible your connect is open / in use
> with another app?

>>>> Thanks for this library. I have downloaded it and have got the following

Daniel Shiffman

unread,
Dec 22, 2010, 1:30:54 PM12/22/10
to openk...@googlegroups.com
Hi Andy,

Does adding this fix the error for you? (it did for me).

void stop() {
kinect.quit();
super.stop();
}

Thanks!
Dan

Andy Clarke

unread,
Dec 27, 2010, 2:45:40 PM12/27/10
to openk...@googlegroups.com
Hi Dan,

Thanks for this suggestion. I have finally had a chance to test this fix
and the results are as follows.

Pressing ESC in sketch:
I get the following in the IDE text area (in stead of the Invalid memory
access error)
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at org.openkinect.processing.Kinect.run(Kinect.java:90)

Pressing Command Q in sketch:
Invalid memory access of location 0xe6 eip=0x921c9a74 (as before)

Pressing Stop in Processing IDE:
fine (as before)

I'll keep having a look at it - let me know if you have any other
suggestions.

Regards.

Andy


> Does adding this fix the error for you? (it did for me).
>
> void stop() {
> kinect.quit();
> super.stop();
> }

>> The "Invalid memory access" error only occurs when I shut down the
>> sketch in certain ways.

Miles

unread,
Dec 29, 2010, 5:38:40 PM12/29/10
to OpenKinect
Hi Dan,

I'm running an Intel Mac with 10.5.8, and I seem to be getting the
unsatsfiedlinkerror as well. Do you know what could be causing this?

Regards,

Miles

On Nov 20, 10:06 pm, Daniel Shiffman <daniel.shiff...@nyu.edu> wrote:
> Ah, sorry, I think the Processing library currently supports intel mac
> only.  I will investigate and get back to you!
>
> Dan
>
>
>
>
>
>
>
> On Sat, Nov 20, 2010 at 9:13 PM, AndyC <a...@kinonet.com> wrote:
> > Hi there,
>
> > Thanks for all of your work on this Processing library, Dan - it looks
> > really great.
>
> > I have tried installing and running the library and have got the
> > following error when running the KinectTest example, even though other
> > people here seem to be having no problem with it.
>
> > Does anyone have any suggestions for what the problem is? Is it for
> > Intel-based Macs only or is there something else wrong? It seems to be
> > finding the library ok, but having some other problem with it.
>
> > At the moment, I am using a G5 running OSX 10.5.8 and Processing 1.2.1
> > on Java 1.5.0 - if you need any more information, just let me know.
>
> > Thanks in advance,
>
> > Andy
>
> > -- start of error message --
>
> > processing.app.debug.RunnerException:UnsatisfiedLinkError: /Users/
> > mpcg/Documents/Processing/libraries/kinect/library/libKinect.jnilib:
> > no suitable image found.  Did find:  /Users/mpcg/Documents/Processing/
> > libraries/kinect/library/libKinect.jnilib: mach-o, but wrong
> > architecture
> >        at processing.app.Sketch.placeException(Sketch.java:1543)
> >        at processing.app.debug.Runner.findException(Runner.java:583)
> >        at processing.app.debug.Runner.reportException(Runner.java:558)
> >        at processing.app.debug.Runner.exception(Runner.java:498)
> >        at processing.app.debug.EventThread.exceptionEvent(EventThread.java:
> > 367)
> >        at processing.app.debug.EventThread.handleEvent(EventThread.java:255)
> >        at processing.app.debug.EventThread.run(EventThread.java:89)
> > Exception in thread "Animation Thread"
> > java.lang.UnsatisfiedLinkError: /Users/mpcg/Documents/Processing/
> > libraries/kinect/library/libKinect.jnilib:  no suitable image found.
> > Did find:  /Users/mpcg/Documents/Processing/libraries/kinect/library/
> > libKinect.jnilib: mach-o, but wrong architecture
> >        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
> >        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1822)
> >        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1739)
> >        at java.lang.Runtime.loadLibrary0(Runtime.java:822)
> >        at java.lang.System.loadLibrary(System.java:993)
> >        at shiffman.kinect.NativeKinect.<clinit>(NativeKinect.java:25)
> >        at KinectTest.setup(KinectTest.java:29)
> >        at processing.core.PApplet.handleDraw(PApplet.java:1583)
> >        at processing.core.PApplet.run(PApplet.java:1503)
> >        at java.lang.Thread.run(Thread.java:613)
>
> > -- end of error message --
>
> > On Nov 16, 3:55 pm, Daniel Shiffman <daniel.shiff...@nyu.edu> wrote:
> >> Hello all,
>
> >> Just wanted to introduce myself and say hello.  I'm working on a port
> >> of OpenKinect for Java.  Currently I did a really quick test with JNI
> >> and ran the results in Processing (on Mac OSX only)
>
> >>http://www.shiffman.net/2010/11/14/kinect-and-processing/
>
> >> If you aren't familiar with Processing it's an open source programming
> >> language and environment for images, animations, and interactions and
> >> is built with Java.
>
> >>http://www.processing.org/
>
> >> I'm under a big deadline for another project that is happening
> >> December 3rd, but my month of December is hopefully going to be
> >> dedicated to Kinect testing and building out Java libraries for it.
>
> >> Advice, comments, etc. are welcome!
>
> >> Thanks for everyone's wonderful hard work and inspiring spirit!
>
> >> Dan

Matt Gilbert

unread,
Jan 9, 2011, 4:08:38 PM1/9/11
to OpenKinect
Andy, I finally got around to trying nrocy's library, and it worked
great, right out of the box, in the Processing IDE. It currently lacks
a few bells and whistles (motor control, etc), but I haven't
encountered any of the UnsatisfiedLinkErrors I was getting with
shiffman's. Still not sure what caused those, but the library works
great:
https://github.com/nrocy/processing-openkinect

I'll keep following the development of shiffman's library and try it
out every once in a while. Personally though, I do hope nrocy's
library continues to be developed, since it works for me with no
complaints.

Matt


On Dec 14 2010, 3:37 pm, Matt Gilbert <gilbm...@gmail.com> wrote:
> Andy, thanks! I will try this out soon.
>
> On Dec 14, 12:57 pm, Andy Clarke <a...@kinonet.com> wrote:
>
> >   The following is a good alternativeProcessinglibrary:https://github.com/nrocy/processing-openkinect

Tom

unread,
Jan 9, 2011, 5:29:02 PM1/9/11
to OpenKinect
Hi Guys,

Is there any news on a library for Windows/Linux yet? Would really
like to have a go with Processing but I can't afford a mac!

Thanks,
Tom
> > > > Matt- Hide quoted text -
>
> - Show quoted text -

Greg Satz

unread,
Jan 9, 2011, 7:49:45 PM1/9/11
to openk...@googlegroups.com
I get the following error on a (64 bit) Mac Pro:

karma.iranger.com>java -jar kinect.jar
Exception in thread "main" java.lang.UnsatisfiedLinkError: /Users/satz/Desktop/kinect/processing-openkinect/build/Debug/libKinect.jnilib: no suitable image found. Did find: /Users/satz/Desktop/kinect/processing-openkinect/build/Debug/libKinect.jnilib: mach-o, but wrong architecture
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1823)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
at java.lang.Runtime.loadLibrary0(Runtime.java:823)
at java.lang.System.loadLibrary(System.java:1045)
at king.kinect.NativeKinect.<clinit>(NativeKinect.java:8)
at Kinect.main(Kinect.java:6)

karma.iranger.com>java -d32 -jar kinect.jar
Found 1 device(s)
Device 0 initialised
Invalid memory access of location 0x0 eip=0x17daa31

Bus error

I played around with the xcode project settings and different architectures but I couldn't get it to generate anything other then an i386 jnilib. Let me know if there is anything else I can try...

Thanks,
Greg

On Jan 9, 2011, at 2:08 PM, Matt Gilbert wrote:

> Andy, I finally got around to trying nrocy's library, and it worked
> great, right out of the box, in the Processing IDE. It currently lacks
> a few bells and whistles (motor control, etc), but I haven't
> encountered any of the UnsatisfiedLinkErrors I was getting with
> shiffman's. Still not sure what caused those, but the library works
> great:
> https://github.com/nrocy/processing-openkinect
>
> I'll keep following the development of shiffman's library and try it
> out every once in a while. Personally though, I do hope nrocy's
> library continues to be developed, since it works for me with no
> complaints.
>
> Matt
>
>
> On Dec 14 2010, 3:37 pm, Matt Gilbert <gilbm...@gmail.com> wrote:
>> Andy, thanks! I will try this out soon.
>>
>> On Dec 14, 12:57 pm, Andy Clarke <a...@kinonet.com> wrote:
>>

>>> The following is a good alternativeProcessinglibrary:https://github..com/nrocy/processing-openkinect


>>
>>> I don't know if it will help with the problem that you have, but I have
>>> found it impressively fast.
>>
>>> Regards,
>>
>>> Andy
>>
>>> On 14/12/2010 5:02 PM, Matt Gilbert wrote:
>>
>>>> Dan, is there a plan to release a new version of this library? Are
>>>> there plans to release the source? I've also been running into the
>>>> UnsatisfiedLinkError inProcessing/Eclipse
>>
>>>> Matt
>

> !DSPAM:4d2a23f6991421219519540!
>

Gavrilo Bozovic

unread,
Mar 4, 2011, 5:36:46 AM3/4/11
to OpenKinect
Hi,

I'm having the same error message, about the memory access, and what I
tracked the error to is very strange. Here's my code:

public class DepthImage extends PApplet {

PImage img, depth;

public static void main(String args[])
{
PApplet.main(new String[]
{ depthimage.DepthImage.class.getName() });
}


public void setup() {
size(640, 480, OPENGL);
noStroke();
noFill();

NativeKinect.init();
NativeKinect.start();

img = createImage(640, 480, RGB);
depth = createImage(640, 480, RGB);
}

public void draw() {

depth.pixels = NativeKinect.getDepthMap();
depth.updatePixels();
img.pixels = NativeKinect.getVideo();

image(depth,0,0);
}
}

And it gives the following output:

Found 1 device(s)
Device 0 initialised
Invalid memory access of location 0x0 eip=0x20e10c86

Now the very strange thing is that it only does that if the rendering
is set to OpenGL, not with default or P3D. In addition, the memory
access issue only occurs when I try to access to the depth map, I can
access the image.

Has anyone experienced this? Do you have any idea what's causing it,
and how to remedy to it?

Thanks a lot in advance for any hints!

Gavirlo
>  smime.p7s
> 5KAfficherTélécharger

Devomat

unread,
Mar 12, 2011, 8:57:00 AM3/12/11
to OpenKinect
"Invalid memory access of location"

This is a Thread error. You cannot use more than 1 Thread to take the
Kinect Data. ProcessUpdate() and other Kinect calls have to be in the
same Thread. After that changes, I've got no problems anymore.

yair reshef

unread,
Mar 15, 2011, 3:23:31 PM3/15/11
to openk...@googlegroups.com
hey tom, 
--
yair99@gmail
050-6301212
tlv, israel

James Britt

unread,
Mar 17, 2011, 2:10:07 PM3/17/11
to openk...@googlegroups.com
Tom wrote:
> Hi Guys,
>
> Is there any news on a library for Windows/Linux yet? Would really
> like to have a go with Processing but I can't afford a mac!
>


I've been using the CLNUI 4 Java library, along with Processing, on Windows.

I'm building GUI apps using JRuby and Monkeybars with these.

(Of course what I really would like is a way to build one cross-platform
app where I can swap in the lowerlevel JNI bindings based on OS while
keeping the high-lelvel API calls the same, but for the time being I'm
OK exploring on WIndows.)


James Britt

--

jamesbritt.com - Playing with Better Toys
neurogami.com - Smart application development
azhackers.com - Feed your head. Hack your world.

Gavrilo Bozovic

unread,
Apr 27, 2011, 6:16:28 AM4/27/11
to OpenKinect
Hi Devomat,

Thanks for your reply. However, I don't quite get what in my code uses
two threads, and what are the changes you had to make to get it to
work?

In addition, as said previously, the problem occurs when passing from
P3D to OpenGL, without any other change to the structure of the
program.

Thanks in advance for any further insights, and sorry if it's a dumb
problem we're missing!


Gavrilo

cketcham

unread,
Feb 13, 2013, 11:48:25 AM2/13/13
to openk...@googlegroups.com, an...@kinonet.com
I was also getting the unsatisfied link error. In case it helps anyone, I solved it by changing the preferences in processing to "Launch programs in 32-bit mode".


On Saturday, November 20, 2010 9:13:06 PM UTC-5, AndyC wrote:
Hi there,

Thanks for all of your work on this Processing library, Dan - it looks
really great.

I have tried installing and running the library and have got the
following error when running the KinectTest example, even though other
people here seem to be having no problem with it.

Does anyone have any suggestions for what the problem is? Is it for
Intel-based Macs only or is there something else wrong? It seems to be
finding the library ok, but having some other problem with it.

At the moment, I am using a G5 running OSX 10.5.8 and Processing 1.2.1
on Java 1.5.0 - if you need any more information, just let me know.

Thanks in advance,

Andy


-- start of error message --

processing.app.debug.RunnerException: UnsatisfiedLinkError: /Users/
mpcg/Documents/Processing/libraries/kinect/library/libKinect.jnilib:
no suitable image found.  Did find:  /Users/mpcg/Documents/Processing/
libraries/kinect/library/libKinect.jnilib: mach-o, but wrong
architecture
        at processing.app.Sketch.placeException(Sketch.java:1543)
        at processing.app.debug.Runner.findException(Runner.java:583)
        at processing.app.debug.Runner.reportException(Runner.java:558)
        at processing.app.debug.Runner.exception(Runner.java:498)
        at processing.app.debug.EventThread.exceptionEvent(EventThread.java:
367)
        at processing.app.debug.EventThread.handleEvent(EventThread.java:255)
        at processing.app.debug.EventThread.run(EventThread.java:89)
Exception in thread "Animation Thread"
java.lang.UnsatisfiedLinkError: /Users/mpcg/Documents/Processing/
libraries/kinect/library/libKinect.jnilib:  no suitable image found.
Did find:  /Users/mpcg/Documents/Processing/libraries/kinect/library/
libKinect.jnilib: mach-o, but wrong architecture
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
Reply all
Reply to author
Forward
0 new messages