Announcing OpenNI
273 views
Skip to first unread message
Joshua Blake
Dec 8, 2010, 12:18:14 PM12/8/10
to OpenKinect
Hello OpenKinect!
A few weeks ago, I was contacted by PrimeSense co-founder Tamir Berliner, who wanted to discuss their plans and ways PrimeSense could engage with the community. (Microsoft licensed PrimeSense’s technology and based Kinect on the PrimeSense reference design.) After seeing the dozens of Kinect videos and amazing enthusiasm from our community, PrimeSense decided to move up their plans and today announced the OpenNI initiative and have open sourced their middleware for natural interaction and drivers for their depth cameras.
PrimeSense is setting up OpenNI is being set up as a standards consortium for natural interaction, including motion tracking, voice, and others. As a part of today's announcement, they have released three things:
1) Open source drivers for the PrimeSensor reference device http://github.com/PrimeSense/Sensor
This is based upon the same SOC as Kinect. There are probably some knowledge in that code!
2) The open source OpenNI framework https://github.com/OpenNI/OpenNI
This is framework supports pluggable nodes that can let us create applications that work against any depth sensor that supports OpenNI and take advantage of any node (for example, skeleton tracking) that is available.
3) NITE (See http://openni.org/?q=node/2 for download link and license key)
This is a binary-only release for Linux and Windows that includes a PrimeSense implementation of skeleton tracking and gesture recognition, plus sample source code. NITE is an OpenNI module.
I'll be posting more information later about how some of this works. I already have started an OpenNI module interface for libfreenect that will allow us to plug Kinect in with OpenNI and take advantage of the capabilities of OpenNI and NITE, as shown in these videos:
Skeleton Extraction:
http://www.youtube.com/watch?v=nr8vgCnb9_0
http://www.youtube.com/watch?v=nr8vgCnb9_0
Scene Analyzer (user separation):
http://www.youtube.com/watch?v=-_j7BzSmlfU
http://www.youtube.com/watch?v=-_j7BzSmlfU
Thanks!
Josh
---
Joshua Blake
Microsoft Surface MVP
OpenKinect Community Founder http://openkinect.org
(cell) 703-946-7176
Twitter: http://twitter.com/joshblake
Blog: http://nui.joshland.org
Multitouch on Windows book: http://manning.com/blake
Eidur Arnason
Dec 8, 2010, 1:16:35 PM12/8/10
to openk...@googlegroups.com
Is it possible to dump the vector points through OSC to other programs ? (I am thinking Pure Data)
Best regards. Eidur
Best regards. Eidur
GUNNM
Dec 8, 2010, 1:26:21 PM12/8/10
to OpenKinect
Congratulation, result is very good, regarding the video.
I will follow your project.
> Hello OpenKinect!
>
> A few weeks ago, I was contacted by PrimeSense co-founder Tamir Berliner,
> who wanted to discuss their plans and ways PrimeSense could engage with the
> community. (Microsoft licensed PrimeSense’s technology and based Kinect on
> the PrimeSense reference
> design<http://www.wired.co.uk/magazine/archive/2010/11/features/the-game-cha...>.)
I will follow your project.
> Hello OpenKinect!
>
> A few weeks ago, I was contacted by PrimeSense co-founder Tamir Berliner,
> who wanted to discuss their plans and ways PrimeSense could engage with the
> community. (Microsoft licensed PrimeSense’s technology and based Kinect on
> the PrimeSense reference
> After seeing the dozens of Kinect videos and amazing enthusiasm from our
> community, PrimeSense decided to move up their plans and today announced the
> OpenNI <http://www.openni.org/> initiative and have open sourced their
> community, PrimeSense decided to move up their plans and today announced the
> middleware for natural interaction and drivers for their depth cameras.
>
> PrimeSense is setting up OpenNI is being set up as a standards consortium
> for natural interaction, including motion tracking, voice, and others. As a
> part of today's announcement, they have released three things:
>
> 1) Open source drivers for the PrimeSensor reference devicehttp://github.com/PrimeSense/Sensor
>
> PrimeSense is setting up OpenNI is being set up as a standards consortium
> for natural interaction, including motion tracking, voice, and others. As a
> part of today's announcement, they have released three things:
>
>
> This is based upon the same SOC as Kinect. There are probably some knowledge
> in that code!
>
> 2) The open source OpenNI frameworkhttps://github.com/OpenNI/OpenNI
> This is based upon the same SOC as Kinect. There are probably some knowledge
> in that code!
>
>
> This is framework supports pluggable nodes that can let us create
> applications that work against any depth sensor that supports OpenNI and
> take advantage of any node (for example, skeleton tracking) that is
> available.
>
> 3) NITE (Seehttp://openni.org/?q=node/2for download link and license key)
> This is framework supports pluggable nodes that can let us create
> applications that work against any depth sensor that supports OpenNI and
> take advantage of any node (for example, skeleton tracking) that is
> available.
>
>
> This is a binary-only release for Linux and Windows that includes a
> PrimeSense implementation of skeleton tracking and gesture recognition, plus
> sample source code. NITE is an OpenNI module.
>
> I'll be posting more information later about how some of this works. I
> already have started an OpenNI module interface for libfreenect that will
> allow us to plug Kinect in with OpenNI and take advantage of the
> capabilities of OpenNI and NITE, as shown in these videos:
>
> Skeleton Extraction:
> http://www.youtube.com/watch?v=nr8vgCnb9_0
>
> Scene Analyzer (user separation):
> http://www.youtube.com/watch?v=-_j7BzSmlfU
>
> Thanks!
> Josh
>
> ---
> Joshua Blake
> Microsoft Surface MVP
> OpenKinect Community Founderhttp://openkinect.org
> This is a binary-only release for Linux and Windows that includes a
> PrimeSense implementation of skeleton tracking and gesture recognition, plus
> sample source code. NITE is an OpenNI module.
>
> I'll be posting more information later about how some of this works. I
> already have started an OpenNI module interface for libfreenect that will
> allow us to plug Kinect in with OpenNI and take advantage of the
> capabilities of OpenNI and NITE, as shown in these videos:
>
> Skeleton Extraction:
> http://www.youtube.com/watch?v=nr8vgCnb9_0
>
> Scene Analyzer (user separation):
> http://www.youtube.com/watch?v=-_j7BzSmlfU
>
> Thanks!
> Josh
>
> ---
> Joshua Blake
> Microsoft Surface MVP
Patrick Bouffard
Dec 8, 2010, 2:14:24 PM12/8/10
to OpenKinect
From their docs, under "Capabilities":
"""
Cropping: Enables a map generator (depth, image, IR) to output a
selected area of the frame as opposed to the entire frame. When
cropping is enabled, the size of the generated map is reduced to fit a
lower resolution (less pixels). For example, if the map generator is
working in VGA resolution (640x480) and the application chooses to
crop at 300x200, the next pixel row will begin after 300 pixels.
Cropping can be very useful for performance boosting.
"""
Does this happen on-board the device itself, so that the extraneous
information is never even sent over USB? I'm hoping the answer is
'yes'... :)
Pat
On Dec 8, 9:18 am, Joshua Blake <joshbl...@gmail.com> wrote:
> Hello OpenKinect!
>
> A few weeks ago, I was contacted by PrimeSense co-founder Tamir Berliner,
> who wanted to discuss their plans and ways PrimeSense could engage with the
> community. (Microsoft licensed PrimeSense’s technology and based Kinect on
> the PrimeSense reference
> design<http://www.wired.co.uk/magazine/archive/2010/11/features/the-game-cha...>.)
"""
Cropping: Enables a map generator (depth, image, IR) to output a
selected area of the frame as opposed to the entire frame. When
cropping is enabled, the size of the generated map is reduced to fit a
lower resolution (less pixels). For example, if the map generator is
working in VGA resolution (640x480) and the application chooses to
crop at 300x200, the next pixel row will begin after 300 pixels.
Cropping can be very useful for performance boosting.
"""
Does this happen on-board the device itself, so that the extraneous
information is never even sent over USB? I'm hoping the answer is
'yes'... :)
Pat
On Dec 8, 9:18 am, Joshua Blake <joshbl...@gmail.com> wrote:
> Hello OpenKinect!
>
> A few weeks ago, I was contacted by PrimeSense co-founder Tamir Berliner,
> who wanted to discuss their plans and ways PrimeSense could engage with the
> community. (Microsoft licensed PrimeSense’s technology and based Kinect on
> the PrimeSense reference
> After seeing the dozens of Kinect videos and amazing enthusiasm from our
> community, PrimeSense decided to move up their plans and today announced the
> OpenNI <http://www.openni.org/> initiative and have open sourced their
> community, PrimeSense decided to move up their plans and today announced the
> middleware for natural interaction and drivers for their depth cameras.
>
> PrimeSense is setting up OpenNI is being set up as a standards consortium
> for natural interaction, including motion tracking, voice, and others. As a
> part of today's announcement, they have released three things:
>
> 1) Open source drivers for the PrimeSensor reference devicehttp://github.com/PrimeSense/Sensor
>
> PrimeSense is setting up OpenNI is being set up as a standards consortium
> for natural interaction, including motion tracking, voice, and others. As a
> part of today's announcement, they have released three things:
>
>
> This is based upon the same SOC as Kinect. There are probably some knowledge
> in that code!
>
> 2) The open source OpenNI frameworkhttps://github.com/OpenNI/OpenNI
> This is based upon the same SOC as Kinect. There are probably some knowledge
> in that code!
>
>
> This is framework supports pluggable nodes that can let us create
> applications that work against any depth sensor that supports OpenNI and
> take advantage of any node (for example, skeleton tracking) that is
> available.
>
> 3) NITE (Seehttp://openni.org/?q=node/2for download link and license key)
> This is framework supports pluggable nodes that can let us create
> applications that work against any depth sensor that supports OpenNI and
> take advantage of any node (for example, skeleton tracking) that is
> available.
>
>
> This is a binary-only release for Linux and Windows that includes a
> PrimeSense implementation of skeleton tracking and gesture recognition, plus
> sample source code. NITE is an OpenNI module.
>
> I'll be posting more information later about how some of this works. I
> already have started an OpenNI module interface for libfreenect that will
> allow us to plug Kinect in with OpenNI and take advantage of the
> capabilities of OpenNI and NITE, as shown in these videos:
>
> Skeleton Extraction:
> http://www.youtube.com/watch?v=nr8vgCnb9_0
>
> Scene Analyzer (user separation):
> http://www.youtube.com/watch?v=-_j7BzSmlfU
>
> Thanks!
> Josh
>
> ---
> Joshua Blake
> Microsoft Surface MVP
> OpenKinect Community Founderhttp://openkinect.org
> This is a binary-only release for Linux and Windows that includes a
> PrimeSense implementation of skeleton tracking and gesture recognition, plus
> sample source code. NITE is an OpenNI module.
>
> I'll be posting more information later about how some of this works. I
> already have started an OpenNI module interface for libfreenect that will
> allow us to plug Kinect in with OpenNI and take advantage of the
> capabilities of OpenNI and NITE, as shown in these videos:
>
> Skeleton Extraction:
> http://www.youtube.com/watch?v=nr8vgCnb9_0
>
> Scene Analyzer (user separation):
> http://www.youtube.com/watch?v=-_j7BzSmlfU
>
> Thanks!
> Josh
>
> ---
> Joshua Blake
> Microsoft Surface MVP
Chris OShea
Dec 8, 2010, 2:57:13 PM12/8/10
to OpenKinect
Great!
Does the driver (1) only work with PrimseSense dev camera, or will it
work with Kinect on Windows, or is there limitations by Microsoft to
do this?
I've only used Kinect on Mac so far. Have installed the "PrimeSensor
PSDK 5.0 Driver build 24 for windows" but with Kinect plugged in,
doesn't seem to do anything. Just curious.
Thanks
Does the driver (1) only work with PrimseSense dev camera, or will it
work with Kinect on Windows, or is there limitations by Microsoft to
do this?
I've only used Kinect on Mac so far. Have installed the "PrimeSensor
PSDK 5.0 Driver build 24 for windows" but with Kinect plugged in,
doesn't seem to do anything. Just curious.
Thanks
Kyle Machulis
Dec 8, 2010, 3:03:41 PM12/8/10
to openk...@googlegroups.com
On Wed, Dec 8, 2010 at 10:16 AM, Eidur Arnason <sprot...@googlemail.com> wrote:
Is it possible to dump the vector points through OSC to other programs ? (I am thinking Pure Data)
Yup, should be possible. We need to analyze what all NITE is capable of, what it outputs, and then build an OSC server from there.
On Wed, Dec 8, 2010 at 11:57 AM, Chris OShea <pixe...@gmail.com> wrote:
Great!
Does the driver (1) only work with PrimseSense dev camera, or will it
work with Kinect on Windows, or is there limitations by Microsoft to
do this?
Their driver only works with the PrimeSense reference implementation, but the framework is written os that new "nodes" supporting other hardware (kinect or otherwise) can be implemented. JoshBlake is working on a libfreenect wrapper currently, and there's also a quick hack version at
I've only used Kinect on Mac so far. Have installed the "PrimeSensor
PSDK 5.0 Driver build 24 for windows" but with Kinect plugged in,
doesn't seem to do anything. Just curious.
Yeah, it's looking for a completely different VID/PID, so it won't find the kinect. Best bet for quickest implementation for skeletal tracking stuff is probably the above solution of someone getting an OSC server running on top of the NITE layer. Takes two OSes at that point to do anything, which kinda blows, but since we're dependant on a binary layer for the fun stuff, we'll need OS X support from PS for NITE before it can go direct. OSC seems viable for this though, since it's a lot less data than "The whole image" (though at this point it also sounds like having a local side image server so multiple sources could share the camera output might be a good idea).
holger
Dec 8, 2010, 3:09:40 PM12/8/10
to OpenKinect
can't wait to try this with the kinect. when will you have the module
ready so that we can try it?
isn't it possible to use their driver directly with the kinect?
thanks!
On 8 Dez., 18:18, Joshua Blake <joshbl...@gmail.com> wrote:
> Hello OpenKinect!
>
> A few weeks ago, I was contacted by PrimeSense co-founder Tamir Berliner,
> who wanted to discuss their plans and ways PrimeSense could engage with the
> community. (Microsoft licensed PrimeSense’s technology and based Kinect on
> the PrimeSense reference
> design<http://www.wired.co.uk/magazine/archive/2010/11/features/the-game-cha...>.)
ready so that we can try it?
isn't it possible to use their driver directly with the kinect?
thanks!
On 8 Dez., 18:18, Joshua Blake <joshbl...@gmail.com> wrote:
> Hello OpenKinect!
>
> A few weeks ago, I was contacted by PrimeSense co-founder Tamir Berliner,
> who wanted to discuss their plans and ways PrimeSense could engage with the
> community. (Microsoft licensed PrimeSense’s technology and based Kinect on
> the PrimeSense reference
> After seeing the dozens of Kinect videos and amazing enthusiasm from our
> community, PrimeSense decided to move up their plans and today announced the
> OpenNI <http://www.openni.org/> initiative and have open sourced their
> community, PrimeSense decided to move up their plans and today announced the
> middleware for natural interaction and drivers for their depth cameras.
>
> PrimeSense is setting up OpenNI is being set up as a standards consortium
> for natural interaction, including motion tracking, voice, and others. As a
> part of today's announcement, they have released three things:
>
> 1) Open source drivers for the PrimeSensor reference devicehttp://github.com/PrimeSense/Sensor
>
> PrimeSense is setting up OpenNI is being set up as a standards consortium
> for natural interaction, including motion tracking, voice, and others. As a
> part of today's announcement, they have released three things:
>
>
> This is based upon the same SOC as Kinect. There are probably some knowledge
> in that code!
>
> 2) The open source OpenNI frameworkhttps://github.com/OpenNI/OpenNI
> This is based upon the same SOC as Kinect. There are probably some knowledge
> in that code!
>
>
> This is framework supports pluggable nodes that can let us create
> applications that work against any depth sensor that supports OpenNI and
> take advantage of any node (for example, skeleton tracking) that is
> available.
>
> 3) NITE (Seehttp://openni.org/?q=node/2for download link and license key)
> This is framework supports pluggable nodes that can let us create
> applications that work against any depth sensor that supports OpenNI and
> take advantage of any node (for example, skeleton tracking) that is
> available.
>
>
> This is a binary-only release for Linux and Windows that includes a
> PrimeSense implementation of skeleton tracking and gesture recognition, plus
> sample source code. NITE is an OpenNI module.
>
> I'll be posting more information later about how some of this works. I
> already have started an OpenNI module interface for libfreenect that will
> allow us to plug Kinect in with OpenNI and take advantage of the
> capabilities of OpenNI and NITE, as shown in these videos:
>
> Skeleton Extraction:
> http://www.youtube.com/watch?v=nr8vgCnb9_0
>
> Scene Analyzer (user separation):
> http://www.youtube.com/watch?v=-_j7BzSmlfU
>
> Thanks!
> Josh
>
> ---
> Joshua Blake
> Microsoft Surface MVP
> OpenKinect Community Founderhttp://openkinect.org
> This is a binary-only release for Linux and Windows that includes a
> PrimeSense implementation of skeleton tracking and gesture recognition, plus
> sample source code. NITE is an OpenNI module.
>
> I'll be posting more information later about how some of this works. I
> already have started an OpenNI module interface for libfreenect that will
> allow us to plug Kinect in with OpenNI and take advantage of the
> capabilities of OpenNI and NITE, as shown in these videos:
>
> Skeleton Extraction:
> http://www.youtube.com/watch?v=nr8vgCnb9_0
>
> Scene Analyzer (user separation):
> http://www.youtube.com/watch?v=-_j7BzSmlfU
>
> Thanks!
> Josh
>
> ---
> Joshua Blake
> Microsoft Surface MVP
Chris OShea
Dec 8, 2010, 3:41:35 PM12/8/10
to OpenKinect
@holger
Kyle Machulis had already replied about this, see the email before
yours.
Kyle Machulis had already replied about this, see the email before
yours.
Radu Bogdan Rusu
Dec 8, 2010, 6:00:42 PM12/8/10
to openk...@googlegroups.com
The driver works with Kinect too. You can try our branch, full with ROS integration at https://github.com/ros-pkg-git/ni
Cheers,
Radu.
--
http://pointclouds.org
Message has been deleted
Message has been deleted
Sebastian Ortiz
Dec 10, 2010, 7:21:39 PM12/10/10
to openk...@googlegroups.com
Anyone have detailed instructions on how to use libfreenect with OpenNI and NITE? I did the folllowing with limited success (after switching to SSE2 since I have an older 3.6Ghz P4):
(1) cloned OpenNI , make && make install without any errors (as far as I can tell)
(2) grabbed kinect branch of github.com/boilerbots/Sensor/ make && make install without any errors
(3) Ran OpenNI/Platform/Linux-x86/Bin/Release/NiViewer. This runs but at less than 1 frame/sec.
(3) Ran OpenNI/Platform/Linux-x86/Bin/Release/NiViewer. This runs but at less than 1 frame/sec.
I also built the NITE samples but none of them run due to this error:
"Error initializing: Device Protocol: Bad Parameter sent!"
Has anyone been able to use NITE with freenect but without the enormous dependency of ROS? I'm running gentoo and after a considerable amount of time trying to get the NI branch of ROS working I'm stuck on this step:
rosinstall ~/ni /opt/ros/cturtle /tmp/ni.ri
I assume this means:
rosinstall /NI_install_destination /ros_install_path/ /tmp/ni.ri
but after running that the command fails with:
other config element is not a ros path /projects/ros/stacks
other config element is not a ros path /projects/ros/ros/core
other config element is a ros path /projects/ros/ros
<snip>
other config element is not a ros path /projects/ros/stacks/vision_opencv
other config element is not a ros path /projects/ros/stacks/visualization
other config element is not a ros path /projects/ros/stacks/visualization_common
other config element is not a ros path /projects/ros/stacks/visualization_tutorials
Bootstrapping ROS build
Rospack failed to build
Traceback (most recent call last):
File "/usr/bin/rosinstall", line 5, in <module>
pkg_resources.run_script('rosinstall==0.5.9', 'rosinstall')
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 468, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 1201, in run_script
execfile(script_filename, namespace, namespace)
File "/usr/lib/python2.6/site-packages/rosinstall-0.5.9-py2.6.egg/EGG-INFO/scripts/rosinstall", line 506, in <module>
sys.exit(not rosinstall_main(sys.argv))
File "/usr/lib/python2.6/site-packages/rosinstall-0.5.9-py2.6.egg/EGG-INFO/scripts/rosinstall", line 497, in rosinstall_main
subprocess.check_call("source %s && rosmake ros --rosdep-install" % (os.path.join(options.path, 'setup.sh')), shell=True, executable='/bin/bash')
File "/usr/lib/python2.6/subprocess.py", line 488, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'source /projects/kinect/NI/setup.sh && rosmake ros --rosdep-install' returned non-zero exit status 255
If anyone has detailed instructions on using NITE with ROS, I suppose I'd also be interested. Thanks for any help.
Reply all
Reply to author
Forward
0 new messages