Skeleton tracking algorithm: theoretical basis
4,267 views
Skip to first unread message
Javi
Dec 14, 2010, 9:26:47 AM12/14/10
to OpenNI
Hello everybody,
I am interested in how the skeleton is tracked.
Does anybody know what are the theoretical basis of the algorithm that
performs the skeleton tracking in OpenNI?
How different is it from Microsoft-Xbox algorithm?
I have read that Microsoft algorithm is based on Machine Learning for
body-part-based recognition, but I could not find any paper that
explained the whole process in detail... Here is a web that explains
quite in detail the origin of the Kinect and its contributors, but I
could not retrieve any relevant publications from them... (http://
www.wired.co.uk/magazine/archive/2010/11/features/the-game-changer?page=4)
Thanks in advance!
Javi
I am interested in how the skeleton is tracked.
Does anybody know what are the theoretical basis of the algorithm that
performs the skeleton tracking in OpenNI?
How different is it from Microsoft-Xbox algorithm?
I have read that Microsoft algorithm is based on Machine Learning for
body-part-based recognition, but I could not find any paper that
explained the whole process in detail... Here is a web that explains
quite in detail the origin of the Kinect and its contributors, but I
could not retrieve any relevant publications from them... (http://
www.wired.co.uk/magazine/archive/2010/11/features/the-game-changer?page=4)
Thanks in advance!
Javi
hxflyer
Dec 15, 2010, 1:37:48 AM12/15/10
to OpenNI
Hi javi:
Microsoft use the machine learning approach to detect the human
pose,the Bayesian probabilistic decision is a very common and robust
way to do the computer vision job.
I don't know how OpenNI did it, but I'm trying to use facial
detection to get
the position of human's head, and then use some kinds of "Hough
transform" (http://en.wikipedia.org/wiki/Hough_transform) to detect
the lines around the human's head, because the limbs are straight
lines in the data from depth camera ,and then I assemble the lines
together to make the human pose, in most of the case it is working,
I made a simple test yesterday: http://www.youtube.com/watch?v=WKwvyRfuxBA
but I have 3 major problems:
1 the result is not stable, shaking all the time.
2 if the angel of limb perpendicular to the camera's projection
plane,
it will get error,because there will be no line on the plane anymore,
just a projection point.
3 if the user turn their head more than 60 degree to the camera, then
the facial recognition will fail, the whole process will get error
I'm doubting my approach may fundamentally wrong ,I'm looking for
anybody who have any fresh ideas about this, or could indicate me some
paper/thesis
> could not retrieve any relevant publications from them... (http://www.wired.co.uk/magazine/archive/2010/11/features/the-game-changer?pa...)
>
> Thanks in advance!
>
> Javi
Microsoft use the machine learning approach to detect the human
pose,the Bayesian probabilistic decision is a very common and robust
way to do the computer vision job.
I don't know how OpenNI did it, but I'm trying to use facial
detection to get
the position of human's head, and then use some kinds of "Hough
transform" (http://en.wikipedia.org/wiki/Hough_transform) to detect
the lines around the human's head, because the limbs are straight
lines in the data from depth camera ,and then I assemble the lines
together to make the human pose, in most of the case it is working,
I made a simple test yesterday: http://www.youtube.com/watch?v=WKwvyRfuxBA
but I have 3 major problems:
1 the result is not stable, shaking all the time.
2 if the angel of limb perpendicular to the camera's projection
plane,
it will get error,because there will be no line on the plane anymore,
just a projection point.
3 if the user turn their head more than 60 degree to the camera, then
the facial recognition will fail, the whole process will get error
I'm doubting my approach may fundamentally wrong ,I'm looking for
anybody who have any fresh ideas about this, or could indicate me some
paper/thesis
>
> Thanks in advance!
>
> Javi
Javi
Dec 15, 2010, 6:19:00 AM12/15/10
to OpenNI
Thanks Huang for your response.
I have found at the OpenKinect forum (the thread is entitled 3D Body
Mapping & Skeletal Tracking Experts ) the following post, which gives
interesting information:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Interesting articles on Kinect's skeleton tracking
algorithms:
This was from an internal MS presentation and goes into the most
detail that I've seen thus far
http://research.microsoft.com/en-us/um/redmond/events/fs2010/presentations/Kudo_Fitzgibbons_Kinect_for_Xbox360_071210_FacSummit.pdf
This wired article also references some work from Dr. Jamie Shotton,
Toshiba, and others that inspired the final algorithms (page 4 in
particular)
http://www.google.com/url?sa=D&q=http://www.wired.co.uk/magazine/archive/2010/11/features/the-game-changer&usg=AFQjCNFhmJcSt4TZdkMFSq3BtIi6KXsLYg
A presser for MS Cambridge touting their contributions to Kinect.
http://www.google.com/url?sa=D&q=http://blogs.wsj.com/tech-europe/2010/11/08/key-kinect-technology-devised-in-cambridge-lab/&usg=AFQjCNGxs-Ujs_oTKVzF89BV0TFXh1iBZA
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
About your questions,
Did you think about doing tracking for each body part to improve the
stability?
In the context of your approach, try to use 3D skeletonization istead
of Hough Transform... Although it may fail when the body limbs are
too close. I found this algorithm on the website, coded with MATLAB.
It's quite interesting. http://www.mathworks.com/matlabcentral/fileexchange/4917-skeleton-in-3d
Javi
> > could not retrieve any relevant publications from them... (http://www.wired.co.uk/magazine/archive/2010/11/features/the-game-cha......)
>
> > Thanks in advance!
>
> > Javi
I have found at the OpenKinect forum (the thread is entitled 3D Body
Mapping & Skeletal Tracking Experts ) the following post, which gives
interesting information:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Interesting articles on Kinect's skeleton tracking
algorithms:
This was from an internal MS presentation and goes into the most
detail that I've seen thus far
http://research.microsoft.com/en-us/um/redmond/events/fs2010/presentations/Kudo_Fitzgibbons_Kinect_for_Xbox360_071210_FacSummit.pdf
This wired article also references some work from Dr. Jamie Shotton,
Toshiba, and others that inspired the final algorithms (page 4 in
particular)
http://www.google.com/url?sa=D&q=http://www.wired.co.uk/magazine/archive/2010/11/features/the-game-changer&usg=AFQjCNFhmJcSt4TZdkMFSq3BtIi6KXsLYg
A presser for MS Cambridge touting their contributions to Kinect.
http://www.google.com/url?sa=D&q=http://blogs.wsj.com/tech-europe/2010/11/08/key-kinect-technology-devised-in-cambridge-lab/&usg=AFQjCNGxs-Ujs_oTKVzF89BV0TFXh1iBZA
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
About your questions,
Did you think about doing tracking for each body part to improve the
stability?
In the context of your approach, try to use 3D skeletonization istead
of Hough Transform... Although it may fail when the body limbs are
too close. I found this algorithm on the website, coded with MATLAB.
It's quite interesting. http://www.mathworks.com/matlabcentral/fileexchange/4917-skeleton-in-3d
Javi
>
> > Thanks in advance!
>
> > Javi
Javi
Dec 15, 2010, 11:10:12 AM12/15/10
to OpenNI
FYI:
More material about Kinect skeleton / gesture tracking on
OpenKinect.org
http://openkinect.org/wiki/Research_Material
On 15 dic, 12:19, Javi <javioli...@gmail.com> wrote:
> Thanks Huang for your response.
>
> I have found at the OpenKinect forum (the thread is entitled 3D Body
> Mapping & Skeletal Tracking Experts ) the following post, which gives
> interesting information:
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> Interesting articles on Kinect's skeleton tracking
> algorithms:
>
> This was from an internal MS presentation and goes into the most
> detail that I've seen thus farhttp://research.microsoft.com/en-us/um/redmond/events/fs2010/presenta...
>
> A presser for MS Cambridge touting their contributions to Kinect.http://www.google.com/url?sa=D&q=http://blogs.wsj.com/tech-europe/201...
More material about Kinect skeleton / gesture tracking on
OpenKinect.org
http://openkinect.org/wiki/Research_Material
On 15 dic, 12:19, Javi <javioli...@gmail.com> wrote:
> Thanks Huang for your response.
>
> I have found at the OpenKinect forum (the thread is entitled 3D Body
> Mapping & Skeletal Tracking Experts ) the following post, which gives
> interesting information:
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> Interesting articles on Kinect's skeleton tracking
> algorithms:
>
> This was from an internal MS presentation and goes into the most
>
> This wired article also references some work from Dr. Jamie Shotton,
> Toshiba, and others that inspired the final algorithms (page 4 in
> particular)http://www.google.com/url?sa=D&q=http://www.wired.co.uk/magazine/arch...
> This wired article also references some work from Dr. Jamie Shotton,
> Toshiba, and others that inspired the final algorithms (page 4 in
>
> A presser for MS Cambridge touting their contributions to Kinect.http://www.google.com/url?sa=D&q=http://blogs.wsj.com/tech-europe/201...
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
>
> About your questions,
> Did you think about doing tracking for each body part to improve the
> stability?
>
> In the context of your approach, try to use 3D skeletonization istead
> of Hough Transform... Although it may fail when the body limbs are
> too close. I found this algorithm on the website, coded with MATLAB.
> It's quite interesting.http://www.mathworks.com/matlabcentral/fileexchange/4917-skeleton-in-3d
>
> About your questions,
> Did you think about doing tracking for each body part to improve the
> stability?
>
> In the context of your approach, try to use 3D skeletonization istead
> of Hough Transform... Although it may fail when the body limbs are
> too close. I found this algorithm on the website, coded with MATLAB.
likeBVH
Dec 15, 2010, 4:38:58 PM12/15/10
to OpenNI
George Toledo
Dec 16, 2010, 10:00:49 AM12/16/10
to openn...@googlegroups.com
It's probably also worth noting that one can detect front or profile face, with OpenCV and haar files, which will indicate basic orientation of a body. Then, by comparing frames, one can start id-ing people, by color coding them, and/or their bodies with some kind of coordinate mask, so that one can isolate the vectors of the extremities of the body.
-GT
--
On Wed, Dec 15, 2010 at 4:38 PM, likeBVH <jim...@googlemail.com> wrote:
FYI
http://www.youtube.com/watch?v=Qca4qDspLyk&NR=1
--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to openn...@googlegroups.com.
To unsubscribe from this group, send email to openni-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.
--
George Toledo
The information contained in this E-mail and any attachments may be confidential.
If you have received this E-mail in error, please notify us immediately by telephone or return E-mail.
You should not use or disclose the contents of this E-mail or any of the attachments for any purpose or to any persons.
likeBVH
Dec 16, 2010, 10:13:31 AM12/16/10
to OpenNI
I think 3D skeletonization or 2D skeletonization using depthmap is a
good option to test. it should be interesting to see how using Nite's
skeleton will help in stabilizing/filtering the skeletonization data
and to extend the number of extracted body parts beyond 15. Any
feedback?
good option to test. it should be interesting to see how using Nite's
skeleton will help in stabilizing/filtering the skeletonization data
and to extend the number of extracted body parts beyond 15. Any
feedback?
Rhys Lewis
Jan 16, 2011, 5:23:35 PM1/16/11
to OpenNI
Reading the Wired article, and judging by contributors efforts with
skeletons, it looks like the current Nite implementation is a few
generations behind the XBox, for example the XBox seems to instantly
calibrate a skeleton to drive an avatar when a new player jumps in
front, but Open NI/Nite requires a PSI pose to calibrate.
Is that correct, or is there a quick way recognise fresh users?
Regards,
Rhys Lewis
On Dec 16 2010, 12:19 am, Javi <javioli...@gmail.com> wrote:
> Thanks Huang for your response.
>
> I have found at the OpenKinect forum (the thread is entitled 3D Body
> Mapping & Skeletal Tracking Experts ) the following post, which gives
> interesting information:
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> Interesting articles on Kinect'sskeletontracking
> algorithms:
>
> This was from an internal MS presentation and goes into the most
> detail that I've seen thus farhttp://research.microsoft.com/en-us/um/redmond/events/fs2010/presenta...
>
> A presser for MS Cambridge touting their contributions to Kinect.http://www.google.com/url?sa=D&q=http://blogs.wsj.com/tech-europe/201...
> > > I am interested in how theskeletonis tracked.
skeletons, it looks like the current Nite implementation is a few
generations behind the XBox, for example the XBox seems to instantly
calibrate a skeleton to drive an avatar when a new player jumps in
front, but Open NI/Nite requires a PSI pose to calibrate.
Is that correct, or is there a quick way recognise fresh users?
Regards,
Rhys Lewis
On Dec 16 2010, 12:19 am, Javi <javioli...@gmail.com> wrote:
> Thanks Huang for your response.
>
> I have found at the OpenKinect forum (the thread is entitled 3D Body
> Mapping & Skeletal Tracking Experts ) the following post, which gives
> interesting information:
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> Interesting articles on Kinect'sskeletontracking
> algorithms:
>
> This was from an internal MS presentation and goes into the most
>
> This wired article also references some work from Dr. Jamie Shotton,
> Toshiba, and others that inspired the final algorithms (page 4 in
> particular)http://www.google.com/url?sa=D&q=http://www.wired.co.uk/magazine/arch...
> This wired article also references some work from Dr. Jamie Shotton,
> Toshiba, and others that inspired the final algorithms (page 4 in
>
> A presser for MS Cambridge touting their contributions to Kinect.http://www.google.com/url?sa=D&q=http://blogs.wsj.com/tech-europe/201...
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
>
> About your questions,
> Did you think about doing tracking for each body part to improve the
> stability?
>
> In the context of your approach, try to use 3D skeletonization istead
> of Hough Transform... Although it may fail when the body limbs are
> too close. I found this algorithm on the website, coded with MATLAB.
> It's quite interesting.http://www.mathworks.com/matlabcentral/fileexchange/4917-skeleton-in-3d
>
> About your questions,
> Did you think about doing tracking for each body part to improve the
> stability?
>
> In the context of your approach, try to use 3D skeletonization istead
> of Hough Transform... Although it may fail when the body limbs are
> too close. I found this algorithm on the website, coded with MATLAB.
>
> > > Does anybody know what are the theoretical basis of the algorithm that
> > > performs theskeletontracking in OpenNI?
> > > Does anybody know what are the theoretical basis of the algorithm that
jack zill
Nov 16, 2012, 12:51:58 AM11/16/12
to openn...@googlegroups.com
hi,
Does anybody have complete Matlab code for Human Skeleton Tracking ?
Thanks.
Jack
Reply all
Reply to author
Forward
0 new messages