Getting joint Orientation in degrees

[kinector]

Hi you all

I've been working with OpenNI for about a week now and so far I'm very
impressed how easy it is to use with kinect and get great results. I'm
working with the .net lib (x86) c#. So far I've got only one problem I
couldn't figure out by my own. In order to capture *.bvh-files I need
to get the joint orientations in degrees but the framework gives me
nine values (X1, X2, X3, Y1, Y2, Y3, Z1, Z2, Z3)
[GetJointOrientation()] and I can't figure out how to get the values
to degrees. So far I realized that these values are cos or sin values
because they aren't bigger or smaller than 1 and -1. I've also found
in the open-kinect specification, that the kinect appears to deliver
degree values that's the reason why I'm writing to you guys.

Anyway, the project is very great work and I'll look forward to read
your reply.

Kind regards

Kinector

[kart0grapher]

I've also been struggling with a similar problem. OpenNI has a strange
convention for the matrices, and I've yet to find a way to correctly
convert them to Euler angles of the right convention. A work around
might be using vectors to point to each joint and calculating Euler
angles from that. The Configurable Math Library (C++) includes a
couple functions (matrix_rotation_vec_to_vec and matrix_to_euler) that
I'm hoping will correctly calculate Euler angles for writing to bvh
files.

The only problem I see with this approach is calculating the Euler
angles of the root joint, which is not relative to anything else and
therefore cannot be calculated in the same manner as the other joints.
Anyone have any thoughts on how to solve this problem?

[Rodrigo Ibañez]

i don't understand the meaning of the joint orientation. 

What is the information that this provides?

[kart0grapher]

The joint orientation is a matrix that describes the rotation of
joints. From the NITE documentation:

Joint orientations are given as a 3x3 rotation (orthonormal) matrix.
This represents a rotation between the joint's local coordinates and
the world coordinates. The first
column is the direction of the joint"s +X axis given as a 3-vector in
the world coordinate system. The second column is the +Y axis
direction, and the third column is
the +Z axis direction. Our "neutral pose" is the T-pose shown in the
figure above. In this pose, each joint's orientation is aligned with
the world coordinate system. That is, its orientation is the identity
matrix.

Unfortunately, it is not clear how to get this rotation matrix into
Euler angles of the correction convention for bvh files (ZXY). Any
thoughts?

[kinector]

I'm glad to see I'm not the only one with this problem. I've written
an Email to the OpenNI support. Luckily we will get our informations
there I let you know when they write back.

[MichaelK]

I would also prefer euler angles. Right now I use following code in
unity3d:

public static Quaternion
SkeletonJointOrientationToQuaternion(SkeletonJointOrientation orient)
{
Vector3 forward = new Vector3(-orient.X3, -orient.Y3, orient.Z3);
Vector3 upwards = new Vector3(-orient.X2, orient.Y2, -orient.Z2);

return Quaternion.LookRotation(forward, upwards);

[kinector]

Hello Michael

Thank you for your reply. It seems like you've got it to work.
Unfourtunatly I'm not familiar with quaternions and I couldn't figure
out how to convert them to degrees. If you know a way to get 3
rotation values in degrees I will finaly be able to get my motion
capturing to work. If there is a .net lib that converts 3 vecs into
degrees that would be assome too.

But anyway thanks you added new parts to the puzzle. I will check if I
can figure out more and maybe get the solution.

> > there I let you know when they write back.- Zitierten Text ausblenden -
>
> - Zitierten Text anzeigen -

[kinector]

Here is what I can do with .net.

private float test(SkeletonJointOrientation orientation)
{
Vector3 forward = new Vector3(-orientation.X3, -
orientation.Y3, orientation.Z3);
Vector3 upwards = new Vector3(-orientation.X2,
orientation.Y2, -orientation.Z2);

}

To bad they haven't these lookAt function.

I belive we are getting closer. I can feel it ;-)

On 21 Jun., 18:34, MichaelK <email.kuz...@googlemail.com> wrote:

[Tim Tregubov]

I had this question at some point too and came up with this probably
overly robust way to convert the orientation to a quaternion (the link
the in comments describes the method):


/// <summary>
/// convert a matrix rotation to a quaternion
/// based on: http://www.euclideanspace.com/maths/geometry/rotations/conversions/matrixToQuaternion/index.htm
/// </summary>
/// <param name="m">
/// A <see cref="SkeletonJointOrientation"/>
/// </param>
/// <returns>
/// A <see cref="Quaternion"/>
/// </returns>
public static Quaternion matrixToQuat(SkeletonJointOrientation m)
{
float tr = m.X1 + m.Y2 + m.Z3;
float qw = 0f;
float qx = 0f;
float qy = 0f;
float qz = 0f;

if (tr > 0)
{
float S = Mathf.Sqrt(tr+1.0f) * 2f; // S=4*qw
qw = 0.25f * S;
qx = (m.Y3 - m.Z2) / S; //(m21 - m12) / S;
qy = (m.Z1 - m.X3) / S; //(m02 - m20) / S;
qz = (m.X2 - m.Y1) / S; //(m10 - m01) / S;
} else if ((m.X1 > m.Y2) && (m.X1 > m.Z3))//((m00 > m11)&(m00 > m22))
{
{
float S = Mathf.Sqrt(1.0f + m.X1 - m.Y2 - m.Z3) * 2f; //sqrt(1.0 +
m00 - m11 - m22) * 2; // S=4*qx
qw = (m.Y3 - m.Z2) / S;//(m21 - m12) / S;
qx = 0.25f * S;
qy = (m.Y1 + m.X2) / S;//(m01 + m10) / S;
qz = (m.Z1 + m.X3) / S;//(m02 + m20) / S;
} else if (m.Y2 > m.Z3)//(m11 > m22)
{
float S = Mathf.Sqrt(1.0f + m.Y2 - m.X1 - m.Z3) * 2f;//sqrt(1.0 +
m11 - m00 - m22) * 2; // S=4*qy
qw = (m.Z1 - m.X3) / S; //(m02 - m20) / S;
qx = (m.Y1 + m.X2) / S; //(m01 + m10) / S;
qy = 0.25f * S;
qz = (m.Z2 + m.Y3) / S; //(m12 + m21) / S;
} else
{
float S = Mathf.Sqrt(1.0f + m.Z3 - m.X1 - m.Y2) * 2f;//sqrt(1.0 +
m22 - m00 - m11) * 2; // S=4*qz
qw = (m.X2 - m.Y1) / S;//(m10 - m01) / S;
qx = (m.Z1 + m.X3) / S;//(m02 + m20) / S;
qy = (m.Z2 + m.Y3) / S;//(m12 + m21) / S;
qz = 0.25f * S;
}

return new Quaternion(qx, qy, qz, qw);
}


Cheers,
Tim

[kinector]

Thanks dude that looks like I can get it easy to work.

On 21 Jun., 22:42, Tim Tregubov <timo...@gmail.com> wrote:
> I had this question at some point too and came up with this probably
> overly robust way to convert the orientation to a quaternion (the link
> the in comments describes the method):
>
> /// <summary>
> /// convert a matrix rotation to a quaternion

> /// based on:  http://www.euclideanspace.com/maths/geometry/rotations/conversions/ma...

> > > - Zitierten Text anzeigen -- Zitierten Text ausblenden -

[MichaelK]

Hi Tim, did you tested this code and are the results right?

On Jun 21, 10:42 pm, Tim Tregubov <timo...@gmail.com> wrote:
> I had this question at some point too and came up with this probably
> overly robust way to convert the orientation to a quaternion (the link
> the in comments describes the method):
>
> /// <summary>
> /// convert a matrix rotation to a quaternion

> /// based on:  http://www.euclideanspace.com/maths/geometry/rotations/conversions/ma...

[Tim Tregubov]

Yep, although you may need to change the orientation depending on what
you are doing you may have to flip the Z axis. I just do it by
creating a new joinorientation and then running it through the matrix
to quat conversion.

/// <summary>
/// converts an openni matrix rotation to a unity quaternion
/// flipping z axis.

/// </summary>
/// <param name="m">
/// A <see cref="SkeletonJointOrientation"/>
/// </param>
/// <returns>
/// A <see cref="Quaternion"/>
/// </returns>

public static Quaternion openNIMatrixToQuat(SkeletonJointOrientation
m)
{
SkeletonJointOrientation n = new SkeletonJointOrientation();
n.X1 = m.X1;
n.X2 = m.X2;
n.X3 = -m.X3;
n.Y1 = m.Y1;
n.Y2 = m.Y2;
n.Y3 = -m.Y3;
n.Z1 = -m.Z1;
n.Z2 = -m.Z2;
n.Z3 = m.Z3;

return matrixToQuat(n);

[Dan]

If you want to get Euler angles from a rotation matrix, you can factor
it as a product of three rotation matrices in any order. For example,
XYZ vs. YZX. The order does matter, so you'll need to figure out which
matches your application and engine use.

Dave Eberly's excellent website has all the math for this as well as a
plethora of other useful information much of which comes from his book
Geometric Tools for Computer Graphics. http://www.geometrictools.com/

All the angle extraction formula were in his Geometric Tools book and
are luckily also posted online along with pseudocode.
http://www.geometrictools.com/Documentation/EulerAngles.pdf

dba

[kinector]

Hello Dan

I tried this but you can only get one angle out of the quarternion and
so I'm stuck again. I don't no if this is stupid but I wanna ask
again: Do somebody know how to get these nine values in three float
values that stands for degrees. I have still no idea what the nine
values are meaning exept that they are part of a rotation matrix. I
realy appreciate the answers and they gave me some new points for
research. But after days of research trough the web I am not able to
complete the program. OpenNI didn't wrote me back neither and all this
is a bit frustrating. But I don't want to be thankless and I'm
impressed by how quick you all react on my request.

Ok enough of the crying. If you have further information let me now. I
will of course post new infos too.

When this is over I will deffinitly write this down at a place where
other people can find it since the documentation is missing it.

> > > > > - Zitierten Text anzeigen -- Zitierten Text ausblenden -

[kinector]

Sorry, Actualy the message should more refer to tim's example. I'm a
bit tired I guess (this thing is F...ING hard to figure out) ^^

On 23 Jun., 21:30, kinector <dw.dominik.wer...@gmail.com> wrote:
> Hello Dan
>
> I tried this but you can only get one angle out of the quarternion and
> so I'm stuck again. I don't no if this is stupid but I wanna ask
> again: Do somebody know how to get these nine values in three float

> values that stands fordegrees. I have still no idea what the nine

[Tim Tregubov]

hey kinector yeah sorry i gave you the conversion from matrix to
quaternion but for euler angles you can find the maths and sample code
here:
http://www.euclideanspace.com/maths/geometry/rotations/conversions/matrixToEuler/index.htm

[ROMBO]

Hi kinector!

Have you tried cv::Rodrigues?

http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html

I began using OpenCV with normal cameras. After purchase a kinect
camera, I use both libraries
OpenCV and OpenNI, getting the best of each ;-)

cv::Rodrigues allows to convert a 3x3 matrix into a three component
rotation vector or vice versa

[c.h...@gmx.de]

Hi,

i had a similar question a view days ago:

https://groups.google.com/forum/#!topic/openni-dev/OePFFmDRFpw

As i understood the XnMatrix3x3 is a Rotationmatrix {ax,bx,cx,ay,by,cy,az,bz,cz} which is calculated for example in this way:
R(x) * R(y) * R(z) = R(xyz), where Rx is the Rotation around x, and so on. The matrix contains 3 vectors with the lenght of 1
a = (ax,ay,az), b = (bx,by,bz), c = (cx,cy,cz), where every vector can be imagined as an axis of the local coordinate system of a joint.
The angle between those vectors is always 90°.

The problem is in the calulation of the R(xyz) Matrix, which could also be  R(zyx) e.g. or for Kardan-angles R(xyx) e.g. Dan is right,
if we would know the order of concatination ,we could calculate the 3 angles by factorisation from the XnMatrix3X3. I tried this for R(xyz) (see my ealier post) and get 3 angles, but they dont seem to be right. I get wired orientations for the arms, but the leg and upper body tracking seems to generate good results.

Either the concatenation order is another than R(xyz), or there is a problem with my code (or maybe the arm orientations).

Anyone tried another order for concatenation than R(xyz)? If i suceed in figure out the right way, i will tell u.

Greets and sorry for my bad english

[Dan]

I haven't used cv::Rodrigues. It may be the easiest approach, but
Eberly's doc should do the trick as well.

Given the OpenNI values (X1, X2, X3, Y1, Y2, Y3, Z1, Z2, Z3) as you
described, construct a matrix where each axis is a column.

[[X1, Y1, Z1]
[X2, Y2, Z2]
[X3, Y3, Z3]]

Then, factor that using Eberly's formula. If you wanted RxRyRz, then
you would use the first bit of pseudocode which I've doctored up here
to use the X1, X2, ... convention rather than Eberly's r00 convention.
You can do the similar transform for other factorizations.
(Disclaimer: I haven't tested this, so please check for a
transcription error and run a test or two on a hand-computed matrix
before you trust my conversion here.)

if (Z1 < +1)
{
if (Z1 > -1)
{
thetaY = asin(Z1);
thetaX = atan2(-Z2,Z2);
thetaZ = atan2(-rY1,X1);
}
else // Z1 = -1
{
// Not a unique solution: thetaZ - thetaX = atan2(X2,Y2)
thetaY = -PI/2;
thetaX = -atan2(X2,Y2);
thetaZ = 0;
}
}
else // Z1 = +1
{
// Not a unique solution: thetaZ + thetaX = atan2(X2,Y2)
thetaY = +PI/2;
thetaX = atan2(X2,Y2);
thetaZ = 0;
}

If you want more info on how to solve these problems in the future, I
seriously recommend buying some of Dave Eberly's books or perhaps get
Van Verth and Bishop's Essential Math for Games Programmers. Van Verth
and Bishop is easier to understand, but it has nowhere near the info
and depth that you'll find in a book like Eberly's Geometric Tools
which dives into a lot more topics.

dba

On Jun 23, 10:49 pm, ROMBO <cmog...@gmail.com> wrote:
> Hi kinector!
>
> Have you tried cv::Rodrigues?
>

> http://opencv.willowgarage.com/documentation/cpp/camera_calibration_a...

[kinector]

Wow that's cool. As soon as I've can get some time I will try your
methods.

[Yassir Ennazk]

Hi Dan,

Your code was extremely helpful but there is some mistyping in there.

Here is the correction :

    if (Z1 < +1)

    {

        if (Z1 > -1)

        {

            thetaY = asin(Z1);

            thetaX = atan2(-Z2,Z3);

            thetaZ = atan2(-Y1,X1);

        }

        else

        {

            thetaY = -M_PI/2;

            thetaX = -atan2(X2,Y2);

            thetaZ = 0;

        }

    }

    else

    {

        thetaY = +M_PI/2;

        thetaX = atan2(X2,Y2);

        thetaZ = 0;

    }

Thanks

[phil217]

Hi,
I've uploaded a few days ago a small algorithm that uses the Microsoft
SDK and exports all the joint angles and coordinates data to XML or
CSV:
http://computing-technologies.com/softwares

I'm still developing the software, maybe the algorithm can help you
figure out everything...

[shorn william]

Hi, Have any get the correct BVH to import into 3D Max

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.

To post to this group, send email to openn...@googlegroups.com.
To unsubscribe from this group, send email to openni-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

--
Good Luck and Have a day!

William Shorn
Mobile: 13811278218
MSN: willia...@hotmail.com
BeiJing People's Republic of China

[shorn william]

Hi

 

I get the eula angle by the orientation matrix from oOpenNI and found that the orientation and coordinate is not compatible, does any know how to convert it from OpenNI to BVH?

I 

[Rodrigo Ibañez]

Hi, Shorn 

I could make bvh header. but when I try to add frames of animation it's breaks everything.

I think I have your same problem.

2012/5/15 shorn william <shor...@gmail.com>

[Burak Orçun Özkablan]

I have same problem. I use OpenNI to get skeleton data, I convert rotation matrices of all joints to Euler angles and I write them into BVH file.

For example, I have a bvh file that contains arms motions. When I play bvh file, my arms in bvh moves weird also my feets are weird although I didnt move them.

I use rotation matrices from OpenNI and know sequence of Euler angles of bvh file, ZXY sequence. 

Anybody who suggests way to calculate correct angles?

[Pedro Ribeiro]

Hi. The rotation matrices from OpenNI don't seem to be correct( at least for the harms).

You can try ViiM SDK (www.viim.pt). We used the openNI position joints and recalculated all the rotation matrices form scratch in a parent child relationship.

It's very easy to use, go to http://wiki.viim.pt/Tutorials and read the stickman example, it explains how to use the rotation matrices.

Then give us your feedback on the experience so that we can improve it.

pedro r
covii.pt

[Burak Orçun Özkablan]

Hi Pedro,

Thank you for ViiM SDK, I browsed samples of SDK. In stickman sample, there are some rotation matrices of all joints from getUserData function.

Are they global rotation matrices or local rotation matrices these are dependent parent? 

If you calculate them with joints positions by local, what is calculation method? Is it directed vector?

In tutorial, there is a sentence as

"In line 108 we get the data from the skeleton joints (rotation matrix, position, quaternions, angles of rotation and confidence)" 

Does getUserData function return all information of joints such as quaternion and angles of rotation? or Did you calculate them?

18 Aralık 2012 Salı 12:06:58 UTC+2 tarihinde Pedro Ribeiro yazdı:

[Pedro Ribeiro]

Hi Burak!

On Tue, Dec 18, 2012 at 12:59 PM, Burak Orçun Özkablan <borcuno...@gmail.com> wrote:

Hi Pedro,

Thank you for ViiM SDK, I browsed samples of SDK. In stickman sample, there are some rotation matrices of all joints from getUserData function.

Are they global rotation matrices or local rotation matrices these are dependent parent? 

They are local rotation matrices, each one relative to its parent. The elbow rotation matrix is relative to the shoulder matrix, the shoulder one to the torso, and so on.

Please read ViiM's documentation: http://wiki.viim.pt/Technical_documents

And there is a chapter about the skeleton where this is clearly explained: http://www.covii.pt/wiki/docs/ViiMchp5.pdf

If you calculate them with joints positions by local, what is calculation method? Is it directed vector?

Each column of the rotation matrix represents the direction of one axis of its own coordinate system (x, y and z), with relation to the current orientation of the coordinate system of its parent.

In tutorial, there is a sentence as

"In line 108 we get the data from the skeleton joints (rotation matrix, position, quaternions, angles of rotation and confidence)" 

Does getUserData function return all information of joints such as quaternion and angles of rotation? or Did you calculate them?

Yes, getUserData returns all this information for all the joints. From the variable you use (in the example case its named _skeletonData) you can select the information you want.

From a scientific point of view it is always better to use either the quaternions or the rotation matrices, because to use the angles of rotation you need to pay attention to the order you rotate things... (Which will certainly give you headaches sooner or latter)
 
happy to help :)

pedro r.

18 Aralık 2012 Salı 12:06:58 UTC+2 tarihinde Pedro Ribeiro yazdı:

Hi. The rotation matrices from OpenNI don't seem to be correct( at least for the harms).

You can try ViiM SDK (www.viim.pt). We used the openNI position joints and recalculated all the rotation matrices form scratch in a parent child relationship.

It's very easy to use, go to http://wiki.viim.pt/Tutorials and read the stickman example, it explains how to use the rotation matrices.

Then give us your feedback on the experience so that we can improve it.

pedro r
covii.pt

--

You received this message because you are subscribed to the Google Groups "OpenNI" group.

To view this discussion on the web visit https://groups.google.com/d/msg/openni-dev/-/umtORnXL7vYJ.