How to compare tangent vectors in a common tangent space

[sun...@gmail.com]

Hi, everyone,

I would like to classify different points on manifold, for example, a manifold of symmetric positive definite matrices. Because I want to use machine learning methods in Euclid space, so the points on manifold need to be projected to the tangent space. I think there are two approaches to solve this problem:

1. Project a set of symmetric positive definite matrices (all the matrices) in one tangent space according to the given reference point (for example, the mean of matrices is a good candidate for reference point ) 

2. By using Parallel translation, move all tangent vector in different tangent space to a common tangent space. 

When the data distribute on manifold in a small scale, the first approach is appropriate since the manifold is isomorphism to a Euclid space locally. However, when the data are “too far apart”, this definition is not satisfactory. So I think the second approach is more accurate. Parallel transport provides a way to compare a vector in one tangent plane to a vector in another, by moving the vector along a curve without changing it.

My question is how to carry out the Parallel transport in Manopt. I have found a function in sympositivedefinitefactory() as:

function zeta = parallel_transport(X, Y, eta)

E = sqrtm((Y/X));

zeta = E*eta*E';

end

But I did not understand how to use it, for example, I would like to move a tangent vector eta in a tangent space rooted X (a point on manifold) to the tangent space rooted by Y. In particular, what is  eta when I want to move the  eta correspond to X, namely, move the tangent vector of X. Is  eta a zero vector in this case? If eta is a zero vector, the return value of the function parallel_transport is always zero matrix. So I'm puzzled about this situation. 

Any help about my question?

Thanks in advance.

Best regards

Jiancheng

[BM]

Hello Jiancheng,

Thanks for the question.

Eta is a tangent vector at X. The function parallel_transport(X, Y, eta) transports the tangent vector eta at X to Y. The output of the function is a tangent vector at Y. Also, if eta is a zero vector, then the output of parallel_transport is a zero vector as well. You can call the function in the following way.

n = 10;
M = sympositivedefinitefactory(n);

X = M.rand(); % A random point.
eta = M.randvec(X); % A random tangent vector at X.
Y = M.rand(); % Another random point.

Zeta = M.paralleltransp(X, Y, eta); % The transported tangent vector at Y.

Zeta is the transported tangent vector at Y. 

Let us know if you have further queries.

Regards,

BM

[sun...@gmail.com]

Hello BM,

Thanks for your timely response and I know how to use  paralleltransp now. However I still puzzled with comparing different points in a common tangent space. For example, I would like to compare two points X and Y on manifold in Euclid space, so this problem can be converted into comparing the tangent vectors of X and Y in a common tangent space rooted by another point Z (a reference point in manifold). Generally, the tangent space respect to X and Y are two different tangent spaces. With regard to X, I think that a appropriate tangent space is rooted by itself. So the tangent vector x corresponds to X is a zero vector (origin of the tangent space). Am I right? If that's the case, by  using  paralleltransp, the transported tangent vector of x at Z is also a zero vector. The same goes for the transported tangent vector for Y. Finally, we can not tell the difference between X and Y in the common tangent space since their transported tangent vectors are same value, namely, zero vectors. I can not figure out where I went wrong. So could you please give some suggestion to me?

Thanks a lot

Best regards

Jiancheng

[BM]

Hello Jiancheng,

The problem lies in how do we compare two points on a manifold in general. There is no notion as such to compare two points. This is why, in my opinion, you end up transporting a zero vector. 

However, for positive definite matrices, there is a "special" natural ordering of elements. Lets assume that X and Y are two positive definite matrices, then X >= Y, iff X - Y >= 0, which is that all the eigenvalues of X-Y are >= 0.

I guess, you are trying to relate this natural ordering to manifold operations. However, I am not sure whether there exists such a relationship.

A guess: one approach would be to say that X >= Y, iff X - I >= Y - I, where I is the identity matrix.  Construct a log map from I to X and from I to Y, both on the tangent space at I. Now, we could compare the magnitude of the log map vectors to get the ordering. We need to verify whether this holds true for all scenarios. 

Regards,

Bamdev

[sun...@gmail.com]

Hi Bamdev,

Thank you very much for your answer. Yes, I think your guess is right ( X >= Y, iff X - I >= Y - I), at least in experiment simulation. However, I think a better solution is to replace I with the mean of points set on manifold. That is to say, as your guess, construct a log map from c to X and from c to Y, both on the tangent space at c, here the c is mean of the points set. I have carried out the simulation and the results based on c is better than I ( identity matrix) in most of cases. In theory, this approach is appropriate when the data distribute on manifold in a small scale. As mentioned in my original questions, project a set of symmetric positive definite matrices (all the matrices) in one tangent space according to the given reference point. So how do we compare all points on the manifold in a common tangent space when the data are “too far apart” ? In this case, I think project all the points to one tangent space is not appropriate since the manifold is isomorphism to a Euclid space locally. Am I right? 

Thanks.

Regards

Jiancheng 

[BM]

Hello Jiancheng, 

Interesting. 

I assume that we are referring positive definite matrices. In this case, in my opinion, local isomorphism is a non issue. In fact, parallel transport on the set of positive definite matrices captures the global (geodesically complete) manifold structure and should be the right way to compare far away points at a reference point, say c. A more appropriate way would be to define c as the geometric mean of X and Y. What are your thoughts on this?

Regards,

Bamdev

[sun...@gmail.com]

Dear Bamdev,

I think your suggestion is appropriate. So, if I take your meaning right, I can compare all points in a common tangent space at a reference point c by parallel transport. But, a problem is how to choose the  points of departure. For example, if I want to move the tangent vector eta of point Z by function parallel_transport(X, c, eta), here c is the reference point (target point). This function move eta at X to the tangent space rooted by c. That is, eta is in the tangent space rooted by X which is different with the tangent space rooted by c. That is, the eta is a tangent vector by log map of Z-X , am I right? If that's the case, the problem is how to choose the X. Can I choose the nearest neighbor of Z based on geodesic distance? Maybe I misunderstood something.

Thanks

Regards

Jiancheng

[BM]

Hello Jiancheng, 

Sorry for the late reply. 

If c is the geometry mean of X and Y, then the distance between X and c is equal to distance between Y and c. The geometric mean, therefore, may not be a good comparison point. 

Ideally, c should be be zero. May be, as a guess, we could say that c is some small constant times the identity matrix. Taking c as the reference point, we could compare distance in the following way without requiring the notion of parallel transport.

n = 10;
M = sympositivedefinitefactory(n);

X = M.rand(); % A random point.

Y = M.rand(); % Another random point.

c = 1e-4*eye(n); % Ideally, we should be comparing with the 0 matrix.

ctoX = M.log(c, X);% Log map rooted at c, captures the notion X - c.

ctoY = M.log(c, Y);% Log map rooted at c, captures the notion Y - c.

% Compare the norms

M.norm(c, ctoX)  % Distance between c and X. 

M.norm(c, ctoY)  % Distance between c and Y. 

(Taking c closer to zero, might lead to numerical problems.)

Let me know whether this helps.

Regards,
Bamdev