[Numpy-discussion] SVD problem - matrices are not aligned

[Daniel Wagner]

Hi,

I'm a new subscriber of this list. I hope to directly start with a question is ok...

My question or problem:
I've a matrix A which is calculated from the data b. The shapes of these matrices are:
>>>A.shape
(954, 9)
>>>b.shape
(954,)

I calculate the SVD of A:
>>> U, w, V = numpy.linalg.svd(A, full_matrices="True")
>>>U.shape
(954, 954)
>>>W.diag(w)
>>>W.shape
(9,9)
>>>V.shape
(9,9)

If I'm doing the check of the SVD results using:
>>>numpy.allclose(A, numpy.dot(U, numpy.dot(W, V)))
I get this error:
"ValueError: matrices are not aligned"

I'just don't get where the error does come from. There seems to be a general problem with my data or how I'm using the svd function. I hope anybody has an idea what I'm doing wrong or where my problem comes from. I really appreciate any suggestions!

Many thanks!
Greetings,
Daniel Wagner
_______________________________________________
NumPy-Discussion mailing list
NumPy-Di...@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Charles R Harris]

On Sat, Oct 23, 2010 at 7:00 PM, Daniel Wagner <daniel.wagner.ml@googlemail.com> wrote:

Hi,

I'm a new subscriber of this list. I hope to directly start with a question is ok...

My question or problem:
I've a matrix A which is calculated from the data b. The shapes of these matrices are:
>>>A.shape
(954, 9)
>>>b.shape
(954,)

I calculate the SVD of A:
>>> U, w, V = numpy.linalg.svd(A, full_matrices="True")
>>>U.shape
(954, 954)

You want full_matrices set false so that U has shape (954, 9).
 

>>>W.diag(w)
>>>W.shape
(9,9)
>>>V.shape
(9,9)

If I'm doing the check of the SVD results using:
>>>numpy.allclose(A, numpy.dot(U, numpy.dot(W, V)))
I get this error:

easier, allclose(A, dot(U*w, V) )

"ValueError: matrices are not aligned"

Mismatched dimensions.

Chuck

[Daniel Wagner]

On Sat, Oct 23, 2010 at 7:00 PM, Daniel Wagner <daniel.wagner.ml@googlemail.com> wrote:

Hi,

I'm a new subscriber of this list. I hope to directly start with a question is ok...

My question or problem:
I've a matrix A which is calculated from the data b. The shapes of these matrices are:
>>>A.shape
(954, 9)
>>>b.shape
(954,)

I calculate the SVD of A:
>>> U, w, V = numpy.linalg.svd(A, full_matrices="True")
>>>U.shape
(954, 954)

You want full_matrices set false so that U has shape (954, 9).
 

thanks! I tried it before with "False" as a string but of course this couldn't work. omgh. (no error message?)

Now I'm using:

>>>U, w, V = numpy.linalg.svd(yz_matrix_by, full_matrices=False)

>>>W.diag(w)
>>>W.shape
(9,9)
>>>V.shape
(9,9)

If I'm doing the check of the SVD results using:
>>>numpy.allclose(A, numpy.dot(U, numpy.dot(W, V)))
I get this error:

easier, allclose(A, dot(U*w, V) )

That's right!

"ValueError: matrices are not aligned"

Mismatched dimensions.

Yeah, this error is away. Now I get:

 >>>print(numpy.allclose(U_by*w_by, V_by)) 

False

I have to think about it if this makes sense ... (for me and prob. my my data)

Thanks and Greetings,

Daniel

[Charles R Harris]

Seems to be a missing "dot" in there.

Chuck

[Daniel Wagner]

The following works:

>>>numpy.allclose(A, numpy.dot(U, numpy.dot(W, V)))

True

But now I've a new problem problem:

When I'm using:

>>> A.shape

(954, 9)

>>> b.shape

(954, )

>>> temp = numpy.linalg.pinv(A, rcond=1.0000000000000001e-15)

>>> temp.shape

(9, 954)

to multiply this with my data b

>>>x = numpy.dot(temp, b)

ValueError: matrices are not aligned

I've missmatched dimensions again....

Greetings,

Daniel

[Charles R Harris]

Looks like you might want to look at lstsq.

Chuck

[Daniel Wagner]

right, I want to compute the least square..... Does lstsq() use svd for it's computations? Is it possible to take a look how it calculates the values?

Thanks.

Greetings,

Daniel

Chuck

[Daniel Wagner]

It works like a charm! The next time I really should use the already implemented functions and if I'm unsure how they work I still can take a look into their source code...

Thank you for the hint!

Greetings, 

Daniel

[josef...@gmail.com]

I'm just going the other way, using pinv and leastsq, and now switch
to decomposition directly.

>From what I can see the main difference in your example to what I
have is to use linalg.diagsvd

u,s,v = np.linalg.svd(x, full_matrices=1)
Sig = linalg.diagsvd(s,*x.shape)

>>> np.max(np.abs(np.dot(u, np.dot(Sig, v)) - x))
3.1086244689504383e-015

for the correct shapes of the sqrt of the x'x and xx':

>>> us = np.dot(u, Sig)
>>> np.max(np.abs(np.dot(us, us.T) - np.dot(x, x.T)))
1.0658141036401503e-014

>>> sv = np.dot(Sig, v)
>>> np.max(np.abs(np.dot(sv.T, sv) - np.dot(x.T, x)))
1.1368683772161603e-013

I'm still trying to collect these things into a class.

statsmodels is doing all the least square calculations with
linalg.pinv directly,

I thought your pinv example should work, you could also try with
explicit 2dimensional b

>>>x = numpy.dot(temp, b{:,None)

(scikits.learn Bayesian Ridge regression is also a good example how to
do regression with svd directly)

Josef

> Greetings,
> Daniel

[Daniel Wagner]

Thanks, this works:
>>>x2 = x = numpy.dot(temp, b[:,None])