Multi input Weiner Filter

[HardySpicer]

For the standard SISO Wiener filter we minimize the cost J

J=E[e^2]=E(d-W'X)^2

where W is a vector of weights and X is a vector of regressers. (d is
desired output) Also ' denotes transpose.
We do this by diferentiating wrt the weight vector W and arrive at the
standard Wiener solution.

However, in the case where W is asymmetric Matrix and d is a vector
(also X is a vector still) we have to differentiate wrt a Matrix and
the error is a vector.

J=e'e = (d-W'X)'(d-W'X)

ie dJ/dW (J is still a scalar)

I have a paper that just says that the answer is the same form but
with no derivation! Differentiating wrt a matrix however is slightly
different.

------------------------------------------------------
For example...
Differentiation of a scalar wrt a vector of the quadratic form

y=x'Ax where a is a matrix and x a vector

dy/dx = Ax+A'x = 2Ax if A is symmetric.

Now in the multidimensional Wiener filter we have a term
X'WW'X which needs differentiating wrt the matrix W.

We also have terms X'Wd and dW'X which need differentiating wrt the
matrix W.

H.

[HardySpicer]

Actually I just found this result from

www.che.iitm.ac.in/~naras/ch544/matrix.pdf

That if you differentiate the norm squared

||AX+b||^2

you get
2AXX' + 2bX'

using this and substituting

||-WX+d||^2

I get

-2WXX' +2dX'=0

Now since E[XX'] = R, the correlation matrix I get

WR=E[dX'] (1)

or

W=RdxR^-1

which is not the same result as in the paper. However, R is symmetric,
from (1)

W'R=E[Xd']

W'=R^-1 Rxd (Rxd is the cross-correlation matrix between X and d)

which is more like it. Now does W'=W ie is the filter weight matrix
symmetric??

H.

[HardySpicer]

Sorry - I should have said

That if you differentiate the norm squared

||AX+b||^2

with respect to A.

[HardySpicer]

On Jan 9, 10:36 am, HardySpicer <gyansor...@gmail.com> wrote:

oops it's right. I Had minimized ||-WX+d ||^2 when it should have been
||-W'X+d ||^2. all ok!

[Tim Wescott]

This is a solved problem if you bear in mind that a steady state Kalman
filter is really just a Wiener filter with a fancy name.

Then web search accordingly.

--
Tim Wescott
Control systems and communications consulting
http://www.wescottdesign.com

Need to learn how to apply control theory in your embedded system?
"Applied Control Theory for Embedded Systems" by Tim Wescott
Elsevier/Newnes, http://www.wescottdesign.com/actfes/actfes.html

[HardySpicer]

> Control systems and communications consultinghttp://www.wescottdesign.com

>
> Need to learn how to apply control theory in your embedded system?
> "Applied Control Theory for Embedded Systems" by Tim Wescott
> Elsevier/Newnes,http://www.wescottdesign.com/actfes/actfes.html

Trouble with Kalman filters is that you need to solve a Ricatti
equation for the gain matrix.
With Polynomial or LMS type adaptive Wiener filters, the computation
is far less.

H.