Optimal control - LQR problem
AsimV
I'm reading an introductionary article about LQR controller design. In
the article is stated that goal of LQR is to find such control signal
- u that will minimize the following performance criteria:
J = inetgral ( y'Qy + u'Ru)dt, where ' means trasnpose operation.
Optimal LQR control minimize both energies terms (one with Q and other
with R).
I have found this statement:
"LQR is a trade-off because decreasing the energy of the output will
require a large control signal and a small control signal will lead to
large outputs. The role of the weighting matrices Q and R is to
establish a trade-off between these conflicting goals".
I'm not really sure, how to interpret this. Can you explain in more
details, why small control signal will lead to large outputs? This is
not obvious to me. Any help is highy appreciated.
Best regards
Bruce Varley
"AsimV" <asim.vod...@gmail.com> wrote in message
news:dc7574da-95d2-4c3d...@b5g2000pri.googlegroups.com...
Read it as a 'weak' control signal. If your control action is weak, you
won't be able to contain variations in the (plant) output well, which means
large output in this context. Strong control can maintain steady conditions
better, but at the expense of increased control effort. In many practical
applications this is a fairly minor consideration, other aspects like
stability are more important. For situations like controlling spacecraft
orientation with limited fuel supplies, it's a key factor.
AsimV
thank you for your reply. I see now that author was refering to
control effort and not amplitudes of signals.
I wonder when it is appropriate to consider LQR comparing to classical
PID control (cascade, or feedforward)?
pnachtwey
> Hello,
>
> I have found this statement:
> "LQR is a trade-off because decreasing the energy of the output will
> require a large control signal and a small control signal will lead to
> large outputs. The role of the weighting matrices Q and R is to
> establish a trade-off between these conflicting goals".
If R is is 0 then there is no cost function for control output. The
control output will do what ever is required to reduce the error is
quickly as possible. Should this happen the response will normally be
very violent because the control output u(t) will go into saturation
to drive the error to 0. This violent reaction is often not
desired.
There must be a cost function for the control output, u'(t)*R*u(t) to
keep the control output from going into saturation trying to minimize
just the error in the state. This is effectively a tuning parameter
because few applications really have a true cost function
u'(t)*R*u(t). In this case R is modified until the desired response
is obtained.
>
> I'm not really sure, how to interpret this. Can you explain in more
> details, why small control signal will lead to large outputs? This is
> not obvious to me.
??? If R is 0 then the optimizing routine will calculate huge gains to
minimize the error. This in turn will result in huge control outputs
that will minimize the error. Some times the best way to learn is to
write a small program that simulates a plant then then minimizes the
cost function by trying different PID gains. Do several runs changing
R after each run. I will see if I have an example some place.
Peter Nachtwey
Bruce Varley
I'm a bit narrowly focussed to answer that question. In my business
(industrial control) processes are generally not well defined, and the cost
of control effort is normally low. PID, sometimes incorporating additional
functions as you mention, is the norm for regulatory loops. Dynamic
multivariable predictive algorithms are also used.
Tim Wescott
Certainly it's fair to answer the question "which of these should I
spend my time implementing".
If you model your system right you can force your LQR to _be_ a PID
controller; in one sense you can think of the process of getting the LQR
design to be just another way of choosing the coefficients for a controller.
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com
Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html
pnachtwey
LQR is mainly just another way to select PID gains. LQR has a couple
problems. First LQR is a regular, it only tries to drive the PV to 0.
Second, as Bruce mentioned, the cost function in most applications is
low and not a concern. I have had only one person ask me about LQR in
a motion control product and he was just trying to impress me because
following the motion profile was most important.
Tim is right about the LQR controller can be a PID or a I-PVA as in my
example below. The LQR work sheet started out as a minimun ITAE work
sheet. The big difference is the cost function and the how PID gains
are generated. The controller with the LQR gains can be used with
feed forwards. LQR just provides yet another way to calculate the
controller gains.
Here is an example
ftp://ftp.deltacompsys.com/public/NG/Mathcad%20-%20LQR.pdf
Notice that the cost function takes into account not only errors but
error rates and the second derivative. If Q only takes the error into
account the underdamped system would oscillate as it approaches the
SP. The Q array works well as it is. The vale of R can be made
bigger to make the response slower or made smaller to make the
response faster but with a chance of an osciallating response. I can
make R=0 so there is not cost for the control effort and the control
output will go in to saturation in a effort to reduce the error as
fast as possible.
There are more efficient ways to calculate the gains. I may update
the work sheet to reflect a more efficient way of calculating the
gains.
Peter Nachtwey
pnachtwey
doesn't make any difference but when solving for the LQR gains it
really is simpler just to use x'*q*x and assume the target is
[0;0;0]. I modified my LQR worksheet to reflect that and added a pole
plot at the bottom.
ftp://ftp.deltacompsys.com/public/NG/Mathcad%20-%20LQR.pdf
AsimV also notified me about how long it takes to compute the LQR
gains the way I did. I admit it. Using a optimizer is slow. I
looked to see how my Digital COntrol System Analysis and Design book
by Phillips and Nagle did it. They use a iterative technique that is
similar to computing steady state Kalman gains.
ftp://ftp.deltacompsys.com/public/NG/Mathcad%20-%20LQR%2010pt2.pdf
This method in the book works but I had to make the Q and A array 4x4
so the method in the book will compute a integrator gain. Since the
Q array is not the same in the two .pdf files the results are not the
same. Also, the method in the book seems to assume that R is a
scaler instead of a matrix. I can see where it might be important to
have a more general solution that the optimizer provides so that R can
be a matrix that will not only penalize control output but the rate of
change in control output. This may be necessary if the feed back
resolution is coarse.
The big advantage that the method in the book has is that is computes
the gains MUCH faster. The optimizer technique takes minutes to
compute the LQR gains but the method in the book takes a fraction of a
second.
Also notice that I didn't really need a cost function for the control
output when using the method in the book. It seems that having a
cost for the derivatives keeps the control output tame enough.
Any comments?
Peter Nachtwey
AsimV
failed because I coudn't find a way to slove ARE (Algebraic Ricatti
equation). Your formulas (in yur book) are very similar but there
something I failed to derive...
For example Optimal gain vector can be found in this way : K =
R^(-1)*B'*P and Algebraic Ricatti equation is: A'P+PA-PBK+Q = 0 => A'*P
+P*(A-B*K)+Q = 0.
I wonder what numerical technique is used to derive formulas you used,
because they guaranteed to return optimal solution.Most theory
atricles and materials I have found on the Internet used formulas I
posted above.
Is there more theory in your book, maybe some derivation?
I tried last night to solve this in analytically but had no success.
Now, when I have these relationships, I 'll try to fit them to
theoretical description I posted on plctalk forum.
K = inv( B' * P * B + R)*B' * P * A;
P = A' *P * (A-B*K)+Q;
>Also notice that I didn't really need a cost function for the control
>output when using the method in the book. It seems that having a
>cost for the derivatives keeps the control output tame enough.
Cost function is simply incorporated in ARE, because ARE is derived
from cost function. This way, solution to ARE is matrix P, and with P,
one can calculate K. It seems that authors used very smart way to
solve ARE that is used in the book. I must admit that most materials I
have found are practically useless, they are really full of high level
theory, while physical meaning is hidden.
Best regards,
Asim Vodencarevic
pnachtwey
> Peter, that was exactly what I was trying to achieve last night, but I
> failed because I coudn't find a way to slove ARE (Algebraic Ricatti
> equation). Your formulas (in yur book) are very similar but there
> something I failed to derive...
> For example Optimal gain vector can be found in this way : K =
> R^(-1)*B'*P and Algebraic Ricatti equation is: A'P+PA-PBK+Q = 0 => A'*P
> +P*(A-B*K)+Q = 0.
> I wonder what numerical technique is used to derive formulas you used,
> because they guaranteed to return optimal solution.
The authors of the book I referenced in the pdf didn't use the Ricatti
equation. They derived it by using the cost function and making
substitutions. The main idea is the current value of the cost
function should be 0 if the state is 0 or at least minimal. Then the
calculations work backwards to find the point where the change in the
cost function from time n to time n-1 doesn't increase. At that point
the LQR gains are stable.
>Most theory
> atricles and materials I have found on the Internet used formulas I
> posted above
I wouldn't know. I have only this one book that covers the LQR topic
and I didn't even bother to look at it seriously until you asked
because I am not a big believer in LQR control. I did remember the
Matlab example is the book looked simple but I was never interested in
regulators more minimizing control effort. I want to follow motion
profiles exactly. Anything was a diversion.
> Is there more theory in your book, maybe some derivation?
Yes, 9 pages of explanation and about 39 formulas
> I tried last night to solve this in analytically but had no success.
> Now, when I have these relationships, I 'll try to fit them to
> theoretical description I posted on plctalk forum.
> K = inv( B' * P * B + R)*B' * P * A;
> P = A' *P * (A-B*K)+Q;
>
> >Also notice that I didn't really need a cost function for the control
> >output when using the method in the book. It seems that having a
> >cost for the derivatives keeps the control output tame enough.
>
> Cost function is simply incorporated in ARE, because ARE is derived
> from cost function. This way, solution to ARE is matrix P, and with P,
> one can calculate K. It seems that authors used very smart way to
> solve ARE that is used in the book. I must admit that most materials I
> have found are practically useless, they are really full of high level
> theory, while physical meaning is hidden.
>
> Best regards,
> Asim Vodencarevic
I have felt your pain. I have had to read a few different books on
the same topic over and over and compare formulas and the way things
are said. Sometimes one author would do a much better explanation
than another. I have seen cases where two different books have
different equations for calculating the same thing when I worked
through the math I could verify one book is right and the other
wrong. The difference was more than just two different ways to
calculate the same thing. In other words, you can't really trust the
text books all the time.
Peter Nachtwey
AsimV
performance index is defined as integral(x' * Q * x + u' * R * u)dt +
x' * Qf * x, where matrix Qf represents so called terminal or final
state cost. This is really weird. What form your book is using?
Matlab has bulit in function to solve LQR problem. It is interesting
that it uses following performance index: integral(x' * Q * x + u' * R
* u +2*x' * N *u)dt.
I don't see any real practical use of this and it complicates
calculations. I'm pretty much sure that cost of final state can be
incorporated in x' * Q * x term, and solvoing simpler form:
integral(x' * Q * x + u' * R * u)dt will do the trick. I'm sure there
are people here who design and use LQR controller in practice. I'd
like to hear their opinions.
Thank you