Article: Risk-Sensitive Reinforcement Learning Applied to Control under Constraints

Skip to first unread message

Jul 19, 2005, 10:05:24 PM7/19/05
JAIR is pleased to announce the publication of the following article:

Geibel, P. and Wysotzki, F. (2005)
"Risk-Sensitive Reinforcement Learning Applied to Control under Constraints",
Volume 24, pages 81-108.

For quick access via your WWW browser, use this URL:

In this paper, we consider Markov Decision Processes (MDPs) with error
states. Error states are those states entering which is undesirable
or dangerous. We define the risk with respect to a policy as the
probability of entering such a state when the policy is pursued. We
consider the problem of finding good policies whose risk is smaller
than some user-specified threshold, and formalize it as a constrained
MDP with two criteria. The first criterion corresponds to the value
function originally given. We will show that the risk can be
formulated as a second criterion function based on a cumulative
return, whose definition is independent of the original value
function. We present a model free, heuristic reinforcement learning
algorithm that aims at finding good deterministic policies. It is
based on weighting the original value function and the risk. The
weight parameter is adapted in order to find a feasible solution for
the constrained problem that has a good performance with respect to
the value function. The algorithm was successfully applied to the
control of a feed tank with stochastic inflows that lies upstream of a
distillation column. This control task was originally formulated as an
optimal control problem with chance constraints, and it was solved
under certain assumptions on the model to obtain an optimal solution.
The power of our learning algorithm is that it can be used even when
some of these restrictive assumptions are relaxed.

The article is available via:

-- (also see

-- World Wide Web: The URL for our World Wide Web server is
For direct access to this article and related files try:

-- Anonymous FTP from Carnegie-Mellon University (USA):
The compressed PostScript file is named

For more information about JAIR, visit our WWW or FTP sites, or

Steven Minton
JAIR Managing Editor

Reply all
Reply to author
0 new messages