Use of Parameterized actions

[Vishma Dias]

Hi,

I am using BURLAP to develop a finite state MDP to learn possible actions to take in order to traverse in a finite automata ( sort of a graph state sequence ). I need to use parameterized actions so that the learner will learn not only which actions to take, but also what value to given to the action parameter. The theoratical implementations are described in this paper [1].

Example in the paper : "consider a soccer playing robot which can kick, pass, or run. We can associate a continuous parameter vector to each of these actions: we can kick the ball to a given target position with a given force, pass to a specific position, and run with a given velocity."

Simply I want to define the action as (a,x) pair where a -> action & x-> parameter value.

Does BURLAP have this sort capabilities to cater parameterized actions and respective learning algorithms?

Thanks in Advance.

Best Regards,
Vishma.


[1] Masson, W., Ranchod, P. and Konidaris, G., 2016, February. Reinforcement Learning with Parameterized Actions. In AAAI (pp. 1934-1940).

[Jessica Rivera]

Hi Vishma,
Did you manage to implement this? I need to do something similar and couldn't find an example in the burlap code.

Regards,
Jessica

[Vishma Dias]

Hi Jessica,

Unfortunately no. I couldn't find any. Since I couldn't find any built in support I didn't use any parameterized actions. Instead I used set of actions which represents certain parameter groups.

Example : Instead of parameterized action "kick(v) " (v stands for velocity) , I used several actions as,

• kick with 5kmph
• kick with 10kmph and so on.

I hope you get the intent. Although this not a good workaround, I managed to get things done due to very limited action space on my application.

Best Regards,

Vishma.