RL/MDP optimization package
312 views
Skip to first unread message
Alex Susu
Jul 11, 2012, 6:16:22 PM7/11/12
to rl-...@googlegroups.com
Hello.
Could you please tell me what projects implement RL algorithms and MDP optimization.
Myself I am aware of the following projects:
- PyBrain - wrote in Python
- Sarsop (point-based POMDP solver) at
http://bigbird.comp.nus.edu.sg/pmwiki/farm/appl/index.php?n=Main.Download - C/C++
- http://library.rl-community.org (wrote mostly in Java)
- Symbolic Perseus (wrote in Java and Matlab)
Do you know if there is any serious software accompanying some of these great books
in RL (the classic Sutton Barto or Powell's Approximate Dynamic Programming, etc)?
The reason I ask is that I developed a tool for MDP optimization, which uses MDP
models exported from PRISM. The package also can use simple state aggregation to cope with
huge state spaces. The tool can be found here
http://sites.google.com/site/alexsusu/home/mdp_opt .
I am currently better documenting the project, migrating part of the code to Python
and implementing some symbolic MDP algorithms, like value iteration for single-cost MDPs.
Basically I used the tool for a couple of papers, but I am looking to make it more
generic.
If you have the time to look at it please let me know if you have questions and don't
hesitate to provide feedback.
With best regards.
Alex
Could you please tell me what projects implement RL algorithms and MDP optimization.
Myself I am aware of the following projects:
- PyBrain - wrote in Python
- Sarsop (point-based POMDP solver) at
http://bigbird.comp.nus.edu.sg/pmwiki/farm/appl/index.php?n=Main.Download - C/C++
- http://library.rl-community.org (wrote mostly in Java)
- Symbolic Perseus (wrote in Java and Matlab)
Do you know if there is any serious software accompanying some of these great books
in RL (the classic Sutton Barto or Powell's Approximate Dynamic Programming, etc)?
The reason I ask is that I developed a tool for MDP optimization, which uses MDP
models exported from PRISM. The package also can use simple state aggregation to cope with
huge state spaces. The tool can be found here
http://sites.google.com/site/alexsusu/home/mdp_opt .
I am currently better documenting the project, migrating part of the code to Python
and implementing some symbolic MDP algorithms, like value iteration for single-cost MDPs.
Basically I used the tool for a couple of papers, but I am looking to make it more
generic.
If you have the time to look at it please let me know if you have questions and don't
hesitate to provide feedback.
With best regards.
Alex
Lucian Busoniu
Jul 12, 2012, 8:41:26 AM7/12/12
to rl-...@googlegroups.com
Hello,
You may want to have a look at our Matlab toolbox here http://busoniu.net/files/repository/readme_approxrl.html, which is focused on function approximation and implements (in addition to others) algorithms in the book "Reinforcement Learning and Dynamic Programming Using Function Approximators", with Babuska, De Schutter, and Ernst, Taylor & Francis, 2010.
The version on the website is a bit outdated, if helpful I can send the latest version which has added e.g. some online receding-horizon planning methods among other things, but it may be more sparsely commented...
Sincerely,
Lucian
"José Antonio Martín H."
Jul 13, 2012, 8:18:46 AM7/13/12
to rl-...@googlegroups.com
Hi.
You can also look here:
* http://www.dacya.ucm.es/Jam/download.htm
I have some python and Matlab demos.
There is not yet an official toolbox but the software has a very
homogeneous structure and can be used as a kind of toolbox as well.
Best,
Jose.
El 12/07/2012 14:41, Lucian Busoniu escribi�:
> --
> You received this message because you are subscribed to the
> "Reinforcement Learning Mailing List" group.
> To post to this group, send email to rl-...@googlegroups.com
> To unsubscribe from this group, send email to
> rl-list-u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/rl-list?hl=en
--
/ .- .-.. .-.. / -.-- --- ..- / -. . . -.. / .. ... / .-.. --- ...- .
Jos� Antonio Mart�n H. (PhD) E-Mail: jama...@fdi.ucm.es
Computer Science Faculty Phone: (+34) 91 3947650
Complutense University of Madrid Fax: (+34) 91 3947527
C/ Prof. Jos� Garc�a Santesmases,s/n 28040 Madrid, Spain
web: http://www.dacya.ucm.es/jam/
LinkedIn: http://www.linkedin.com/in/jamartinh (Let's connect)
.-.. --- ...- . / .. ... / .- .-.. .-.. / .-- . / -. . . -..
You can also look here:
* http://www.dacya.ucm.es/Jam/download.htm
I have some python and Matlab demos.
There is not yet an official toolbox but the software has a very
homogeneous structure and can be used as a kind of toolbox as well.
Best,
Jose.
El 12/07/2012 14:41, Lucian Busoniu escribi�:
> You received this message because you are subscribed to the
> "Reinforcement Learning Mailing List" group.
> To post to this group, send email to rl-...@googlegroups.com
> To unsubscribe from this group, send email to
> rl-list-u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/rl-list?hl=en
--
/ .- .-.. .-.. / -.-- --- ..- / -. . . -.. / .. ... / .-.. --- ...- .
Jos� Antonio Mart�n H. (PhD) E-Mail: jama...@fdi.ucm.es
Computer Science Faculty Phone: (+34) 91 3947650
Complutense University of Madrid Fax: (+34) 91 3947527
C/ Prof. Jos� Garc�a Santesmases,s/n 28040 Madrid, Spain
web: http://www.dacya.ucm.es/jam/
LinkedIn: http://www.linkedin.com/in/jamartinh (Let's connect)
.-.. --- ...- . / .. ... / .- .-.. .-.. / .-- . / -. . . -..
Yohannes Kassahun
Jul 13, 2012, 8:26:59 AM7/13/12
to rl-...@googlegroups.com, Jan Hendrik Metzen
Hello,
Please have a look at MMLF http://sourceforge.net/projects/mmlf/. The project implements in addition to RL algorithms evolutionary methods.
Regards,
Yohannes.
--
Dr.-Ing. Yohannes Kassahun
Cognitive Robotics
Universität Bremen
FB 3 - Mathematik und Informatik
AG Robotik
Robert-Hooke-Straße 5
28359 Bremen, Germany
Besuchsadresse im Gebäude Unicom 1:
Mary-Somerville-Str. 9
28359 Bremen, Germany
Phone: +49 (0)421 178 45-4114
Fax: +49 (0)421 178 45-4150
E-Mail: kass...@informatik.uni-bremen.de
Weitere Informationen: http://www.informatik.uni-bremen.de/robotik
Please have a look at MMLF http://sourceforge.net/projects/mmlf/. The project implements in addition to RL algorithms evolutionary methods.
Regards,
Yohannes.
2012/7/12 Alex Susu <alex...@gmail.com>
Alex
--
You received this message because you are subscribed to the "Reinforcement Learning Mailing List" group.
To post to this group, send email to rl-...@googlegroups.com
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/rl-list?hl=en
--
Dr.-Ing. Yohannes Kassahun
Cognitive Robotics
Universität Bremen
FB 3 - Mathematik und Informatik
AG Robotik
Robert-Hooke-Straße 5
28359 Bremen, Germany
Besuchsadresse im Gebäude Unicom 1:
Mary-Somerville-Str. 9
28359 Bremen, Germany
Phone: +49 (0)421 178 45-4114
Fax: +49 (0)421 178 45-4150
E-Mail: kass...@informatik.uni-bremen.de
Weitere Informationen: http://www.informatik.uni-bremen.de/robotik
Matthijs Spaan
Jul 16, 2012, 10:23:34 AM7/16/12
to rl-...@googlegroups.com
Hello,
On 12-07-2012 00:16, Alex Susu wrote:
> Hello.
> Could you please tell me what projects implement RL algorithms and
> MDP optimization.
You might also be interested in the MultiAgentDecisionProcess C++
toolbox, which besides multiagent models also covers some MDP and POMDP
methods (mainly model-based).
The latest released version is at
http://staff.science.uva.nl/~faolieho/index.php?fuseaction=software.madp
and unfortunately it is a bit outdated, drop me a line in case you're
interested in a newer version.
Regarding POMDP software, at 2010 workshop we compiled a list of
available software packages, check page 7 of
http://users.isr.ist.utl.pt/~mtjspaan/POMDPPractitioners/ICAPSws_discussion.pdf
.
Cheers,
Matthijs
On 12-07-2012 00:16, Alex Susu wrote:
> Hello.
> Could you please tell me what projects implement RL algorithms and
> MDP optimization.
toolbox, which besides multiagent models also covers some MDP and POMDP
methods (mainly model-based).
The latest released version is at
http://staff.science.uva.nl/~faolieho/index.php?fuseaction=software.madp
and unfortunately it is a bit outdated, drop me a line in case you're
interested in a newer version.
Regarding POMDP software, at 2010 workshop we compiled a list of
available software packages, check page 7 of
http://users.isr.ist.utl.pt/~mtjspaan/POMDPPractitioners/ICAPSws_discussion.pdf
.
Cheers,
Matthijs
Todd Hester
Jul 16, 2012, 10:47:54 AM7/16/12
to rl-...@googlegroups.com
Hi,
I have also released a set of RL algorithms as a ROS package:
http://www.ros.org/wiki/reinforcement_learning
Thanks,
Todd
I have also released a set of RL algorithms as a ROS package:
http://www.ros.org/wiki/reinforcement_learning
Thanks,
Todd
Alex Susu
Sep 13, 2012, 4:49:52 PM9/13/12
to rl-...@googlegroups.com
Hello.
Thank you all for the various replies regarding the existing tools. I compiled these
answers and what I also found in a page here:
http://sites.google.com/site/alexsusu/rl_and_mdp_pointers .
In case you know any other website presenting a list of such tools, or tool, etc
please let me know.
Best regards.
Alex
On 7/12/2012 11:24 AM, Thomas Degris wrote:
> Hello Alex,
>
> RLPark (http://rlpark.github.com/) is another reinforcement learning package in Java.
>
> Thomas
Thank you all for the various replies regarding the existing tools. I compiled these
answers and what I also found in a page here:
http://sites.google.com/site/alexsusu/rl_and_mdp_pointers .
In case you know any other website presenting a list of such tools, or tool, etc
please let me know.
Best regards.
Alex
On 7/12/2012 11:24 AM, Thomas Degris wrote:
> Hello Alex,
>
> RLPark (http://rlpark.github.com/) is another reinforcement learning package in Java.
>
> Thomas
>> --
>> You received this message because you are subscribed to the "Reinforcement Learning Mailing List" group.
>> To post to this group, send email to rl-...@googlegroups.com
>> To unsubscribe from this group, send email to
>> rl-list-u...@googlegroups.com
>> You received this message because you are subscribed to the "Reinforcement Learning Mailing List" group.
>> To post to this group, send email to rl-...@googlegroups.com
>> To unsubscribe from this group, send email to
Saminda Abeyruwan
Sep 13, 2012, 9:39:24 PM9/13/12
to rl-...@googlegroups.com
Hi,
RLLib (http://web.cs.miami.edu/home/saminda/rllib.html). C++ porting of RLPark.
Thank you!
Sam
For more options, visit this group at
http://groups.google.com/group/rl-list?hl=en
--
You received this message because you are subscribed to the "Reinforcement Learning Mailing List" group.
To post to this group, send email to rl-...@googlegroups.com
To unsubscribe from this group, send email to
Thorsten Kiefer
Sep 14, 2012, 1:04:11 PM9/14/12
to rl-...@googlegroups.com
maybe my project is interesting for you :
http://sourceforge.net/projects/polebalancing2d/?source=directory
Best Regards
Thorsten
Tim Kovacs
Sep 21, 2012, 2:09:33 PM9/21/12
to rl-...@googlegroups.com
Hi Alex,
I coauthored a survey of RL software you may find interesting. It was done in October 2010.
Tim Kovacs and Robert Egginton, On the analysis and design of software for reinforcement learning, with a survey of existing systems. Machine Learning, 84(1--2), pp. 7-49. February 2011.
http://www.springerlink.com/content/101122m157423142/?MUD=MP
Best wishes,
Tim
On Wednesday, July 11, 2012 11:16:22 PM UTC+1, AlexSusu wrote:
I coauthored a survey of RL software you may find interesting. It was done in October 2010.
Tim Kovacs and Robert Egginton, On the analysis and design of software for reinforcement learning, with a survey of existing systems. Machine Learning, 84(1--2), pp. 7-49. February 2011.
http://www.springerlink.com/content/101122m157423142/?MUD=MP
Best wishes,
Tim
On Wednesday, July 11, 2012 11:16:22 PM UTC+1, AlexSusu wrote:
Hello.
Could you please tell me what projects implement RL algorithms and MDP optimization.
...
With best regards.
Alex
Reply all
Reply to author
Forward
0 new messages