Python repository for Sutton and Barto Book Codes akin to the one on Prof Sutton's page?

[Siddharth Kotwal]

Hello everyone!

I've been dabbling with RL for the past months and RL is a very delightful subject. The book by Sutton & Barto has especially been very helpful in grasping some obtruse concepts. However since I haven't taken a formal course on RL, I'm finding it a little difficult to implement traditional examples. Especially because unlike traditional ML where you can make powerful and plot errors out of a library like sklearn, RL doesn't seem to have a standard repository of examples in Python. 

I'm working on a continuous domain problem and tabular learning seems to have failed miserably, therefore I am now using Tile Coding in Python to approximate the value function. Since I don't know Lisp too well, I'm currently going through the C implementation on Prof Sutton's page here http://incompleteideas.net/sutton/book/code/code.html and was wondering if anyone in the academia or otherwise would pass on some advice to me regarding working Pythonic implementations. 

Mainly, what would be a good investment in time (languages or related material) right now to pursue it when I move to gradschool? How is it being implemented in the industry?  Say any intuitive explanation of how Nest might be using RL to control HVACs and optimize energy consumption? 

Thanks! 

[Peter Stone]

Siddarth,

Regarding controlling HVACs to control energy consumption, you may be interested in this paper:

   http://www.cs.utexas.edu/~pstone/Papers/bib2html/b2hd-AAMAS13-urieli.html

Best,
     Peter

[Pierre-Luc Bacon]

Some of the continuous control environments presented in the book are now part of Gym: https://gym.openai.com/envs

I would suggest starting with this, and implementing the algorithms yourself (they tend to be very concise). There is also a Python wrapper for the tile coding software: http://incompleteideas.net/rlai.cs.ualberta.ca/RLAI/RLtoolkit/tilecoding.html

Best, 

Pierre-Luc

--
You received this message because you are subscribed to the "Reinforcement Learning Mailing List" group.
To post to this group, send email to rl-...@googlegroups.com
To unsubscribe from this group, send email to
rl-list-u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rl-list?hl=en
---
You received this message because you are subscribed to the Google Groups "Reinforcement Learning Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rl-list+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[José Antonio Martín H.]

BTW, should we all be moving to gym? 

There is still RLGlue, RLPy...

--

--
/ .- .-.. .-.. / -.-- --- ..- / -. . . -.. / .. ... / .-.. --- ...- .
José Antonio Martín H. (PhD)           E-Mail: jama...@fdi.ucm.es  
Computer Science Faculty               Phone: (+34) 91 3947650        
Complutense University of Madrid       Fax: (+34) 91 3947527          
C/ Prof. José García Santesmases,s/n   28040 Madrid, Spain            
web: http://www.dacya.ucm.es/jam/          
LinkedIn: http://www.linkedin.com/in/jamartinh (Let's connect)        
.-.. --- ...- . / .. ... / .- .-.. .-.. / .-- . / -. . . -..

[Siddharth Kotwal]

Thanks Pierre! Just finished solving moutain car using tile coding. 

Hopefully more algorithms will be added to gym, like neural fitted q iteration.