rl-applet
W Bradley Knox
RandomAgent playing CartPole at
http://www.cs.utexas.edu/~bradknox/appletPlay/.
I'm ready to make a first attempt at handing it off. I've attached a
zip file that contains working jar and html files, an archived Eclipse
project with the source, the jar files that rl-applet uses, and a
README file that should be the starting point for anyone trying to use
this.
This applet should work with any Java-based RL-Glue agent and any
Java-based, rl-viz-compatible RL-Glue environment. I know those are
significant restrictions, but luckily most available RL-Library
environments already fit this restriction. And using sockets, which
would expand the number of programming languages available, is beyond
what I want to attempt in creating this applet.
Brian or someone else that maintains rl-library, could you test this
out? If it works for you, I'm happy to put it on my website and
announce its availability. If you want to put it in rl-library, that's
fine with me too.
I unfortunately only know how to compile this in Eclipse. The strategy
I found is to have Eclipse take all of the jars on the build path and
package them all as a single jar, which avoids server-side file access
restrictions that I'm still learning about. I'm sure this can be done
with a more general method, but I'm going to leave it as is for now.
Also, I'm still extending this to work with TAMER. Some of those
features shouldn't be in a general rl-applet package, but some should.
For example, I still need to figure out how to save log files to the
server. I'll update the code as those features are ironed out.
I hope this is useful to someone. It definitely will be to me. :)
Cheers,
Brad
On Thu, Sep 1, 2011 at 4:44 PM, Brian Tanner <br...@tannerpages.com> wrote:
> Yeah sorry I didn't reply. I couldn't see any deal breakers but also didn't think it looked easy so I figured I'd just lurk and see what happens.
>
> --
> Brian Tanner
> br...@fireplan.ca
> 204-663-9272 Extension 2
>
> Fire Plan Strategies: We provide fire safety training, planning, signs, and supplies across Canada!
>
> Our business grows by referrals. If you have any associates who may be interested in our services, we would appreciate your referral.
>
> WHEN THE FIRE TRUCKS ROLL, IT'S TOO LATE TO PLAN!
>
> On 2011-09-01, at 4:40 PM, W Bradley Knox wrote:
>
>> In case Brian or someone else is still planning on responding, I'm
>> finding success with a modified plan, so don't worry about any
>> response. I'll explain what I did and offer the code to any interested
>> parties once it's more of a finished product.
>>
>> - Brad
>>
>> On Tue, Aug 30, 2011 at 2:05 PM, W Bradley Knox <brad...@cs.utexas.edu> wrote:
>>> Hey, everyone.
>>>
>>> I'd like to follow up on Simos' email, since I'm doing something in
>>> the same vein. I'm trying to convert my RLGlue
>>> agent-environment-experiment combinations into a Java applet to run on
>>> a website. Here's the process as I imagine it, having only read a
>>> short tutorial on applets :
>>>
>>> 1. Get the agent, environment, and the RL-Viz experiment working as
>>> one program without sockets. (I've converted
>>> RunMinesSarsaExperimentNoSockets.java to do this successfully, except
>>> that the environment visualization window doesn't come up with the
>>> RLVizApp window. But I'm looking into that.)
>>> 2. Simplify RL-Viz to display in a single window. I'll probably make
>>> the load, start, etc. buttons and speed slider smaller and place them
>>> below the environment visualization in this single window display.
>>> 3. Just plug it in as an applet.
>>>
>>> I figure I should put this out there in case there's an unforeseen
>>> problem with it that's immediately apparent to someone else. I'll be
>>> happy to share the applet code if/when I get it to work, which could
>>> provide a nice feature to RL-Library. (How cool would it be to be able
>>> to show our agents learning on our websites?)
>>>
>>> Thanks!
>>> Brad Knox
>>> www.cs.utexas.edu/~bradknox
>>>
>>> On Thu, Jun 23, 2011 at 1:57 AM, Simos Gerasimou <s.ger...@gmail.com> wrote:
>>>> Hello guys,
>>>>
>>>> I'm trying to integrate the Super Mario environment with the JavaRLGlueCodec
>>>> in order to execute the game without sockets just like the Mines Sarsa
>>>> Example in the JavaRLGlueCodec.
>>>> As it is mentioned in the JavaRLGlueCodec manual, if the agent, the
>>>> environment and the experiment are in Java then the simulation can run
>>>> without any sockets.
>>>>
>>>> I tried to integrate it into a single project following the instructions
>>>> from the manual and the mines-sarsa example -
>>>> RunMinesSarsaExperimentNoSockets.java -
>>>> but It doesn't work.
>>>> This is what I did:
>>>> 1) kept from the agents package only ExmarioAgent and RandomAgent
>>>> 2) kept from the trainers the consoleTrainerJava (consoleTrainer.java and
>>>> consoleTrainerHelper.java) and guiTrainerJava packages (no java source code
>>>> inside)
>>>> 3) added all the jar files from system to the build path
>>>> 4) created a new package experiments and a class experimentFirst following
>>>> the example from the mines sarsa
>>>>
>>>> All the other information and data were removed.
>>>>
>>>> Unfortunately, it doesn't work.
>>>> The problem is when the consolerTrainer.main(args) is called and passes the
>>>> information to consoleTrainerHelper.
>>>>
>>>> Does anybody have and idea how to solve this issue or to integrate
>>>> everything together?
>>>>
>>>> Thanks in advance,
>>>> Simos
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "rl-library" group.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msg/rl-library/-/ZVsnQYT1WaUJ.
>>>> To post to this group, send email to rl-li...@googlegroups.com.
>>>> To unsubscribe from this group, send email to
>>>> rl-library+...@googlegroups.com.
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/rl-library?hl=en.
>>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups "rl-library" group.
>> To post to this group, send email to rl-li...@googlegroups.com.
>> To unsubscribe from this group, send email to rl-library+...@googlegroups.com.
>> For more options, visit this group at http://groups.google.com/group/rl-library?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups "rl-library" group.
> To post to this group, send email to rl-li...@googlegroups.com.
> To unsubscribe from this group, send email to rl-library+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/rl-library?hl=en.
>
>
Brian Tanner
In any case yes I'd love to have this in RL-Library. As for testing, I will try to get around to that at some point but I'm way behind here at work so if someone else could try it that would be a huge help.
This is really great Brad, very exciting stuff.
--
Brian Tanner
br...@fireplan.ca
204-663-9272 Extension 2
Fire Plan Strategies: We provide fire safety training, planning, signs, and supplies across Canada!
Our business grows by referrals. If you have any associates who may be interested in our services, we would appreciate your referral.
WHEN THE FIRE TRUCKS ROLL, IT'S TOO LATE TO PLAN!
> <rl-applet.zip>
W Bradley Knox
though), but some of those controls are already implemented. To use
them, click on the applet. Then you can speed up or slow down the time
steps by pressing '+' or '-'. You can pause ("stop" in rl-viz) with
'0', step with '1', and unpause ('start') with '2'. (Hmmm... I just
tried these and the step button is killing it. I'll try to fix that
after sending this.)
Restarting would be useful for me, so I'll probably add that in there
at some point. As for agent and environment loading, that wouldn't be
too hard, but I might leave it for someone else to do if they find it
valuable.
No rush on trying it out. I *think* setup should be quite simple
though. I wrote thorough directions in the README, and if nothing goes
wrong, it should be up and running for you within 30 minutes of
effort. I've pasted the README below to give you an idea of what you'd
be embarking on.
- Brad
README.txt:
**Before this will work**, changes to rl-viz needed:
1) In rlVizApp/src/org/rlcommunity/rlviz/app/VisualizerFactory.java,
make all createVisualizer() methods public.
2) In rlVizLibJava/src/rlVizLib/visualization/AbstractVisualizer.java, make
redrawCurrentImage() public and add
public BufferedImage getEnvImage() {
return this.bufferEnvImage;
}
****RLVizLib.jar and RLVizApp.jar with these changes are already
included in the lib directory.****
---------------------------------
To run:
On local machine, run "appletviewer rlapplet.html" from within the
workingExample directory.
On a website, run rlapplet.html with rl-applet.jar in the same directory.
[Key-based experiment controls]
With the applet window in focus (clicking on the applet itself seems necessary),
'+' increases time step frequency
'-' decreases time step frequency
'0' pauses the domain
'1' moves one time step forward
'2' unpauses the domain
---------------------------------
To compile:
Right now, I only know how to compile this via Eclipse. If someone can
put together more general instructions, that'd be helpful.
1) In Eclipse, go to File->Import. Choose "Existing Projects into
Workspace". Import rl-applet-eclipse_project.zip as an archive.
2) There should be a new project for rl-applet now. Right click on it
in the Package Explorer, choose "Properties", and click on "Java Build
Path"->"Libraries". You're going to add a bunch of external jars. I've
included all of the ones I use in the lib folder. Add these jars:
JavaRLGlueCodec.jar
RLVizApp.jar
RLVizLib.jar
commons-codec-1.3.jar
AgentShell.jar (maybe unnecessary and already covered by JavaRLGlueCodec.jar)
EnvironmentShell.jar (maybe unnecessary and already covered by
JavaRLGlueCodec.jar)
Also add any environmental and agent external jars you'll want to use.
These are included:
Acrobot.jar
CartPole.jar
MountainCar.jar
Tetris.jar
RandomAgent.jar
3) Make any changes you want.
4) Create a run configuration to run the applet from within Eclipse
(also necessary for the next step). Click "Run"->"Run Configurations".
Create a new Java Applet configuration using the bar on the left. Type
in the imported project's name and put
"edu.utexas.cs.tamerProject.applet.RLApplet" in the Applet field.
Click "Apply" and test the configuration with "Run". You should see
the RL-Applet window pop up.
5) Export the applet code as a jar. To do this, right-click on the
project name (e.g. "rl-applet") and choose "Export". Then choose
"Runnable Jar File", pick the launch configuration that runs the
applet from within Eclipse, choose an export destination, and use
"Extract required libraries…" for library handling (this option makes
some of the file access restrictions of applets easy to deal with).