Super Mario & JavaRLGlueCodec integration

[Simos Gerasimou]

Hello guys,

I'm trying to integrate the Super Mario environment with the JavaRLGlueCodec in order to execute the game without sockets just like the Mines Sarsa Example in the JavaRLGlueCodec.
As it is mentioned in the JavaRLGlueCodec manual, if the agent, the environment and the experiment are in Java then the simulation can run without any sockets.

I tried to integrate it into a single project following the instructions from the manual and the mines-sarsa example - RunMinesSarsaExperimentNoSockets.java -
but It doesn't work.
This is what I did:
1) kept from the agents package only ExmarioAgent and RandomAgent
2) kept from the trainers the consoleTrainerJava (consoleTrainer.java and consoleTrainerHelper.java) and guiTrainerJava packages (no java source code inside)
3) added all the jar files from system to the build path
4) created a new package experiments and a class experimentFirst following the example from the mines sarsa

All the other information and data were removed.

Unfortunately, it doesn't work.
The problem is when the consolerTrainer.main(args) is called and passes the information to consoleTrainerHelper.

Does anybody have and idea how to solve this issue or to integrate everything together?

Thanks in advance,
Simos

[Brian Tanner]

Hi Simos.  Mario is, to begin with, one of the more complicated RL-Glue environments in the way that it executes.  This is because it was first a game and then was adapted to work with RL-Glue.  So from that angle I'm not sure how easy it will be to do what you want.

Having said that, it should be possible.  I feel like it won't be too hard but it might require cutting out some of that competition stuff and trying to build something that is a bit simpler.  The competition code has all sorts of junk in there to make it easy to use for multiple agents/environments which may be complicating matters for you.

Unfortunately I'm super busy at my new job and can't really afford to take the couple hours it would require to dive into this on my own.

I would suggest trying to cut everything unrelated out and see what is happening.  When everything is in Java, a lot of the complicated-seeming code might disappear.  For example, you can just instantiate each of the agent and environment by creating objects instead of by passing all sorts of complicate parameter structures over sockets.

But alas it's been years since I helped write that code and those tutorials.

Maybe ask more questions and we'll see what we can do?  Sorry I'm not more helpful, I would like to be.

--

Brian Tanner

br...@fireplan.ca

204-663-9272 Extension 2

Fire Plan Strategies:  We provide fire safety training, planning, signs, and supplies across Canada!

Our business grows by referrals. If you have any associates who may be interested in our services, we would appreciate your referral.

WHEN THE FIRE TRUCKS ROLL, IT'S TOO LATE TO PLAN!

--
You received this message because you are subscribed to the Google Groups "rl-library" group.
To view this discussion on the web visit https://groups.google.com/d/msg/rl-library/-/ZVsnQYT1WaUJ.
To post to this group, send email to rl-li...@googlegroups.com.
To unsubscribe from this group, send email to rl-library+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rl-library?hl=en.

[Simos Gerasimou]

Hello Brian,

I know that your workload is very high at the moment and I understand that you are very busy with your new job.
That's why I didn't requested any further help from you, but I appealed to the other RL library members.
However, I appreciate that you try to help me as much as you can. Thanks.

Anyway, I tried to do all the stuff that you mentioned, 10 days ago, removing all the unnecessary classes and following the example of Sarsa Mine java codec.
In the end, I should have missed something, because the Environment(Mario) was trying to connect to the RLGlue core through sockets; though it was unnecessary because all the classes where implemented in Java.

I struggled a lot and then decided to move on and use a newer Super Mario environment, that of MarioAI.
I must say that it is richer and more powerfull than the RL Mario Environment.

Now I'm intending to use the EpsilonGreedyTileCodingSarsaLamba package you wrote.
That's why I am asking at another post whether anyone has tried this package and could verify if its working properly (because as you said it should work but there might be some bugs :)



Simos

[Brian Tanner]

That's my fault Simos. I had both of your messages in my inbox and I got my head screwed on backwards and responded to the older e-mail thinking it was the newer one. Sorry about that.

--

Brian Tanner

br...@fireplan.ca

204-663-9272 Extension 2

Fire Plan Strategies:  We provide fire safety training, planning, signs, and supplies across Canada!

Our business grows by referrals. If you have any associates who may be interested in our services, we would appreciate your referral.

WHEN THE FIRE TRUCKS ROLL, IT'S TOO LATE TO PLAN!

--
You received this message because you are subscribed to the Google Groups "rl-library" group.

To view this discussion on the web visit https://groups.google.com/d/msg/rl-library/-/C8ONP6nKhyUJ.

[W Bradley Knox]

Hey, everyone.

I'd like to follow up on Simos' email, since I'm doing something in
the same vein. I'm trying to convert my RLGlue
agent-environment-experiment combinations into a Java applet to run on
a website. Here's the process as I imagine it, having only read a
short tutorial on applets :

1. Get the agent, environment, and the RL-Viz experiment working as
one program without sockets. (I've converted
RunMinesSarsaExperimentNoSockets.java to do this successfully, except
that the environment visualization window doesn't come up with the
RLVizApp window. But I'm looking into that.)
2. Simplify RL-Viz to display in a single window. I'll probably make
the load, start, etc. buttons and speed slider smaller and place them
below the environment visualization in this single window display.
3. Just plug it in as an applet.

I figure I should put this out there in case there's an unforeseen
problem with it that's immediately apparent to someone else. I'll be
happy to share the applet code if/when I get it to work, which could
provide a nice feature to RL-Library. (How cool would it be to be able
to show our agents learning on our websites?)

Thanks!
Brad Knox
www.cs.utexas.edu/~bradknox

> --
> You received this message because you are subscribed to the Google Groups
> "rl-library" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/rl-library/-/ZVsnQYT1WaUJ.

[W Bradley Knox]

In case Brian or someone else is still planning on responding, I'm
finding success with a modified plan, so don't worry about any
response. I'll explain what I did and offer the code to any interested
parties once it's more of a finished product.

- Brad

[Brian Tanner]

Yeah sorry I didn't reply. I couldn't see any deal breakers but also didn't think it looked easy so I figured I'd just lurk and see what happens.

--
Brian Tanner
br...@fireplan.ca
204-663-9272 Extension 2

Fire Plan Strategies: We provide fire safety training, planning, signs, and supplies across Canada!

Our business grows by referrals. If you have any associates who may be interested in our services, we would appreciate your referral.

WHEN THE FIRE TRUCKS ROLL, IT'S TOO LATE TO PLAN!