Jsoar RL and reset

[Angel Gavalda Pina]

Hello!! I´am workin in a jSoarRL bot.

In Soar the Reinforcement learning works with repeated executions, in every execution the agent run until it halt and when this occur you must init-soar and run the agent again.
How i can run init-soar in jsoar? Agent.initialize() sirves for this propose of Reinforcement Learning?

When i try to use Q-learning policy, with adding the sentence "rl --set learning-policy q-learning" in the soar file that i load.
I get this error: org.jsoar.kernel.SoarException: Don't know how to set RL parameter 'rl.learning-policy' to value 'q-learning'.
Q-learning works with jsoar?

Thanks and greetings to all!!!
Angel.

[Dave Ray]

Angel,

Hi. Yes, Agent.initialize() is equivalent to init-soar:

https://code.google.com/p/jsoar/source/browse/jsoar-core/src/main/java/org/jsoar/kernel/commands/InitSoarCommand.java

Also note that you could also do
agent.getInterpreter().eval("init-soar") to programmatically execute
the command.

Dave

> --
> You received this message because you are subscribed to the Google Groups
> "jsoar-user" group.
> To post to this group, send email to jsoar...@googlegroups.com.
> To unsubscribe from this group, send email to
> jsoar-user+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/jsoar-user?hl=en.
>

[Angel Gavalda Pina]

Hi! I have problems with re-initialize the agent.
When the agent halt, i execute:  agent.initialize() or agent.getInterpreter().eval("init-soar") and run the agent again, but does not fire any rule and state no-change impasse arises.

Execution log:
 0: ==>S: S1
Agent reinitialized
1:    ==>S: S3 (state no-change)

I think this rule should be fire but does not.
sp {propose*initialize-jSoarbotRL
   (state <s> ^superstate nil
             -^name)
-->
   (<s> ^operator <o> + =)
   (<o> ^name initialize-jSoarbotRL)
   (write |    -> SOAR: propose initialize | (crlf))
}

What could be the problem?
I need to reload the soar source file?¿?

[Dave Ray]

Hi Angel,

Can you execute a matches command on that rule after the agent is
reinitialized? That might give a hint as to why it's not matching.

Dave

[Angel Gavalda Pina]

Hi Dave!!
I could solve the problem thanks to the matches command.
The problem arose because the elements of the input-link didn´t match after init-soar command, because I didn´t create the InputBuilder again.

I solve the problem restarting the input-link structure with
builder = InputBuilder.create(io);
builder.add("Heath", 100).markWme("Heath").

[Angel Gavalda Pina]

Forgiveness by repeating the mail my computer has gone crazy

Hi Dave!!
I could solve the problem thanks to the matches command.
The problem arose because the elements of the input-link didn´t match after init-soar command, because I didn´t create the InputBuilder again.

I solve the problem restarting the input-link structure with repeating this code:

builder = InputBuilder.create(io);
builder.add("Heath", 100).markWme("Heath").

            add()......

Thank you very much for your help!!
Best Regards!
Angel.

[Dave Ray]

Angel,

That's good to hear. You might want to consider registering for the
AfterInitSoarEvent and put that code there. Then the input-link will
be reinitialized regardless of how the agent is reinitialized
(Agent.initialize(), init-soar in debugger, etc).

Cheers,

Dave