Learning Track Suggestions

[Erek Speed]

Hi,

So, it seems like the interface to the Learning Track will be changing
which seems positive so I will post some ideas about the Learning
Track in general.

First, I'll talk about the interface. So, it seems the ideal solution
gives people control over their evaluations so they can write whatever
algorithms they want. You could just use the setNumberOfTrials() and
check their code to make sure they don't violate it but that is bad
because it doesn't lead well to automation and can possibly be
cheated.

I think the best solution is to have the task keep track of the number
of trials. So, you pass in a task with setTask() and then the
algorithm can use it however they want by calling Task.evaluate() for
example.

At the same time, the task keeps an internal counter of how many times
evaluate() has been called, so that when the magic number has been
reached it will no longer run the trial.

The task would probably need a query function so people could know the
status of their trial quota. Functions like Task.canEvaluate() and
Task.getRemainingTrialCount() or something. (Obviously these
functions wouldn't be inside the actual Task class, but in some
subclass.)

Second, I want to talk about the general execution of the track since
I think it was pretty good but still have some suggestions.

1) Run each 'level' multiple times to get an average. These
algorithms are stochastic so any one run could be terrible or awesome.
It seems that maybe running each 'level' 5 or 10 times and taking the
average would be more consistent. It seems bad if the winner of the
competition would change each time you ran it. On the other hand, if
people's algorithms are very slow then maybe it's not feasible to do
it unless the deadline is ahead of time or there's a cluster of
computers to run it on. :)

2) I don't know about the levels of the last competition, but it seems
that the top 3 competitors completed all the levels. First, it'd be
cool if the final runs were shown (the recorder will make that easy).
Moreover, it'd be good if the final level was hard enough that it was
expected that no AI could beat it. There's all sorts of interesting
features that make this possible but for instance having dead ends
with hidden blocks on difficulties greater than 20 is pretty hard.
Especially if you can make it dead ends deep. Underground levels are
also hard but easily have impossible levels which aren't so
interesting. (the new level generation might correct for this.)
Finally, it seems like it would be interesting if levels were custom
made to be tricky for the AI. That's a lot of work but people could
submit level designs, though there might not be enough interest. (A
level editor would be an interesting addition to the code.)

That's a lot of text, but thanks for reading or skimming or whatever.

Erek

[Sergey Karakovskiy]

Hi, Erek

Yes, thank you for your essential feedback!

On Oct 29, 8:01 am, Erek Speed <melin...@gmail.com> wrote:
> Hi,
>
> So, it seems like the interface to the Learning Track will be changing
> which seems positive so I will post some ideas about the Learning
> Track in general.
>
> First, I'll talk about the interface.  So, it seems the ideal solution
> gives people control over their evaluations so they can write whatever
> algorithms they want.  You could just use the setNumberOfTrials() and
> check their code to make sure they don't violate it but that is bad
> because it doesn't lead well to automation and can possibly be
> cheated.
>

yep, that was my initial idea, but with growing number of submission
it becomes really tough to be sure that this interface is followed as
it supposed to be.

> I think the best solution is to have the task keep track of the number
> of trials.  So, you pass in a task with setTask() and then the
> algorithm can use it however they want by calling Task.evaluate() for
> example.

Then the Task Implementation have to be a Singleton

>
> At the same time, the task keeps an internal counter of how many times
> evaluate() has been called, so that when the magic number has been
> reached it will no longer run the trial.

right, it is going to be like that.

>
> The task would probably need a query function so people could know the
> status of their trial quota. Functions like Task.canEvaluate() and
> Task.getRemainingTrialCount() or something.  (Obviously these
> functions wouldn't be inside the actual Task class, but in some
> subclass.)
>
> Second, I want to talk about the general execution of the track since
> I think it was pretty good but still have some suggestions.
>
> 1) Run each 'level' multiple times to get an average.

using the class StatisticalSummary, which is already implemented and
ready to be used, yes!

 These
> algorithms are stochastic so any one run could be terrible or awesome.
>  It seems that maybe running each 'level' 5 or 10 times and taking the
> average would be more consistent.  It seems bad if the winner of the
> competition would change each time you ran it.  On the other hand, if
> people's algorithms are very slow then maybe it's not feasible to do
> it unless the deadline is ahead of time or there's a cluster of
> computers to run it on. :)

That's why we keep this tight bound (42ms) on evaluation per single
decision making process (action).

>
> 2) I don't know about the levels of the last competition, but it seems
> that the top 3 competitors completed all the levels.  First, it'd be
> cool if the final runs were shown (the recorder will make that easy).

now we have the recorder/replayer and all runs will be available.

> Moreover, it'd be good if the final level was hard enough that it was
> expected that no AI could beat it.

yep, this first run on Learning track was sort of experimental to just
give chance for AI to make some progress.

 There's all sorts of interesting
> features that make this possible but for instance having dead ends
> with hidden blocks on difficulties greater than 20 is pretty hard.


> Especially if you can make it dead ends deep.

sure, variance of deepness of a dead-end automatically scales and is
proportional to the difficulty of the level

 Underground levels are
> also hard but easily have impossible levels which aren't so
> interesting. (the new level generation might correct for this.)

what exactly is not so elegant as it could be?

> Finally, it seems like it would be interesting if levels were custom
> made to be tricky for the AI.

We will include 4 levels produced manually with a Level Generator

 That's a lot of work but people could
> submit level designs, though there might not be enough interest. (A
> level editor would be an interesting addition to the code.)

there is one ;-) consider class LevelEditor. I'll post a "Call for
Levels" Message to the group where all of you can submit interesting
levels. Now, using the -fly option and recordings levels can be easily
examined, how actually curious they are.

>
> That's a lot of text, but thanks for reading or skimming or whatever.

Thank you for your thorough review! That helps a lot!

>
> Erek

[Erek Speed]

Thanks for the responses. I have a few clarifications below.

On Wed, Nov 10, 2010 at 1:37 AM, Sergey Karakovskiy
<sergey.ka...@gmail.com> wrote:
> That's why we keep this tight bound (42ms) on evaluation per single
> decision making process (action).

There's a bound on each single action. But I think people can do as
much processing as they want in between trials which might be
expensive.

> what exactly is not so elegant as it could be?

So, underground levels (and castles) have roofs so it's easy for auto
generated levels to have something like cannons blocking every path.
This type of level creates an unwinnable level. Above ground, mario
can jump above the top of the screen and get past almost all levels.