rulebook discussions and final version for 2011

[Dirk Holz]

Hi all,

it seems that there has not been any email correspondence on the list
for more than one week now. Since I'd like to have a preliminary
rulebook for 2011 fixed as soon as possible, we should try to summarize
our discussions and see if we came up with a solution or consensus.

---> Not having any discussion on the list suggests that we came up with
a consensus. Furthermore, 2011 is going to be the second year in the
rulebook cycle, which basically means that we do not have to change
anything, unless there is a reason for the change.

Discussions that we had were basically:
* Parallel groups in Stage I
* Parallel groups in Stage I and II
* Merging "Who Is Who" and "Enhanced Who Is Who"
* Which tests in Stage I, which tests in Stage II
* Intention and (story behind) the demo challenge
* Open Challenge: Slides vs. Demonstration, and what should happen
* RIPS: what should happen during RIPS
* Changing Shopping Mall description (getting adapted to the 2011 setup,
whatever is possible in Istanbul)

Regarding the latter points, I guess we had a consensus:

* Shopping Mall:
the test description needs to be more general (I'll do that), it will
definitely be online SLAM, safe navigation and manipulation in a
real-world shopping scenario.
"... Details are going to be made available as soon as EC/TC and local
organizers have agreed on some particular location ..."
The test itself is not going to be changed

* RIPS:
* There are two arenas:
* For each arena, we specify two registration desks (before the test,
most probably some table within the scenario), the (all) robot(s)
through one door (specified before the test), go to the registration,
hand over the registration form, stand still for getting expected, can
be told to move (e.g., by a team mate, by pressing some button, doesn't
matter), so we can check the emergency button(s).
IMPORTANT here: we want to make clear, that it is the team's
responsibility to get the robot(s) re-checked if there have been issues:
1.) Robot's not being successfully inspected, are _not_ allowed to
participate in a test (despite RIPS).
2.) We do not want to run around the venue, searching for all robots
(and teams) that had issues during RIPS, to get the robot's inspected.
We might do a short second slot the next day after RIPS, in the morning,
where teams have the possibility to get their robot's re-checked. After
that, it is really the team's responsibility to get their robot's
checked. Everything is in the rules, so issues are minor and I assume
that they can get fixed...

If you agree to the things above, I can take care of that and that it is
going into the rules. For the other points (and in case I forgot a
discussion), let's just get through the email discussions, make up our
mind, and let's get the rules fixed asap. Details within a test or
issues we had in previous competitions can still be discussed when the
draft is ready. This year I'd like to have the rules fixed at least one
month before the local competitions start (which is probably February or
March).

Best,
Dirk

[Javier Ruiz-del-Solar]

Fine with me.

------------------------------------------------------------
Prof. Dr. Javier Ruiz-del-Solar
Universidad de Chile
Department of Electrical Engineering
Av. Tupper 2007, 837-0451 Santiago, Chile

Ph. +56-2-978 4207
Fax +56-2-6720162
Email: jru...@ing.uchile.cl
WWW: http://www.cec.uchile.cl/~jruizd

[Tijn van der Zant]

Hi Dirk,

You're right. I would appreciate it is people could take
responsibility and pick a test to summarize the results of the
discussions. Putting my money where my mouth is: I will write a
description for the Open Challenge.

--Tijn

On Dec 14, 12:59 pm, Javier Ruiz-del-Solar <jru...@ing.uchile.cl>
wrote:

[Tijn van der Zant]

LOL

I meant I will write the description for the Demo Challenge

Cheers,

Tijn

[Jesus Savage]

Hello Dirk,

date		Thu, Dec 2, 2010 at 2:05 PM
subject		Re: Parallel groups and Stage I & II
mailing list		<athome2011.googlegroups.com> Filter messages from this mailing list

>Yes, that's something that I proposed last year already. To have some
>mini challenges within a first stage I test. Something like having the
>robot enter the scenario and then accomplishing some simple tasks, like
>"follow me", "stop", "move to the kitchen table". Basically the GPSR has
>arisen from that, but I agree. We could come up with some minimal set of
>capabilities that every robot should have and test them in something
>like a technical challenge. The most important thing here is (see
>examples above), to have very simple commands (no GPSR here) and, more
>important, tasks that can be done independent of each other, e.g.
>recognizing a person doesn't make sense without training at least two
>persons (and that's more or less a complete test again just like who is
>who). But if we manage to extract some minimal set of tasks (defining
>the "challenges" described by Jesus above) that can easily be tested and
>evaluated as it is done in other leagues, than "YES!", let's do it!

Should we write also for the preliminary rulebook the mini challenges test for stage I?

Best

Jesus

[Javier Ruiz-del-Solar]

I prefer to keep the same tests than last year, with just some minor modifications, and to decide which test will be included in stage I and which in stage II.

[Tijn van der Zant]

I updated the tests of the rulebook which is not my task... I also
wrote the DC.
Also I added the GPSR I test to Stage I, and Stage II has the GPSR II
The lack of activity of the TC regarding the rulebook is frustrating,
to say the least.

Still stuff has to be done:
- update TC/OC members. In case of no update before next Wednesday, I
just delete these sections.
- check the www links
- pictures from 2009 and 2010 in 3.2 scenario
- something about parallel competition
- 5 or 6 teams to the finals

can someone pick this up? I've been working already for about 8 hours
on this...

I would like to publish next week...

[Komei]

Hi,

> - pictures from 2009 and 2010 in 3.2 scenario

I'll take care of the pictures.

@TC/OC list
I have updated the TC section, but some information such as
affiliations is still missing.
Also, It would be helpful if someone could tell me the OC
information.

Best,

Komei

[Komei]

Hi all,

I have updated the picture section, but the picture of 2009's local
competition is still missing. Currently, the rulebook has two German
Open and one Japan Open pictures, so I would like to include a picture
from other local competition (Iran Open, Mexican Open, etc.). Could
any one of you upload a picture?

Best,

Komei

[Komei]

Hi all,


@10 or half of the teams to Stage II
The rulebook says "ten teams advance into Stage II", but isn't it
max(10, N/2) teams? N is the number of participating teams.
e.g.
- 28 teams (Stage I) -> 14 teams (Stage II)
- 18 teams (Stage I) -> 10 teams (Stage II)
- 25 teams (Stage I) -> 12 teams (Stage II)


@Outstanding performance
The rulebook says that an extra bonus for outstanding performance is
10%
of the maximum test score. But, it seems ambiguous to me how much is
given to a team which deserves the bonus but has failed some part of
the
task. There will be some options, what do you think?

Option A)
TC can give an extra 10% bonus to a team if it has obtained 100% of
the score and shown an innovative approach. TC can give an extra 5%
bonus to a team if it has obtained at least 50% of the score and shown
an innovative approach. If the score is less than 50%, no extra bonus
is
given.

[pseudo code]
if(outstanding_performance == true){
if(score == 100%){
outstanding_performance_bonus = 10%
}
elsif(score >= 50%){
outstanding_performance_bonus = 5%
}
else{}
}

Option B)
if(outstanding_performance == true){
if(score >= 50%){
outstanding_performance_bonus = 10%
}
else{
outstanding_performance_bonus = 5%
}
}

Option C)
if(outstanding_performance == true){
outstanding_performance_bonus = 10%
}


Best,

Komei

[Dirk Holz]

Hi all,

sorry for not replying up to now, had lots of stuff getting done
including moving back to germany ... so back again and ready to work
(will take care of some of the TODOs).

Regarding number of teams, yes let's do max(10, N/2). We also need to
decide if stage II teams are determined per group, i.e., max(5,
N_group/2), or over all teams with max(10, N/2). Over all groups
definitely makes more sense assuming that Oopen Challenge scores will be
similar (what i assume) or we just normalize them ...

Regarding outstanding performance, option A is the one that we used in
the last competitions and is the one which is the most reasonable in my
opinion...

Dirk

--
----------------------------------------------------------------------
Dirk Holz
Autonomous Intelligent Systems Group http://www.ais.uni-bonn.de/~holz
Department of Computer Science VI ho...@ais.uni-bonn.de
University of Bonn Tel.: +49 228 73 4434
D-53117 Bonn, Germany Fax.: +49 228 73 4425
----------------------------------------------------------------------

[Komei]

Hi all,

@ max(10, N/2) or max(5, N_group/2)
I agree with Dirk.

Best,

Komei

> Department of Computer Science VI     h...@ais.uni-bonn.de

[Tijn van der Zant]

Hi,

Can people proofread the rulebook. I am also doing that, but due to an
ear infection and the accompanying headaches this is going a bit slow.
If people can read the tests, and also the new stage I and II part
about the gpsr.
Let's get this out there this weekend!

Cheers,

Tijn