Who's Who? and Enhanced Who's Who?
Tijn van der Zant
of similar. I would like to propose to collapse them into a single
test, leaving more time at the competition.
An idea is to change the setting a bit. Does anybody have an idea for
a setting that is different than what we have now?
Perhaps the robot should find the children to get them to the dinner
table?
Dirk Holz
so first of all, I think we should keep rule changes at a minimum. That
is, imho it is the best way for this year to keep all the tests and to
restrict rule changes to clarifications and minor updates according to
the issues we had (especially those of singapore and the german open
because there have been many issues). Some of the GermanOpen issues
already made it into the final rulebook of 2010.
Merging the two who is who tests is, however, a good idea since we can
relax the schedule a bit. Since both tests are very similar, i don't
think that it is going to be a problem to come up with a good merger. In
principle, the only difference is the ordering+manipulation part in the
enhanced who is who. Since I really like the part with initially unknown
but learned, initially known and completely unknown persons in who is
who, I suggest to simply add the ordering+manipulation part to this
test, i.e.,
who is who + enhanced who is who = who is who + manipulation, e.g.
in the form that one of the four known persons calls the robot, orders
drinks for himself and and two other (or simple another (one)) person
and the robot needs to get these drinks. I guess that this merge is the
most intuitive one, but correct me when I'm wrong and you came up with
an even easier solution.
Once we know, how the merger should look like, we can decide on whether
to do the test in stage1 or stage2. As far as I remember, the 2010
decoupling was done to have not so many tests in stage1 that require
manipulation. But now it's 2011 RoboCup@Home. We already had 5 years, so
I do not think that it is "too much" to require manipulation in more and
more tests. @Home is about domestic service robots, so manipulation is
really a necessary capability of the robot in my opinion.
Best,
Dirk
Javier Ruiz-del-Solar
Prof. Dr. Javier Ruiz-del-Solar
Universidad de Chile
Department of Electrical Engineering
Av. Tupper 2007, 837-0451 Santiago, Chile
Ph. +56-2-978 4207
Fax +56-2-6720162
Email: jru...@ing.uchile.cl
WWW: http://www.cec.uchile.cl/~jruizd
Komei Sugiura
Komei
> Email: jru...@ing.uchile.cl <mailto:jru...@ing.uchile.cl>
> WWW: http://www.cec.uchile.cl/~jruizd
>
>
>
>
>
>
Tijn van der Zant
the 2 Who's Who they can do, with the more difficult one giving a
higher score.
It is, I think, not only about the sameness of or difference between
the two tests, but also to relieve the stressfulness of the schedule
during the competitions.
-Tijn
Javier Ruiz-del-Solar
Dirk Holz
still do not understand why we shouldn't have more and more tests
involving manipulation. We have in every test navigation and HRI for
example. Why not having manipulation? Of course, there are teams that
cannot handle manipulation up to now, but having more and more tests
necessitating manipulation to get full points, will surely encourage
_all_ teams to also work on manipulation.
Throughout, we want to see a generally applicable service robot doing
almost everything, safe navigation in domestic environments, interacting
with the user and, of course, providing a "helping hand" ... so
manipulation should become a fundamental capability for @Home in my
opinion. Again, we already had five years of competition where more and
more became fundamental capabilities. IMHO, it's getting time for
manipulation now...
Cheers,
Dirk
Javier Ruiz-del-Solar
Jesus Savage
manipulation, but a good manipulation
system requires not only to have a good manipulator but also a good
vision system to detect
objects.
Unfortunately with the light conditions in the arena and the vision
techniques used so far some of the robots' vision systems totally
fail, even
that some of the teams have good manipulation techniques, they don't
achieve picking up an object and thus not gaining points at all in the
test.
I know that this issue has been discussed in previous years, but can
we consider that some of the objects could
have marks in order that the vision system can detect objects to help
the manipulators to pick up them?
The teams that would put marks in the objects wont gain any points for
detecting the objects, but this will allow that
to see the performance of the whole operation of their service robot system.
When the vision techniques improve in the future we can take away the marks
Jesus
Dirk Holz
example, standardized things for handicapped etc. and those might be
reasonable to have in service robot applications, but, up to now I do
not see the need to put something like that into the arena. Furthermore,
I've seen a lot of successful grasps during the last competitions.
Jesus Savage
> Well, I do not like markers of any kind. Of course, there are, for
> example, standardized things for handicapped etc. and those might be
> reasonable to have in service robot applications, but, up to now I do
> not see the need to put something like that into the arena. Furthermore,
> I've seen a lot of successful grasps during the last competitions.
I would like to see the statistics of how many teams were able to
grasp objects in Singapore?
In Singapore, in some of the test, the robots basically did nothing
because their
vision system fail and we were not able to evaluate the overall
performance in the service robots area.
For the teams that do not have good vision systems could the @Home
league provide one in the same way
other leagues are doing, like in the small size league?
Mohan Rajesh Elara
Best Regards,
Mohan
Tijn van der Zant
Test Manipulation
RIPS 0
Follow Me 0
Go Get It 1
Who's who 0
Open challenge -
E Who's Who 1
GPRS 1
Shopping 1
Cleaning 1
So obviously there is not so much manipulation in stage 1, but a lot
in stage 2. Although many points can be scored with the GPRS without
manipulation.
I think that it can be possible to merge the who's who into the E
Who's who where the manipulation can be skipped (missing some points
of course) because a human hands over the item to bring. This would be
natural and no artificial markers have to be used.
A merger could look like this:
-robot at entrance
-people enter and introduce themselves
-people scatter around
-robot has 1 minute to find waving person (extra points) but after one
minute the closest person orders something (food/drink)
-robot goes to kitchen
-robot either grasps correct item (extra points) or gets the item from
another person
-robot brings the item to (hopefully) the correct person
-robot goes to door
-people leave, robot recognizes people
The way this test is described now, all the 2 who's who are merged and
one can skip manipulation and/or vision (earning less points of
course)
Since half of the test include manipulation, we should make certain
that the total scoring for manipulation does not exceed, say, 20% max.
over all the tests.
I think that it is important to merge these tests because we need the
time in the schedule.
-Tijn
On Nov 9, 4:22 am, Mohan Rajesh Elara <elar...@gmail.com> wrote:
> I agree with Javier, many teams have not mastered the manipulation yet
> (vision being a key contributor). The rate of change in technical difficulty
> must match the evolution of atleast above average and average performing
> teams. I would suggest to have two separate tests each in stage 1 and 2. We
> could study the statistics from 2011 competitions and could review the
> merger for the following year.
>
> Best Regards,
> Mohan
>
> > >> >> ------------------------------------------------------------
> > >> >> Prof. Dr. Javier Ruiz-del-Solar
> > >> >> Universidad de Chile
> > >> >> Department of Electrical Engineering
> > >> >> Av. Tupper 2007, 837-0451 Santiago, Chile
>
> > >> >> Ph. +56-2-978 4207
> > >> >> Fax +56-2-6720162
> > >> >> Email: jru...@ing.uchile.cl
Javier Ruiz-del-Solar
Dirk Holz
> a. Two separate who's who test.
leagues" run in parallel, that shouldn't be a problem.
> b. GPRS in stage 1. This allows that all teams try to solve this test.
> Considering the difficulty of the test, still it is not an integration
> test, but a one that consists (for most of the teams) in trying to
> understand the orders/wishes of humans.
Well, two things here.
First of all, the "focus on understanding the command" is just for now,
for 2012 this test might look quite different (and, again, it was meant
as a stage II test necessitating all the capabilities that we have in
the other tests at once). Second, in the whole there are only very few
points that you get for just understanding without doing anything else.
That is, having the underspecified command in cat II and asking the
correct questions is the basically the part of the test where you can
score by just understanding. In all other categories you only get points
after solving (at minimum) half of the test. In case of some object
delivery plus finding a user for example: half of the test is
approaching the object location, finding the object, and trying to grasp
it with unsuccessful delivery, or finding both person and object. Just
understanding some commands is not providing any points ...
> c. Optimizing the time required for the tests, by running most of the
> test in parallel ...
As many tests in parallel as possible. Finals, Open Challenge and Demo
Challenge should, of course, not run in parallel. Open and Demo
challenge only in parallel if we are running two leagues in parallel
until this point.
> > Let's count:
> >
> > Test Manipulation
> >
> > RIPS 0
> > Follow Me 0
> > Go Get It 1
> > Who's who 0
> > Open challenge -
> >
> > E Who's Who 1
> > GPRS 1
> > Shopping 1
> > Cleaning 1
> >
So there's almost no manipulation in stage I.
YES. That's a possible merger. And if we want to merge them, we should
do it like this. I think we already had two or three similar proposals
of which all make sense. Since it's mostly about reducing the number of
tests to have more time for testing and a relaxed schedule, I'd propose
to merge. In case we do not merge, we might still relax the schedule
enough by having two leagues in parallel.
> > -Tijn
> >
> > On Nov 9, 4:22 am, Mohan Rajesh Elara <elar...@gmail.com> wrote:
> > > I agree with Javier, many teams have not mastered the manipulation
> > > yet
> > > (vision being a key contributor). The rate of change in technical
> > > difficulty
> > > must match the evolution of atleast above average and average
> > > performing
> > > teams. I would suggest to have two separate tests each in stage 1
> > > and 2. We
> > > could study the statistics from 2011 competitions and could review
> > > the
> > > merger for the following year.
> > >
So, what average are we using then? Average of all teams? Average of
teams that make in into stage II? Average of teams making it into the
finals?
---> More than 50 percent of the finalists from 2010 successfully find
and grasp objects (for years now). We had successful grasping in the
league since at least 2007.
I do not think that it is the right way to "wait" for a lot of teams to
improve the league and adapt the rules to the ongoing state-of-the-art.
1.) As soon as there are at least something like three teams doing
almost full (partial) score for some basic capability, we should think
about making it more complicated, or let's say more _realistic_.
2.) Of course, object detection and recognition is difficult in the
@Home arena as it is (at least) more realistic than some lab setup,
grasping is also not easy, but there are a lot of approaches around that
you can try. There are a lot of manipulators around from high end arms
to low-cost approaches. Not every team might be handle manipulation of
objects in the future, but as more as it is required to score, or as
more points as it might get you, the more teams will try working on
that. We want to improve here, advance the league, and (ideally!)
advance the state-of-the-art.
3.) We even had a successful grasp in the toys'r'us store and we
shouldn't neglect an achievement like this.
>
>
>
>
Jesus Savage
ability to navigate
correctly in an environment, showing AI planning and reactive behaviors.
I think that in the previous competitions we haven't really grade this.
Suppose that in the Go Get It test a robot needs to go to an
area where the objects are, and to enter in this region there are two
doors and it plans to go
by one of them, then during the competition that door is closed, thus
the robot needs to show that
is able re-plan its path by using the other door.
For small objects that block the robot's path, like chairs, small
toys, etc, the robot should
show reactive behaviors to avoid them.
The rules don't need to be changed, they just need to specify that
dynamically the entrances to the regions could be blocked and small
objects can be put in front of the robot.
Jesus
Dirk Holz
where we had a lot of small objects lying around (quite reasonable if
you think, e.g., of toys your kids tend to keep lying around). In
principle, we can have things like that always in the arena during every
test without doing any rule change (and that already since 2008) but we
haven't applied this "increased complexity" so far (as far as I
remember). So why not starting with it. It's not only on appropriate
reactions to sudden dynamic changes like closing doors or having the way
blocked by some object but also about the robot's perception
capabilities.
Hopefully, we get Graz-like environments in Istanbul. Then, we can
easily close or block certain passages before a team enters the arena. I
appreciate that you get this point back into the discussions, Jesus. I
remember we already had this thing in other discussions. Summarizing, I
support the idea of making navigation during tests (slightly) more
complicated by blocking certain areas during the tests. We just need to
make sure that we have similar conditions for all teams (e.g. not 50m to
reach location X for team A and only 5m for team B etc.)
Dirk
Jörg Stückler
some suggestions if we go for a merger..
if we move the test completely to stage II, we dont have person
finding in stage I anymore (as a basic capability that we want to see
from all teams and that we want to use in the GPR test).
so i prefer to keep the merger in stage I.
if it's in stage I, 5 to 7 minutes will be very short to grasp 3
objects (note that every team has to perform it, so we probably wont
be able to go for 10 minutes).
we should therefore reduce the number of objects to grasp from 3 to 2
and give 7 minutes.
also, manipulation takes more time than simply handing over, so we
should decrease the available time by 2 minutes if the objects are
handed over by the user.
otherwise the test has an implicit penalty for manipulation.
best,
joerg
Tijn van der Zant
do it.
But I also like the idea of using some obstacles and changing the
environment more.
This could even be a general procedure:
1st year: change many tests, have easier environments
2nd year: little changes in tests, have more complex environments
So if we keep the tests (after a little debugging) as they are, we
should increase the scenario.
@scenario concerns
The execs are on top of this. We are aiming for 2 apartments of
approx. 7.5 by 15 meters, fully decorated and it seems that we get
that, fully decorated as in Graz.