Planning for RMRC activities going forward - we need your input!
84 views
Skip to first unread message
Raymond Sheh
Jun 29, 2020, 10:18:04 PM6/29/20
to The Open Academic Robot Kit, Raymon...@curtin.edu.au
Hi All!
I hope you, your fellow team members, family, friends, and colleagues
are staying safe during these interesting times!
For those of you keeping track, the closing ceremony of RoboCup 2020 in
Bordeaux would have been happening today. Although we cannot meet in
person this year, we can still have interaction, collaboration, and
competition in an online setting!
As we're starting to see schools re-opening in some areas, we think that
now is the time to start discussing what RoboCupRescue RMRC might look
like between now and RoboCup 2021 next June in Bordeaux. To this end, we
would like to take a survey of all teams - including teams who may not
normally join us but may be able to participate in online activities.
See https://forms.gle/7mqBNyeUUYTjf4596 for the form. Please have a
mentor (over 18 for recordkeeping reasons) fill out the form, and please
only fill out one form per team. It asks questions that will allow us to
plan for both a potential online competition, as well as schedule common
times for teleconference meetings where we can all chat with each other,
see what progress we've made, and discuss what we can do with the
competition going forward. We will send out more surveys later to get
more details as necessary.
The plans for the online competition are still in flux and depend on
both when teams are likely to be able to start getting back together to
work on their robots, and how many teams are able to have access to
arenas soon afterwards. But generally, the idea is:
- Teams build the test lanes that they're interested in.
- For each run, teams provide uncut video that first shows a clock to
provide the time and date stamp, then a close up 360 of the robot to
verify the robot configuration, then a close up of the test lane, with
rulers at specific locations to show the test configuration, and then
the robot actually performing the test, ensuring that crucial details
(the robot hitting one end then the other, the operator in the
background not facing the robot, etc.) are visible.
- Teams also score themselves on the run.
- The video and scoresheet is uploaded to a video sharing site so that
anyone can verify the scoring.
- To ensure that the performance is statistically significant, teams
must provide video of some number of runs (say 10?) of each test that
they wish to be scored in (each run being the usual 5 minutes). The idea
is that teams *should* be upfront and submit videos of every single run,
even ones that didn't go so well. Yes, it is still conceivably possible
that a team can still "cherry pick" the best runs ... but if a
competitive team is able to produce a robot, in the one configuration,
that can get a good score in a test 10 times over the course of a month
(as well as get good scores in 10 runs each of a bunch of other tests),
hey that's still a good indication.
- Teams may change their robot configuration as usual but each
configuration is considered separately and only one is considered per
team for the final scoring. So there is a significant incentive to do a
lot of your own testing and development, then run all the lanes you want
to run over the course of a week or 2 once you're pretty sure of your
system.
- We can't call this an official RoboCup competition, with RoboCup
awards, because even with all these measures, there are still
uncontrolled variables. For example, while we hope we can rely on team
honesty, it is still possible, for instance, for the operator in the
videos to be fake and for the real operator to actually be eyes-on and
out of view of the video. However, we can issue each team with a
completion certificate with their performance on each test displayed
both in the customary radar chart as well as in comparative bar charts,
showing your performance relative to other teams who also competed in
the same tests.
Now of course the details of how to run this competition are not set yet
so we welcome feedback on what teams think of this. Hopefully we can
have some of these discussions online. We're also not guaranteed to run
- this will depend on when schools open. If most teams won't be able to
start working on their robots again until early next year, it can be
argued that teams might as well concentrate on RoboCup 2021.
Stay safe, we hope to see you soon, even if only virtually for now!
Cheers!
- Raymond
--
Dr. Raymond Sheh
Intelligent Robots Group
www.intelligentrobots.org
Senior Lecturer (Adjunct)
Department of Computing
Curtin University of Technology
Street: Building 314, Kent Street, Bentley WA 6102, AUSTRALIA
Mail: GPO Box U1987, Perth WA 6845, AUSTRALIA
Research Professor
Institute for Soft Matter Synthesis and Metrology
Georgetown University
506 Reiss Science Building
37th and O Streets NW, Washington DC 20057, USA
Guest Researcher
Intelligent Systems Division
National Institute of Standards and Technology
U.S. Department of Commerce
100 Bureau Drive, Stop 8230, Gaithersburg MD 20899-8230, USA
I hope you, your fellow team members, family, friends, and colleagues
are staying safe during these interesting times!
For those of you keeping track, the closing ceremony of RoboCup 2020 in
Bordeaux would have been happening today. Although we cannot meet in
person this year, we can still have interaction, collaboration, and
competition in an online setting!
As we're starting to see schools re-opening in some areas, we think that
now is the time to start discussing what RoboCupRescue RMRC might look
like between now and RoboCup 2021 next June in Bordeaux. To this end, we
would like to take a survey of all teams - including teams who may not
normally join us but may be able to participate in online activities.
See https://forms.gle/7mqBNyeUUYTjf4596 for the form. Please have a
mentor (over 18 for recordkeeping reasons) fill out the form, and please
only fill out one form per team. It asks questions that will allow us to
plan for both a potential online competition, as well as schedule common
times for teleconference meetings where we can all chat with each other,
see what progress we've made, and discuss what we can do with the
competition going forward. We will send out more surveys later to get
more details as necessary.
The plans for the online competition are still in flux and depend on
both when teams are likely to be able to start getting back together to
work on their robots, and how many teams are able to have access to
arenas soon afterwards. But generally, the idea is:
- Teams build the test lanes that they're interested in.
- For each run, teams provide uncut video that first shows a clock to
provide the time and date stamp, then a close up 360 of the robot to
verify the robot configuration, then a close up of the test lane, with
rulers at specific locations to show the test configuration, and then
the robot actually performing the test, ensuring that crucial details
(the robot hitting one end then the other, the operator in the
background not facing the robot, etc.) are visible.
- Teams also score themselves on the run.
- The video and scoresheet is uploaded to a video sharing site so that
anyone can verify the scoring.
- To ensure that the performance is statistically significant, teams
must provide video of some number of runs (say 10?) of each test that
they wish to be scored in (each run being the usual 5 minutes). The idea
is that teams *should* be upfront and submit videos of every single run,
even ones that didn't go so well. Yes, it is still conceivably possible
that a team can still "cherry pick" the best runs ... but if a
competitive team is able to produce a robot, in the one configuration,
that can get a good score in a test 10 times over the course of a month
(as well as get good scores in 10 runs each of a bunch of other tests),
hey that's still a good indication.
- Teams may change their robot configuration as usual but each
configuration is considered separately and only one is considered per
team for the final scoring. So there is a significant incentive to do a
lot of your own testing and development, then run all the lanes you want
to run over the course of a week or 2 once you're pretty sure of your
system.
- We can't call this an official RoboCup competition, with RoboCup
awards, because even with all these measures, there are still
uncontrolled variables. For example, while we hope we can rely on team
honesty, it is still possible, for instance, for the operator in the
videos to be fake and for the real operator to actually be eyes-on and
out of view of the video. However, we can issue each team with a
completion certificate with their performance on each test displayed
both in the customary radar chart as well as in comparative bar charts,
showing your performance relative to other teams who also competed in
the same tests.
Now of course the details of how to run this competition are not set yet
so we welcome feedback on what teams think of this. Hopefully we can
have some of these discussions online. We're also not guaranteed to run
- this will depend on when schools open. If most teams won't be able to
start working on their robots again until early next year, it can be
argued that teams might as well concentrate on RoboCup 2021.
Stay safe, we hope to see you soon, even if only virtually for now!
Cheers!
- Raymond
--
Dr. Raymond Sheh
Intelligent Robots Group
www.intelligentrobots.org
Senior Lecturer (Adjunct)
Department of Computing
Curtin University of Technology
Street: Building 314, Kent Street, Bentley WA 6102, AUSTRALIA
Mail: GPO Box U1987, Perth WA 6845, AUSTRALIA
Research Professor
Institute for Soft Matter Synthesis and Metrology
Georgetown University
506 Reiss Science Building
37th and O Streets NW, Washington DC 20057, USA
Guest Researcher
Intelligent Systems Division
National Institute of Standards and Technology
U.S. Department of Commerce
100 Bureau Drive, Stop 8230, Gaithersburg MD 20899-8230, USA
Raymond Sheh
Jan 21, 2021, 10:58:25 AM1/21/21
to The Open Academic Robot Kit, raymon...@georgetown.edu
Hi All!
Here are the ideas for a virtual RMRC ... originally this was conceived
as a way of running an intermediate competition before the World
Championships this year but, because this year's World Championships are
going virtual, this is now the current plan to run the World Championships.
A few questions to focus on.
- If we were to run this in the May-June timeframe, do you think you
would be able to build your robot and your arenas by then?
- Which tests would you be interested in, and able to build?
- Does the procedure look reasonable to follow?
- Do you have any specific thoughts about the details - expected level
of statistical significance, what to show in the videos to verify test
method and robot configuration, etc.?
Depending on time availability, we're also thinking that perhaps this
process could be used to select finalists, who will then do runs live
(Youtube or Zoom or the like). What do you think about that possibility,
both in terms of need as well as logistics?
Cheers!
- Raymond
Here are the ideas for a virtual RMRC ... originally this was conceived
as a way of running an intermediate competition before the World
Championships this year but, because this year's World Championships are
going virtual, this is now the current plan to run the World Championships.
A few questions to focus on.
- If we were to run this in the May-June timeframe, do you think you
would be able to build your robot and your arenas by then?
- Which tests would you be interested in, and able to build?
- Does the procedure look reasonable to follow?
- Do you have any specific thoughts about the details - expected level
of statistical significance, what to show in the videos to verify test
method and robot configuration, etc.?
Depending on time availability, we're also thinking that perhaps this
process could be used to select finalists, who will then do runs live
(Youtube or Zoom or the like). What do you think about that possibility,
both in terms of need as well as logistics?
Cheers!
- Raymond
matej...@gmail.com
Jan 24, 2021, 12:38:01 PM1/24/21
to The Open Academic Robot Kit
Hello
From Mechatronics Rescue Team - we think that we will be able to build robot. Depence of pandemic situation. But we think that, yes.
We have some thoughts. Can every team made exact the same labirint-test??
We will think about it...
See you virtualy
Best regards
Matej
četrtek, 21. januar 2021 ob 16:58:25 UTC+1 je oseba Raymond.Sheh napisala:
Timothy Jump
Jan 25, 2021, 1:47:23 PM1/25/21
to matej...@gmail.com, The Open Academic Robot Kit
Team RoboBlues:
We have all the 30cm testing field elements with certain variations (no walls on any of the fixtures; shorter end runs on the gravel/sand fixture; only one gravel/sand fixture so we must switch out from one debris element to the next; hurdle fixture is set
to make a right turn/L-shape rather than the parallel end runs).
We have sensor crates with all elements but the magnet test (we never finalized the concept or design integration of this test).
Uncertain what our status is to generate a fully functional robot. We are on virtual classes until March 4, and the virtual progress toward preparing a robot has been extremely limited. We may be limited to sensor tests as we can adapt sensors onto a basic
robot chassis for these.
Timothy E. Jump
Director, Quadrivium Engineering and Design
From: oar...@googlegroups.com <oar...@googlegroups.com> on behalf of matej...@gmail.com <matej...@gmail.com>
Sent: Sunday, January 24, 2021 12:38 PM
To: The Open Academic Robot Kit <oar...@googlegroups.com>
Subject: CAUTION - POSSIBLE SPAM! - [OARKit] Re: Planning for RMRC activities going forward - we need your input!
Sent: Sunday, January 24, 2021 12:38 PM
To: The Open Academic Robot Kit <oar...@googlegroups.com>
Subject: CAUTION - POSSIBLE SPAM! - [OARKit] Re: Planning for RMRC activities going forward - we need your input!
--
You received this message because you are subscribed to the Google Groups "The Open Academic Robot Kit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to oarkit+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/oarkit/ff82397e-7719-44b0-8aa7-98f8bc977e26n%40googlegroups.com.
You received this message because you are subscribed to the Google Groups "The Open Academic Robot Kit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to oarkit+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/oarkit/ff82397e-7719-44b0-8aa7-98f8bc977e26n%40googlegroups.com.
Raymond Sheh
Jan 29, 2021, 3:54:55 PM1/29/21
to The Open Academic Robot Kit, raymo...@gmail.com
Hi Tim, All,
Thanks for the update and sorry to hear of the continued restrictions on your preparations.
Sensor tests might be something that all teams could work on. Would this also stretch to, say, the Survey Acuity test on continuous ramps (or even on flat flooring), so it doesn't need advanced mobility?
Cheers!
- Raymond
Thanks for the update and sorry to hear of the continued restrictions on your preparations.
Sensor tests might be something that all teams could work on. Would this also stretch to, say, the Survey Acuity test on continuous ramps (or even on flat flooring), so it doesn't need advanced mobility?
Cheers!
- Raymond
To view this discussion on the web, visit https://groups.google.com/d/msgid/oarkit/MN2PR18MB34476D6E66D72FF3EE6ADAB0A0BD1%40MN2PR18MB3447.namprd18.prod.outlook.com.
Raymond Sheh
Jan 29, 2021, 4:08:48 PM1/29/21
to oar...@googlegroups.com, raymo...@gmail.com
Hi Matej, All,
Great to hear that you may be able to put together a robot for the competition!
If by labyrinth you mean the finals arena (where the lanes are put together into one big maze), I think we may need to let go of that for this year and have the competition end at what would normally be preliminaries. The logistics of making sure everyone can build the same finals arena would be too great I think.
There is also the idea from Major, whereby we don't award first, second, and third place and, instead, concentrate on Best-in-Class awards, with broad award bands for top X%, in recognition of the fact that with remote scoring there will be large uncertainties in the measurements, even with everyone operating in good faith.
Of course you're all welcome to join us in brainstorming these ideas. Perhaps we should resurrect the Discord server? It's certainly very quiet in there, I notice that a few folks (looks like mostly King's Legacy and Smoking LiPos) are still active from time to time. Given the time zone differences, maybe discussing there may be easier than in a teleconference.
If you're not already in there, see https://discord.gg/Mnpc9fV to join us. Once we get a vaguely critical mass (pun intended) we can start bouncing around some more ideas.
Cheers!
- Raymond
Great to hear that you may be able to put together a robot for the competition!
If by labyrinth you mean the finals arena (where the lanes are put together into one big maze), I think we may need to let go of that for this year and have the competition end at what would normally be preliminaries. The logistics of making sure everyone can build the same finals arena would be too great I think.
There is also the idea from Major, whereby we don't award first, second, and third place and, instead, concentrate on Best-in-Class awards, with broad award bands for top X%, in recognition of the fact that with remote scoring there will be large uncertainties in the measurements, even with everyone operating in good faith.
Of course you're all welcome to join us in brainstorming these ideas. Perhaps we should resurrect the Discord server? It's certainly very quiet in there, I notice that a few folks (looks like mostly King's Legacy and Smoking LiPos) are still active from time to time. Given the time zone differences, maybe discussing there may be easier than in a teleconference.
If you're not already in there, see https://discord.gg/Mnpc9fV to join us. Once we get a vaguely critical mass (pun intended) we can start bouncing around some more ideas.
Cheers!
- Raymond
Timothy Jump
Jan 29, 2021, 4:27:56 PM1/29/21
to Raymond Sheh, The Open Academic Robot Kit
We have robot options for low mobility.
We have a testing field simulated building with 30cm pathways and a ramp to a second floor (designed after the RoboCup Jr. Rescue arena, but with some space saving variations and accommodations for mobility elements and sensor crates). We can use this testing
arena to operate as a Sensor Acuity Testing field, it is just not the 60cm testing lanes like we used in Sydney.
Timothy E. Jump
Director, Quadrivium Engineering and Design
From: oar...@googlegroups.com <oar...@googlegroups.com> on behalf of Raymond Sheh <raymo...@gmail.com>
Sent: Friday, January 29, 2021 3:54 PM
Sent: Friday, January 29, 2021 3:54 PM
Cc: raymo...@gmail.com <raymo...@gmail.com>
Subject: [OARKit] Re: Planning for RMRC activities going forward - we need your input!
Subject: [OARKit] Re: Planning for RMRC activities going forward - we need your input!
To view this discussion on the web, visit
https://groups.google.com/d/msgid/oarkit/b42a28dc-8d3a-92cb-1acd-4dcd540b5f51%40gmail.com.
Reply all
Reply to author
Forward
0 new messages