Cumulative Tournament Results

[Dave Churchill]

https://spreadsheets.google.com/spreadsheet/ccc?key=0An3xUNEy2rixdHUza1BzZE50cGdUdHEwbGlhSU5rckE&hl=en_US

Almost 2 complete round robins have finished (out of the 25 we have
scheduled) so I am posting initial results here.

These results are not live, they have to be manually updated by me
(due to the nature of the results reporting).

Please keep an eye out for the last updated time on the spreadsheet. I
will try to update as often as possible.

Good luck to everyone involved!

--
-------------------------------
Dave Churchill
cda...@cs.ualberta.ca
dave.ch...@gmail.com

That's the thing about people who say they hate computers. What they
really hate is lousy programmers.

[Dave Churchill]

Also, apologies for not being able to sort the bots by win percentage,
but it breaks the formulas in Google Docs for some reason

Right now columns are sorted by registration date

[Dave Churchill]

I have provided a link on the spreadsheet to the raw results file:

http://ugweb.cs.ualberta.ca/~scbw/raw_results.txt

I may or may not remember to update that file every time I update the
spreadsheet, so it's data may be old at some point.

Keep in mind that the spreadsheet will NOT be the final official
result of the tournament, as we are planning on scanning the replay
files to verify game results after all games have completed.

- dave

On Tue, Aug 16, 2011 at 6:52 PM, Dave Churchill

[Dave Churchill]

Last update to scores before I come back in the morning, results will
not update for another 9 hours or so.

Also, here's a video I just took of the lab with some matches running:
http://www.youtube.com/watch?v=2evSo-8eP7I

On Tue, Aug 16, 2011 at 9:30 PM, Dave Churchill

[Dave Churchill]

Reporting an error on my part:

My parser program which parses the true raw data results (not yet
released) is reporting some games vs. BroodwarBotQ as crashes for the
opponent instead of rightfully for BroodwarBotQ

This situation is occurring because BroodwarBotQ often takes an
extremely long time on frame 0 and thus times out due to our rules.

The net result of this fix should be that several previously recorded
crashes of bots playing against BroodwarBotQ will now be listed
correctly as wins.

I will notify the mailing list when this fix has been made.

- dave

On Tue, Aug 16, 2011 at 11:51 PM, Dave Churchill

[Dave Churchill]

I have fixed the error.

Bots were incorrectly being reported as having crashed (and therefor
lost) if they were waiting on the "Drop Players" screen.

We can infer that this is the opposite case in the original raw data
file when the bot who it was waiting for reported a -1 frame count
(stalled on frame 0).

So, even though the crash is being reported erroneously, we can infer
the correct result by checking the frame count of its opponent. If the
exact scenario is a "Drop Players" timeout at the beginning of the
game (as it is in this case), the bot that was dropped will never pass
frame 0, report -1 as its final frame, while the bot doing the waiting
will report 2 as its final frame (the frame at which the drop players
screen appears).

This has now been fixed in the parser and the results have been
updated, resulting in a much higher (but truer) crash rate for
BroodwarBotQ

Sorry for any confusion. This does not affect the tournament running
whatsoever, it was just a misinterpretation of the results, which can
be discussed at length after the tournament, if necessary

- dave

[Dave Churchill]

Replace "waiting on the Drop Player screen" with "had been waiting on
the Drop Players screen for 60 seconds" (and therfore the other player
was supposed to be dropped, however we cannot push the button from
BWAPI, nor does the onFrame() get updated, so we can't leave the game
properly, thus the mislabeled crash)

On Wed, Aug 17, 2011 at 4:52 AM, Dave Churchill

[Florian Richoux]

You didn't sleep, did you?

2011/8/17 Dave Churchill <dave.ch...@gmail.com>

[gabriel synnaeve]

What? We can't run as long as we want on frame 0? I asked for it a long time ago... it's pointless to run BBQ without its onStart terrain analysis. I was intrigued by reported crashes for BBQ as I have none at home, so you're saying you're counting my terrain analysis as crashes? Sorry if I got it wrong, I'm on holidays and on my android.

Best regards,
Gabriel

[Dave Churchill]

The update time on the spreadsheet was being updated with =NOW()
overnight which I didn't know updates the time the spreadsheet was
last opened. Sorry about that. It is correct now.

I just posted a new update to the results

[Dave Churchill]

We announced the hard cap of a single frame going over 1 minute being
a loss a while ago.

We updated this by giving bots a total of 2 minutes for the first 10
frames of the game, with a hard-cap of 1 minute for any given frame.

We figured this would be more than enough time to 'set up' the bot,
especially since things like custom terrain analysis could have been
pre-computer for each of the 10 maps and included in a submission.

Apologies for anyone affected by this, but we thought it was fair to
enforce this rule in a real-time strategy game setting, especially
since the in-game "Drop Players" timer is only 50 seconds, and is
being seen by the bot during play while this analysis appears to be
being done.

- dave

On Wed, Aug 17, 2011 at 5:10 AM, gabriel synnaeve
<gabriel....@gmail.com> wrote: