Scrambles on the WCA Website

[Lucas Garron]

The last year has shown that we are capable of collecting scrambles from every competition. For transparency, for the benefit of reconstructions (which are getting quite popular), and to provide a more consistent mechanism for organizers to publish scrambles, I would like us to start publishing these with results in 2014.

I would like someone (preferably the results team, or someone who can be on it) to take on the following responsibility starting for competitions in 2014:

- Ensure that all competition results are accompanied with scrambles that are ready to post.

- Decrypt all files, and combine multiple files into one .zip if necessary.

- Upload the files on the competition results page. (Although we could do cool things like provide scramble downloads associated with each round, an unencrypted .zip should be sufficient for now. If it contains the JSON files, we can expand the functionality later.)

Stefan had a mockup that provided a simple download on a competition page, below the Information/events navigation.

1) Would someone on the website team be able to implement a simple way to make this work?

2) Would the results team be able to incorporate this into their flow? What needs to be as simple as possible to enable this?

Thanks,

»Lucas Garron

[Jeremy Fleischman]

On Tue, Dec 10, 2013 at 3:29 PM, Lucas Garron <crease...@gmail.com> wrote:

The last year has shown that we are capable of collecting scrambles from every competition. For transparency, for the benefit of reconstructions (which are getting quite popular), and to provide a more consistent mechanism for organizers to publish scrambles, I would like us to start publishing these with results in 2014.

I would like someone (preferably the results team, or someone who can be on it) to take on the following responsibility starting for competitions in 2014:

- Ensure that all competition results are accompanied with scrambles that are ready to post.

 

- Decrypt all files, and combine multiple files into one .zip if necessary.

Should this be the job of TNoodle, or something else?

 

- Upload the files on the competition results page. (Although we could do cool things like provide scramble downloads associated with each round, an unencrypted .zip should be sufficient for now. If it contains the JSON files, we can expand the functionality later.)

Stefan had a mockup that provided a simple download on a competition page, below the Information/events navigation.

1) Would someone on the website team be able to implement a simple way to make this work?

2) Would the results team be able to incorporate this into their flow? What needs to be as simple as possible to enable this?

Thanks,

»Lucas Garron

--
You received this message because you are subscribed to the Google Groups "WCA Scrambler Team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wca-scramble...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Lucas Garron]

Ideally, each competition would just have one .zip with everything. Of course, that doesn't always happen.

We just need some way to go from [multiple encrypted .zip files] to [everything online unencrypted].

Either that, or generate something more composable than encrypted .zip files.

TNoodle is welcome to help with this process, as long as it doesn't require anyone to start TNoodle if they weren't already going to run it.

»Lucas Garron

[ILKYOO CHOI]

While we are at it, I would like to make a suggestion as well.

This is not the most urgent action item, but I would like to see this implemented in the long run. 

On the competitions manage page, (https://www.worldcubeassociation.org/results/admin/competitions_manage.php), or another place with restricted access that seems appropriate, can we have a couple tick boxes that indicates that we have received the results, scrambles, and delegate report?

As far as I know, the only way to check that we have received the results is by manually going through all the competitions every week.

-- ILKYOO

You received this message because you are subscribed to the Google Groups "WCA Results" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wca-results...@googlegroups.com.

[Jeremy Fleischman]

I can't think of any good way for TNoodle to enforce that the zip or zip files the WCA is given correspond with the rounds of the competition. I wonder if the best place for this would be someplace like cubecomps, or Lars's workbook checker, where we have knowledge of what actually happened at the competition.

On Tue, Dec 10, 2013 at 3:49 PM, Lucas Garron <crease...@gmail.com> wrote:

Ideally, each competition would just have one .zip with everything. Of course, that doesn't always happen.

We just need some way to go from [multiple encrypted .zip files] to [everything online unencrypted].

Either that, or generate something more composable than encrypted .zip files.

TNoodle is welcome to help with this process,

 

as long as it doesn't require anyone to start TNoodle if they weren't already going to run it.

What do you mean by this?

[Lucas Garron]

I mean that in practice, we sometimes end up with an organizer/Delegate sending us multiple .zip files. If there were just 1, we could just remove the password and upload it.

However, there *are* multiple. So we either need to deal with this, or prevent it.

There might be several possible solutions. It would be great if TNoodle can help make things easier on the uploaders, as long as it doesn't make things *more* complicated.

»Lucas Garron

[Jeremy Fleischman]

Do you have something concrete in mind? I'm aware of the issue here, but I can't think if any easy solution. You could imagine adding a "merge zip files" option to TNoodle, but that would definitely require people to start up TNoodle where they otherwise wouldn't. In the general case, there could be tons of conflicts between zip files, dealing with that is going to be a bitch, and it will still be possible for people to send in zip files that are inconsistent with the rounds that actually happened at the competition. That's why I'm leaning towards a solution using Lars's workbook checker, or cubecomps. You could even imagine scrambles being a part of the JSON file format that we eventually want people to send to the WCA.

[Sébastien Auroux]

It would be way easier (and in my opinion more reasonable) to require the scrambles to be sent combined in one file by the delegates, similar to the requirement of using the workbook assistant before submitting the results.

2013/12/11 Jeremy Fleischman <jeremyfl...@gmail.com>

[Lucas Garron]

Then half the submissions will get it wrong, and get annoyed because when we email them back they were just missing some minor detail.

I wouldn't mind adding such functionality to the Workbook uploader... just something that we know will consistently do the same thing.

»Lucas Garron

[Jeremy Fleischman]

I agree with Sebastien, I think it's more than ok for the WCA to be strict in what it accepts. That's always been the case with the excel workbooks, why should it be any different with scrambles? That doesn't mean we can't provide delegates and organizers with tools to make it easier to get things right, which is why I suggested adding this to either Lars's workbook checker, or cubecomps.

[Sébastien Auroux]

Not everyone uses cubecomps, but adding this to the Workbook assistant sounds reasonable.

2013/12/11 Jeremy Fleischman <jeremyfl...@gmail.com>

[Jeremy Fleischman]

I'd be willing to add this functionality to Lars's checker. Lars, how do people contribute to https://www.worldcubeassociation.org/wca-workbook-assistant? Is there a github repo somewhere =)?

[Sébastien Auroux]

is it realistic that we can get this done for 2014?

[Lucas Garron]

It's very realistic; most of the problem will be getting Delegates or results folks to do their part.

I consider this overdue, and I would spend some time making it happen if I didn't have *the entire Regulations* to worry about.

»Lucas Garron

[Tim McMahon]

A scramble number box could be added to the left of the result numbers on a scorecard. The delegate can write "6" in this box if an extra scramble is awarded for the 4th solve. For example:

[2x2 - First - Group 1] [Tim McMahon] [2009MCMA01]

[ ] [#1] [9.32] [sign] [sign]
[ ] [#2] [7.11] [sign] [sign]
[ ] [#3] [9.88] [sign] [sign]
[6] [#4] [5.55] [sign] [sign]
[ ] [#5] [9.04] [sign] [sign]

A "Group" column could be added to the results workbook so that it's possible to relate scrambles to solves when there are multiple groups. This could default to "empty" if there's just one group. We'd then need to start noting down which group a competitor is in instead of randomly splitting the groups up on the spot...

A list of "Scramble Exceptions" could be added as a new spreadsheet in the result workbook. This could have the columns: Name, Country, WCA ID, Event, Round, Group, Scramble Number. Whenever an extra scramble is awarded a new entry is added to this spreadsheet.

TNoodle could then just produce one CSV file to relate the scrambles with the results.

TL;DR: /rant (the above is just brainstorming a tactical solution...)

Tim.

--
You received this message because you are subscribed to the Google Groups "WCA Results" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wca-results...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Tim McMahon
M: +61404522168
E: t...@speedcubing.com.au
W: http://www.speedcubing.com.au

[Lucas Garron]

Although that's a bit ambitious right now, there's no reason we can't work towards it.

For this year, I'd like to focus on collecting and publishing all scrambles in the first place.

We could start work on a fancier system that competition organizers can opt into, but it should also provide efficiency benefits for them.

How much would you be willing to help get things to the "publish all the scrambles in *some* format" point, Tim?

»Lucas Garron

[Tim McMahon]

I can mock up a Scrambles table and propose a patch for TNoodle that builds upon the JSON export to produce a CSV file (i.e. merge "scrambles" and "extraScrambles", assign scrambles a numeric index, export the eventId instead of "scrambler", export the event/round/group columns instead of "title"). A group column may be required for Results if we're going to relate the scrambles to them.

I have some spare time from the 26th of December until the 5th of January (mostly offline 21st to 23rd).

Tim.

[Jeremy Fleischman]

I can see the value in removing the "title" field, and making TNoodle's backend aware of events, rounds, and groups (right now, the frontend concatenates them to form the "title" field). Would you like to push on that?

I don't see why we want to "assign scrambles a numeric index", or "merge scrambles and extraScrambles". Why would we want to do CSV instead of JSON?

Regarding scheduling, I can work on making Lars's workbook checker aware of scrambles. It's not a very high priority for me right now, though. When does it need to be done by?

[Lars Vandenbergh]

Jeremy,

Here's the GitHub repository:

https://github.com/jazzthief81/wca-workbook-assistant

It would be good if someone else contributes to this project in order to increase its bus factor. So far I'm the only person who has developed on it. 

Let me know if you need any help. 

I think this all could fit very well into the workflow of the tool:

- people open a workbook like before

- they can attach a number of TNoodle zip files and provide a password for each of them

- the Assistant checks that all rounds of all events have scrambles and reports missing scrambles as a validation error

- using a button 1 consolidated zip file can be generated

- the consolidated scrambles are sent with the results to the results team and the person uploading can double check if they match using the Assistant. Results with bad or incomplete scramble files will be rejected. 

[Jeremy Fleischman]

Awesome, I'm glad you're on board, Lars. I was picturing something similar to the flow you've described, except I was thinking the Assistant would provide the ability to add/fix mislabelled scrambles and delete extra scrambles (I know the Assistant doesn't help people fix excel files, but I think it's too much to ask people to edit the JSON by hand). I also don't think that the Assistant should (or even can!) create a combined zip files, as it would require the ability to draw scrambles and generate pdfs, all of which is TNoodle code. I think the Assistant should instead produce a single JSON file that is consistent with the excel file it was asked to verify. I also believe the Assistant should accept either TNoodle zip files or just the JSON files from those zip files.

[Tim McMahon]

> I don't see why we want to "assign scrambles a numeric index", or "merge scrambles and extraScrambles".

We might end up with situations where 3-4 extra scrambles are given because some poor competitor is having a bad day (e.g. interruptions, judges messing up etc). It'd be handy if more scrambles could be generated for a particular round, on-demand, instead of just 5 + 2 extra. E.g. Delegate prints 7 scrambles and finds that a total of 8 or 9 scrambles are needed so he generates scramble #8 and #9 on the spot.

If the order is retained in the array then a numeric index isn't really required.

> Why would we want to do CSV instead of JSON?

If we're going to process Excel documents then CSV might be easier to process. JSON is really easy to process too though.

Tim.

[Jeremy Fleischman]

On Dec 19, 2013 7:14 AM, "Tim McMahon" <t...@speedcubing.com.au> wrote:
>
> > I don't see why we want to "assign scrambles a numeric index", or "merge scrambles and extraScrambles".
>
> We might end up with situations where 3-4 extra scrambles are given because some poor competitor is having a bad day (e.g. interruptions, judges messing up etc). It'd be handy if more scrambles could be generated for a particular round, on-demand, instead of just 5 + 2 extra. E.g. Delegate prints 7 scrambles and finds that a total of 8 or 9 scrambles are needed so he generates scramble #8 and #9 on the spot.

I admit that this may happen,  but I don't see why that means we need to get rid of the concept of "extra"  scrambles.

>
> If the order is retained in the array then a numeric index isn't really required.

Order is maintained.

>
>
> > Why would we want to do CSV instead of JSON?
>
> If we're going to process Excel documents then CSV might be easier to process. JSON is really easy to process too though.

I think we should stick with JSON.

> You received this message because you are subscribed to the Google Groups "WCA website and results page work" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to wca-website...@googlegroups.com.
> Visit this group at http://groups.google.com/group/wca-website.

[Lucas Garron]

Poke on this. There won't (yet?) be a Regulation explicitly mandating these to be public, but that was the intention with he 2013 changes already. I'd really like this to start happening for all 2014 competitions onward.

»Lucas Garron

[James Mertens]

Can we just make the entire .zip available for download for now?  We can get something better going down the road.

[Sébastien Auroux]

Sure, but won't that consume too much traffic?

[Lucas Garron]

Any decent webhost should be able to handle the size of the .zip files we have right now.

We've been fine with TNoodle downloads, and I've thrown significantly more traffic at Dreamhost.

»Lucas Garron

[Lucas Garron]

And yes, .zip should be fine. But we still need to handle the case where they send us two .zip files (or make them combine for us).

»Lucas Garron

[James Mertens]

We have "unlimited" bandwidth and storage (there probably is a technical limit we won't hit), so it shouldn't be a problem.

I guess we might be slightly concerned about security; what if something bad makes it in the zip file?  Short-term, we might just have them upload the text files?

[Lucas Garron]

While that could happen, I'm not especially worried about it. I'd prefer to upload the scrambles "as they were used".

People might appreciate being able to view and try the scrambles from the PDF.

»Lucas Garron

[James Mertens]

Since I don't think we anticipate having a user system for delegates any time soon, is the results team willing to upload scramble files?

[Sébastien Auroux]

I can't imagine much objection there. If we can just create a new "Upload_scrambles" script in the results admin section, where you simply have to choose the scramble file and click "Upload" this should not take longer than 10 seconds extra.

2013/12/30 James Mertens <jbc...@gmail.com>

[Jeremy Fleischman]

I'm convinced that if we don't do this right (that is, only accept json that has gone through a modified version of Lars's workbook checker), we're going to end up with lots of competitions where the events aren't inconsistent with the contents of the zip file.

I am happy to get working on this as soon as my TNoodle obligations for 2014 are done. Is there any chance this could wait until sometime later in 2014?

[Lucas Garron]

I'd be happy to let this wait until later in 2014, but I'd like every single competition in 2014 to have its scrambles uploaded at some point.

I note that if we wait too long before checking the attachments, we may receive a bunch of scrambles whether the Delegate sent an encrypted .zip and has forgotten the password.

»Lucas Garron

[James Mertens]

I agree with Jeremy here about wanting to do this well from the start.  I will make an effort to work on this soon, with json and not .zip support.

[Jeremy Fleischman]

Thanks! Just to be clear, you're going to take on the website/backend side of things, and I'll work on Lars's workbook parser?

[James Mertens]

Yes, something like that.  I'm not going to do anything too elaborate right now - just store and display the scrambles (no images/PDF downloads, no validation).

[Lucas Garron]

So, it's already February, and Delegates are still sending scrambles in various inconsistent ways (zipped without password), PDF only, ...), which means that something won't even be recovered.

How soon can we have *some* minimal process?

»Lucas Garron

[Jeremy Fleischman]

I haven't looked at Lars's tool yet. James, if I get that done, is there anything you still need to do? Will it work for delegates to send in a Json file for scrambles?

[James Mertens]

I haven't pushed this to the live WCA website nor tested it very thoroughly, but the functionality does exist.  It depends on having correctly formatted JSON objects (uploading multiple objects is fine), but it will need more metadata to be output by Lars's tool.

https://github.com/cubing/wca-website/commit/e78d2158d4ff63cb65bdf85d5c0525a3231e876f

There also isn't any code for actually displaying the scrambles yet.

[James Mertens]

In particular, scrambles will only be uploaded for rounds that have the 'eventId', 'roundId', and 'groupId' fields in the JSON object (Lars's tool should allow delegates to add this information).  These codes currently aren't validated against WCA event/round codes, even though they should match the codes to correspond to a particular round.

[Jeremy Fleischman]

I haven't read your code yet, but could you add strict checks (complaining about missing scrambles and scrambles for rounds that didn't happen)? I'd rather the code catch these sorts of issues, instead of humans.

[James Mertens]

It catches errors, just doesn't necessarily fail. Here's a screenshot:
http://oi61.tinypic.com/2z886lk.jpg

The blank name is for an eventId that doesn't exist in the Events table... I'll make that more user-friendly to fix or something I guess.

Oh, and I didn't implement a way to delete scrambles.

[Jeremy Fleischman]

Looks great! Not being able to delete scrambles is fine. I just want to make sure you error out if there are scrambles present for rounds that didn't happen. I'm not sure how you'd integrate that into your Y/N gui right now.

[James Mertens]

Well, it displays an error.  Would you rather not import scrambles for a round with no results?  I was thinking it could be used as a cross-check to indicate results may not have been uploaded properly (hence the table).  I was assuming the workbook assistant would just not include unused scrambles in it's json output.

I would also like a 'delete' feature... better to not edit the db directly if needed.

[Jeremy Fleischman]

On Tue, Feb 4, 2014 at 8:00 PM, James Mertens <jbc...@gmail.com> wrote:

Well, it displays an error.  Would you rather not import scrambles for a round with no results?  I was thinking it could be used as a cross-check to indicate results may not have been uploaded properly (hence the table).

 

I was assuming the workbook assistant would just not include unused scrambles in it's json output.

It won't, but I thought the general philosophy was that the website shouldn't make any assumptions about the JSON it is given.

 

I would also like a 'delete' feature... better to not edit the db directly if needed.

I wouldn't expect people to edit the db directly. When something is wrong with the scrambles, the results team should tell the delegate/organizer, and they should fix it.

[James Mertens]

I thought the general philosophy was that the website shouldn't make any assumptions about the JSON it is given.

Right now it doesn't, other than to make sure enough information exists to import the json.

I wouldn't expect people to edit the db directly. When something is wrong with the scrambles, the results team should tell the delegate/organizer, and they should fix it.

Well, there are always use cases like what if someone uploaded the wrong scrambles and needs to clear them out, etc.

Currently too much editing is done directly on the db - I don't think there is a way to delete data without directly querying the database.  As another use case example, Ron just had difficulty clearing out the 2AVG results.

[Jeremy Fleischman]

On Tue, Feb 4, 2014 at 8:35 PM, James Mertens <jbc...@gmail.com> wrote:

I thought the general philosophy was that the website shouldn't make any assumptions about the JSON it is given.

Right now it doesn't, other than to make sure enough information exists to import the json.

I talked to James about this over chat. I didn't realize that the rounds in the table from his screenshot is showing all the rounds found in the results *and* all the rounds found in the uploaded scrambles. For some reason, I thought it was only using the rounds found in the results.

 

I wouldn't expect people to edit the db directly. When something is wrong with the scrambles, the results team should tell the delegate/organizer, and they should fix it.

Well, there are always use cases like what if someone uploaded the wrong scrambles and needs to clear them out, etc.

Currently too much editing is done directly on the db - I don't think there is a way to delete data without directly querying the database.  As another use case example, Ron just had difficulty clearing out the 2AVG results.

I agree. Also talked to James about this online, and neither of us would like to see too many features added to the current php system if we're going to be switching over to Stefan's django system.

[James Mertens]

neither of us would like to see too many features added to the current php system if we're going to be switching over to Stefan's django system.

(But still send in feature requests/ideas!  We'll hopefully implement them in django.)