Measuring the running time in case of a tie

[Marik]

Hello,

we have a question about ties. 

The official statement is as follows:

"Ties will be broken based on running time for creating the solution file (training excluded) in our environment during the reproduction process."

We assume that the "creation of the solution file" should start from scratch, 

that is, parsing the json files to extract the necessary information.

This seems to be the fairest approach.

Otherwise, one could prepare the data in the "training" in a format that makes the "creation of the solution file" a trivial or less intensive task.

 

Once again our assumption:

The raw "Specs Dataset" with the json files is the input to measure the running time of the code, 

NOT a clean table with relevant spec_id's and additional information created during the "training".

Is our assumption correct?

Thanks for the clarification,

Marik

[alaska benchmark]

Hello Marik,

The raw "Specs Dataset" with the json files is the input to measure the running time of the code, 

NOT a clean table with relevant spec_id's and additional information created during the "training".

Yes, your assumption is correct.

We put more details about the reproducibility phase in the "Evaluation Process" section of the task page. Please don't hesitate to ask if you have other questions.

Kind regards,

Andrea - Programming Contest Co-Chair