Information regarding automatic evaluation on the evaluation server
32 views
Skip to first unread message
Khanh-Tung Tran
unread,
May 29, 2023, 8:35:32 PM5/29/23
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to VOT Challenge technical support
Hi everyone, here are some information and important notes for evaluating on the server:
Result submission Follow https://votchallenge.net/howto/overview.html for how to create your submission. Do not forget to pack the results with the vot pack command. Make sure that in the tracker identifier in the manifest.yml (by default is inside the output zip file) match with the tracker short name you register through our Google Form. Then submit your zip file in Participatetab at: AIHUB.ML - Competition.
Important notes
Successful submission will result in a FINISHED message, and you will receive an email about the results of submission shortly after.
For each submission, the evaluation will run for roughly 45 minutes to 1 hours and 30 minutes, sometimes it can take even longer depends on server loads. To avoid bottlenecking the server, you should try to submit earlier, especially when the deadline is close.
After 10 successful submissions (meaning necessary files are submitted, input structure of your zip file is correct, etc) you will not receive evaluation results for any of your submission after that.
Each successful submission needs to be at least 24h apart from each other to mitigate overfitting attempts.
Each submission that resulted in an error will not be count toward the limit of 10 successful submissions.
Feels free to ask any questions, or you can contact: aihub...@openfactor.org for questions specific to the evaluation server.
Message has been deleted
Message has been deleted
Khanh-Tung Tran
unread,
May 30, 2023, 4:50:16 AM5/30/23
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to VOT Challenge technical support
Update: You can see something like this on the submission page:
Please note that this counts both failed and successful submissions. The number of successful submission left for each tracker is sent with the results email.