Collaboration offer: SOW 2025-Codabench-Design
anne-catherine letournel
Period: March to August 2025
Background:The HUMANIA project https://guyon.chalearn.org/projects/humania
(Université Paris- Saclay) would like to hire an expert in
challenge design to re-inforce the Codabench team in specifying
future needs and accompanying the growth and the development of
the Codabench production platform, an open-source project
https://github.com /codalab/codabench. The platform is growing
steadily and has been ranked first in an independent study:
https://mlcontests.com/state-of-competitive-machine-learning-2023/.
Constant development is required to keep the platform at the state
of the art and to select the new functionalities expected by the
research community using this tool. The prospective hire will be
integrated in a team of administrators, developers, and challenge
organizers, and have the mission of designing new features,
answering organizers questions and capitalizing hints into a
structured documentation, participating in the release strategy of
platform components. The ideal candidate will already be familiar
with benchmark organization, have already worked with Codabench,
and have experience with open-source projects.
The candidate will have competence in challenge organization and knowledge in the codalab/codabench platform software:
* Extended knowledge in platform features and usage,
* Great experience in challenge design,
* Robust technical background: docker, web servers, storage services (Minio),
* Familiarity with GPUs, cloud computing environments, Linux operating system.
* Diagnosis and problem solving,
* Quality methods, tools: git version control system,
* Programming language: Python and javascript.
* Technical English: language of the Codalab community, of documentation and of meetings.
Scope of work:
Work to be performed will include:
1- Participating and animating the weekly meetings (with the team at large and with the developers).
2- Designing new features as github issues or as CallForBids. Controlling the development and the documentation of new features by reviewing them and integrating them into the main branch of the platform.
3- Helping analyzing bugs and classifying them as github issues.
4- Guiding the organizers into a reasonable usage of the platform, helping the project to control the use of material resource.
5- Insuring the project dissemination and promote it through dedicated events.
Deliverables:
1- Meeting minutes and mail communication when necessary.2- Contributions to github issues (improvements) specification and validation. Insuring PR merges and deployment on targets instances.
3- Answers and labels github issues (bugs).
4- Answering organizers questions and providing structured notes for FAQ list.
5- Participating into planned projects promotion events.
Terms and conditions:
The HUMANIA project https://guyon.chalearn.org/projects/humania (Université Paris- Saclay) will represent the client. Representatives of the client will review the delivered work and approve payment. Contractor will send monthly reports and a mid-term bill.
How to apply?Please provide a CV accompanied by a letter with your motivation, your achievements on codabench platform in 2024 and a detailed proposition with cost evaluation according to the time you plan to spend on the project in answer to the requirements. Email answers or questions to: info[AT]codalab[DOT]org.
Application deadline: February, 21st, 2025.
Regards,Codalab team