Project Numina used SymPy to win a Kaggle competition

[Francesco Bonazzi]

The recent AI Mathematical Olympiad competition on Kaggle was a competition about solving problems similar to those seen in mathematical olympiads with artificial intelligence. It was won by the team of Project Numina, a team that is developing fine-tuned LLMs to solve math problems. They were able to solve 29 out of 40 problems in the closed test set.

This is their interview on YouTube after winning the competition.

In this interview, they explain their training solution, and the two steps involved. The first involved Chain of Thought training, where the model is trained to generate a textual description of the steps needed to solve the problem. The second step translates the chain-of-thought output into python code, most of the times using SymPy.

Their solution is open-source on GitHub. They also share on HuggingFace the two datasets they have used to fine-tune an open source LLM by a different team.

Their HuggingFace space hosts a web tool to run their model.

This work looks amazing, especially as it is fully open-source!