VNN-COMP'24 Call for Participation

[Taylor Johnson]

The 5th International Verification of Neural Networks Competition (VNN-COMP'24), to be held with the 7th International Symposium on AI Verification (SAIV'24) at the 36th International Conference on Computer Aided Verification (CAV'24) over July 22-23, 2024, in Montreal, Canada, aims to bring together researchers interested in formal methods and tools providing guarantees about the behaviors of neural networks and systems built from them. Participation will again be conducted remotely in advance of the conference and attendance at the conference will not be required, but results will be finalized and presented there.

The website has been updated with tentative dates for the 2024 edition, which will overall follow similar procedures as the prior couple iterations. We plan one change, in that we aim to have two tracks: one will consider prior benchmarks proposed in prior editions of VNN-COMP and the other that will consider novel benchmarks not previously considered in a prior edition of VNN-COMP. We will coordinate with the participants on procedures for the existing and novel benchmarks tracks, for instance, possibly considering different inputs (e.g., different random seeds used at evaluation) when this makes sense, and discuss this more in the rules formation process. We hope that these 2 tracks will encourage increased participation in VNN-COMP, as based on feedback from the last iteration, some teams were unable to participate, as many benchmarks proposed had novel architectures and would require tool modification to support, and there was not sufficient time to add such support in many cases. 

https://sites.google.com/view/vnn2024 

If you are interested to participate with a tool, propose a benchmark, or participate otherwise, please submit the registration/interest form here so we can keep you apprised of next steps.

https://forms.gle/TDng8k7Kd2qd8Pcu8  

Important Dates:

• Intention to participate: March 18, 2024
• Rules meeting (Zoom): April 3, 2024, 9am US central time
• Finalization of the rules: April 26, 2024
• Submission of new benchmarks: May 6, 2024
• Participants finalize tool scripts and organizers begin running tools: June 28, 2024
• SAIV with presentation of VNN-COMP results and report: July 22-23, 2024

Finally, if you are interested to be more involved in the organization of the competition this iteration or have any suggestions, please let us know by email.

Best regards,

Taylor Johnson (on behalf of the organizers)

http://www.taylortjohnson.com/