[CFP] EBeM@IJCAI2022: 📐IJCAI International Workshop on Evaluation Beyond Metrics

[Wout Schellaert]

[Apologies for cross-posting]

2nd CALL FOR PAPERS - EBeM@IJCAI2022

The goal of this workshop is to challenge the widespread yet limited approach of evaluating the performance of intelligent systems with aggregated metrics over a benchmark or distribution of tasks. We consider papers on all areas of AI evaluation for all kinds of systems (RL agents, machine vision, language models, planning, etc.). We especially welcome alternative approaches that draw on ideas and recent progress in cognitive and developmental psychology, psychometrics, software testing, and other areas.

We will consider technical and position papers. The submission deadline is May 13th. The workshop will be part of the International Joint Conference on Artificial Intelligence (IJCAI-2022) in Vienna (Austria), with the workshop being held on July 24th. All information about topics, format and submission instructions available on the website: https://sites.google.com/view/ebem2022.

We have already confirmed two extraordinary invited speakers, Adina Williams (Facebook AI Research) and Amanda Seed (University of St Andrews). The workshop will also contain two panels, one on "Cognitive Evaluation with the Animal AI Environment", and the other one on "Big Language Models and their Generality", as well as a special session on OECD’s Artificial Intelligence and the Future of Skills (AIFS). More speakers and activities to be announced.

Main contact: ebemwo...@gmail.com

From the EbEM Organizing Team:

José Hernández-Orallo (Universitat Politècnica de València), Lucy Cheke (Cambridge University), Josh Tenenbaum (MIT), Tomer D. Ullman (Harvard), Fernando Martínez-Plumed (European Commission), Danaja Rutar (Cambridge), John Burden (Cambridge), Ryan Burnell (Cambridge), Wout Schellaert (Universitat Politècnica de València)