EBeM 2022
AI Evaluation Beyond Metrics 2022

Table of Contents

• Preface
  Summary: There were 16 papers submitted for peer-review to this workshop. Out of these, 10 papers were accepted for this volume, and 3 extra were accepted but elected not to be published.

• Evaluating Sports Analytics Models: Challenges, Approaches, and Lessons Learned
  Jesse Davis, Lotte Bransen, Laurens Devos, Wannes Meert, Pieter Robberechts, Jan Van Haaren, Maaike Van Roy
• Evaluating Object Permanence in Embodied Agents using the Animal-AI Environment
  Konstantinos Voudouris, Niall Donnelly, Danaja Rutar, Ryan Burnell, John Burden, José Hernández-Orallo, Lucy Cheke
• A Framework for Categorising AI Evaluation Instruments
  Anthony G Cohn, José Hernández-Orallo, Julius Sechang Mboli, Yael Moros-Daval, Zhiliang Xiang, Lexin Zhou
• Reject Before You Run: Small Assessors Anticipate Big Language Models (short paper)
  Lexin Zhou, Fernando Martínez-Plumed, José Hernández-Orallo, Cèsar Ferri, Wout Schellaert
• The Relevance of Non-Human Errors in Machine Learning (short paper)
  Ricardo Baeza-Yates, Marina Estévez-Almenzar
• Robustness Testing of Machine Learning Families using Instance-Level IRT-Difficulty
  Raül Fabra-Boluda, Cèsar Ferri, Fernando Martínez-Plumed, Maria Jose Ramirez-Quintana
• Item Response Theory to Evaluate Speech Synthesis: Beyond Synthetic Speech Difficulty (short paper)
  Chaina Oliveira, Ricardo Prudêncio
• Evaluating Understanding on Conceptual Abstraction Benchmarks
  Victor Vikram Odouard, Melanie Mitchell
• On Young Children’s Exploration, Aha! Moments and Explanations in Model Building for Self-Regulated Problem-Solving
  Vicky Charisi, Natalia Díaz-Rodríguez, Barbara Mawhin, Luis Merino
• FERM: A FEature-space Representation Measure for Improved Model Evaluation
  Yeu-Shin Fu, Wenbo Ge, Jo Plested