Proceedings of the Recent Advances in Artificial Intelligence for Sustainable Development (RAISD 2025)

Beat the Machine: A Gamified Approach to Identifying Systematic Errors in Predictive Models

Authors
Akash Kota Raju1, *
1Independent Researcher, New York University, New York, USA
*Corresponding author. Email: ak9381@nyu.edu
Corresponding Author
Akash Kota Raju
Available Online 17 July 2025.
DOI
10.2991/978-94-6463-787-8_54How to use a DOI?
Keywords
machine learning evaluation; model robustness; unknown unknowns; gamified testing; human-in-the-loop; adversarial examples
Abstract

In predictive modeling, systematic errors—often referred to as unknown unknowns—can lead to critical failures, especially when models are overconfident in their incorrect predictions. This paper presents a gamified framework, Beat the Machine (BTM), which incentivizes the identification of such systematic misclassifications. BTM rewards participants based on error severity, emphasizing high-confidence failures that conventional quality assurance methods tend to overlook. The pro- posed approach is benchmarked against the widely used Stratified Random Sampling method (for a detailed discussion, see Cochran’s Sampling Techniques [4]) and demonstrates superior performance in uncovering both the frequency and severity of misclassifications, as supported by im- proved AUC metrics [6]. The results underscore the potential of gamified error detection to enhance model robustness and offer a new paradigm for systematic quality assurance in machine learning applications.

Copyright
© 2025 The Author(s)
Open Access
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

Volume Title
Proceedings of the Recent Advances in Artificial Intelligence for Sustainable Development (RAISD 2025)
Series
Advances in Intelligent Systems Research
Publication Date
17 July 2025
ISBN
978-94-6463-787-8
ISSN
1951-6851
DOI
10.2991/978-94-6463-787-8_54How to use a DOI?
Copyright
© 2025 The Author(s)
Open Access
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Cite this article

TY  - CONF
AU  - Akash Kota Raju
PY  - 2025
DA  - 2025/07/17
TI  - Beat the Machine: A Gamified Approach to Identifying Systematic Errors in Predictive Models
BT  - Proceedings of the Recent Advances in Artificial Intelligence for Sustainable Development (RAISD 2025)
PB  - Atlantis Press
SP  - 721
EP  - 735
SN  - 1951-6851
UR  - https://doi.org/10.2991/978-94-6463-787-8_54
DO  - 10.2991/978-94-6463-787-8_54
ID  - Raju2025
ER  -