Hugging Face Adds 'Benchmaxxer Repellant' to Open ASR Leaderboard — Private Test Data to Prevent Gaming
Hugging Face Blog·medium signal
Hugging Face introduced private test data to the Open ASR (Automatic Speech Recognition) Leaderboard to prevent models from overfitting to public benchmarks. The 'benchmaxxer repellant' approach addresses the growing problem of models being tuned specifically to game public leaderboard metrics rather than delivering genuine performance improvements. This methodology could spread to other open benchmarks facing similar gaming.