Concept Activation Vectors Used to Audit Automated L2 Speaking Tests for Dependence on Speaker Attributes

arXiv 2608.06300·low signalXBlueskyLinkedInCopy link

This paper (2608.06300, submitted 2026-08-06) applies concept activation vectors to automatic speaking assessment systems, which are increasingly deployed in high-stakes second-language testing where scores must reflect speaking proficiency rather than irrelevant speaker attributes such as first language. The technique probes whether internal representations encode those attributes, offering a bias audit that operates on the model's learned concepts rather than only on output score distributions. The method is the transferable part for builders — CAV-based auditing generalizes to any classifier where you need to show a protected attribute is not driving the decision.