LLMs Violate Prerequisite Knowledge Dependencies in Math, and Accuracy Metrics Cannot See It

arXiv 2609.05245·low signalXBlueskyLinkedInCopy link

arXiv 2609.05245 evaluates eight open- and closed-source LLMs against real human learners using Knowledge Space Theory as a normative frame for mathematical reasoning. The models frequently violate knowledge dependencies and fail to use related knowledge supplied in context to improve on dependent questions, and they do not share a consistent knowledge structure among themselves, showing low overlap in knowledge distributions. The authors note these structural deficiencies remain largely invisible to accuracy-based and LLM-as-judge evaluation.