ConceptGuard Tests Unlearning on Dual-Use Concepts and Finds Current Methods Cannot Separate Harmful From Benign Use

arXiv 2608.20338·medium signalXBlueskyLinkedInCopy link

Existing unlearning benchmarks use disjoint forget and retain sets of independent facts and score simple factual recall, which never tests the real requirement: removing a harmful application while keeping the same knowledge available for legitimate use. ConceptGuard introduces dual-use concepts, where forget and retain sets are explicitly complementary in concept usage rather than in subject matter, making evaluation intent-sensitive and measured at the concept level instead of over sparse facts. Current unlearning techniques perform poorly under this framing, showing weak contextual separation, strong forgetting-utility trade-offs and inconsistent concept-level control across methods; the dataset is publicly released.