Eamonn Keogh shows 100-year-old statistical process control beats SOTA on the TSB-AD-M anomaly detection benchmark

r/MachineLearning·high signalXBlueskyLinkedInCopy link

Keogh posted to r/MachineLearning (369 upvotes) that on most of the TSB-AD-M benchmark datasets used across NeurIPS, SIGKDD and VLDB time-series anomaly detection papers, plain Statistical Process Control matches or beats the published state of the art, scoring perfect results on the ECG trace he shows and doing so trivially on the "TAO" traces. His argument is not that the proposed algorithms are wrong but that the benchmark is too easy to support the claims built on it; he notes one of the datasets is a classification problem solved 27 years ago that was converted to a TSAD task without introspection. He says he has done 90% of the work on harder replacement datasets (sled dogs, tuna, fuel cells, smart manufacturing).