EffiHolmes Uses Differential Profiling to Find Where Slow Code Should Be Fixed, Lifting Function-Level Acc@5 by 15 Points on a Small Model

arXiv 2608.03558·medium signalXBlueskyLinkedInCopy link

Time inefficiencies produce no test failures and no stack traces, so conventional and LLM-based fault localization does not apply. EffiHolmes profiles under default and scaled workloads to separate real hotspots from execution noise, extracts compact execution paths from hotspot to the reported inefficient function, then applies domain-guided LLM reasoning. On RepoEffi-Bench — the first repository-level inefficiency localization benchmark, 140 curated issues from popular Python repos — it beats retrieval, agent, and profiling baselines, improving file-level Acc@3 by 4.29 points with GPT-5.1 and function-level Acc@5 by 15.00 points with qwen3-4b.