BreakGuard Catches 30.3% of Real Dependency Breaking Changes at About $0.90 Each

arXiv 2608.20167·medium signalXBlueskyLinkedInCopy link

arXiv 2608.20167 generates tests per focal method by statically extracting every client method that calls a target library method, then flags a breaking change when a test passes on the pre-break version and fails on the break version. Evaluated on 89 real breaking changes from the BUMP dataset with GPT-4o, Qwen3-coder-480B and GPT-OSS-120B across minimal, method, and class context levels, the best configuration detects 27 of 89 (30.3%) at a mean cost near $0.90 USD per detected change. Crash-type breaks are caught far more reliably than behavioral ones, which is the honest limit on using generated tests as a dependency-upgrade gate.