RefactorPlatform: AST-aware chunking beats token-window chunking by 25-30%, and naive retrieval is worse than no retrieval

arXiv·high signalXBlueskyLinkedInCopy link

arXiv 2609.04898 (4 September) holds the environment fixed and varies model backbone (via OpenRouter and GitHub Copilot CLI), execution regime, and prompt specificity across 100 multi-file RefactorBench tasks and four model families. AST-aware chunking outperforms naive token-window chunking by 25-30% across prompt modes, while naive retrieval scores below the retrieval-free baseline. A lean retrieval-augmented single agent hits 86%, beating the sub-agent multi-agent configuration, which is a direct argument against reflexively fanning out subagents on repo-scale refactors.