Hacker News
Microsoft Research: LLMs Corrupt 25% of Document Content in Long Delegation Workflows — DELEGATE-52 Benchmark — 451 Points on HN
Microsoft Research published DELEGATE-52, a benchmark simulating long delegated workflows across 52 professional domains (coding, crystallography, music notation), revealing that even frontier models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) corrupt an average of 25% of document content by the end of long workflows. The errors are sparse but severe — silently compounding over time — and agentic tool use does not improve performance. Degradation worsens with document size, interaction length, and presence of distractor files, directly challenging the 'vibe coding' and autonomous delegation paradigm.
↳ Follow the thread