A pre-registered study found that a 267-word specification preamble cut defects in generated backend code for all five frontier models, and the frame arm won 95 of 100 contested comparisons
arXiv 2609.23270 (20 Sep) ran 50 finance, healthcare and insurance backend tasks through five models from five vendors. Each task ran once bare and once with a fixed specification frame stating what must be true of the result for money arithmetic, time handling, idempotency and access control. Nine AST checkers and an independent Bandit scan both favored the frame, with Bandit medium/high issues falling from 53 to 11, and the frame never made a model worse in any domain. The author contrasts this with a large study where general instruction files showed no benefit. The practical takeaway is to prepend a short, domain-specific 'must be true' spec to generation prompts rather than growing a general rules file. All prompts and checkers are published.
↳ Follow the thread