A Bid-Writing Startup Found Chunked Extraction Silently Dropped Half Its Requirements — 366 Down to 184, With No Error
Lucius AI published a post-mortem on making an AI bid writer refuse to fabricate, and the failure modes are more useful than the fixes. Chunked requirement extraction silently collapsed from 366 to 184 requirements because empty chunk results unioned quietly into totals; the citation checker flagged 90 of 158 citations as unverified purely from whitespace and hyphenation mismatches; and the model invented consortium partners with bracketed `[PARTNER_NAME]` placeholders that downstream verifiers then credited as real bidder evidence. Their architecture answers with page-cited verbatim extraction machine-verified against source PDF text, a second coverage pass so 'completed' is earned rather than assumed, capability-fit gating before drafting, and outright refusal to draft when the system's own bid verdict says don't bid.
↳ Follow the thread