Conformal Filtering on Inter-Agent Agreement Nearly Doubles Mixture-of-Agents Claim Precision From 0.41 to 0.75

arXiv·medium signalXBlueskyLinkedInCopy link

C-MoA turns semantic support between heterogeneous agents into a claim-level nonconformity score and calibrates a retention threshold, giving distribution-free factuality control that transferred across domains without recalibration. It failed on short-form answers, where consensus is cheap. The CONTRA-MoA extension, which adds falsification signals, halved false medical claims only when the verifier held domain knowledge. With a memory-only judge its signals were near chance (AUC 0.531 and 0.511), and naive max fusion lowered the working agreement signal from 0.687 to 0.652. Agreement is a usable filter, but moving past consensus requires a verifier that knows the domain.