Semantic Invariance Testing Across 7 Foundation Models: Reasoning Stability Under Equivalent Inputs Varies Widely — Metamorphic Testing as Agentic Reliability Audit

arXiv·medium signalXBlueskyLinkedInCopy link

Applies metamorphic testing — paraphrased but semantically equivalent prompts — across seven foundation models in agentic settings to measure reasoning stability. Results show substantial cross-model variation in invariance, meaning equivalent task descriptions trigger different reasoning paths and outcomes. Establishes metamorphic testing as a practical pre-deployment reliability audit for agentic systems, complementing benchmark performance with consistency measurement.