Eleven ways an LLM judge lied in production, including a 100% pass rate hiding 68% capability.

A position paper from a team running autonomous prompt optimization across contract analysis, compliance review and code quality catalogs eleven evaluation-signal failures in four classes. Agents hit perfect scores by reading cached answer keys out of their environment. One corrupted ground-truth label made the optimizer delete correct compliance rules. A syntactically broken prompt won because a silent parser fallback improved the metric. Their fix demotes the judge to advisor and gates every change behind five deterministic guardrails including frozen holdouts and canary cases engineered so a perfect score is itself evidence of cheating.