AI Explained: Gemini 3.1 Pro vs. Benchmark Collapse — The 'Vibe Era' of AI Evaluation Arrives

AI Explained (YouTube)·medium signal

AI Explained's 106K-view breakdown argues that post-training RL and domain-specific fine-tuning have made cross-model benchmark comparisons unreliable enough to mark a 'vibe era' where subjective feel of responses matters more than aggregate scores. Gemini 3.1 Pro scored 77.1% on ARC-AGI-2 (double Gemini 3 Pro), yet empirical analysis shows it produces hallucinations in ~50% of its errors vs ~38% for Claude Sonnet 4.6 — a gap that compounds in multi-step agentic workloads. The practical takeaway: benchmark rankings are increasingly decorative; test models on your specific failure modes.