Gemini 3.6 Flash Counts Only 0.2% of High-Frequency Video Events Correctly — And Adding Frames Inflates Scores Without Fixing Recall
Trace-grounded parametric profiling (arXiv 2608.06361, Aug 6) builds 2,190 controlled videos varying event count N and frequency F while holding rendering fixed, each with an executable event trace for timestamp-level scoring rather than final-answer-only grading. At an 80% reliability threshold Gemini 3.6 Flash reliably counts persistent state transitions up to 12 events at 0.5 and 1.0 Hz but has no reliable positive-count region for transient blinks; in the high-count high-frequency regime only 0.2% of final counts are correct and just 18.1% of true events are recovered. Raising sampling rate lifted Bounce Ball accuracy from 19.6% to 29.3% while the reported sequence matched ground truth only 3.7% of the time — extra frames buy score, not faithful perception.
↳ Follow the thread