Zvi's worry about Astra isn't the benchmark scores, it's that they hold with reasoning off.

His September 12 post calls Astra a bigger jump than Fable 5 to 5.1, citing 98% on FrontierMath Tier 4, 98.1% on extended NYT Connections against Fable 5.1's 90%, the first autonomous Montezuma's Revenge clear and a one-shot Portal completion. The monitorability problem is Neel Nanda's finding that Astra scores 159 on Epoch ECI with no chain-of-thought, four points off Fable 5.1's full-reasoning 163 and well above Fable's ~128 no-CoT score. Zvi calls the 100% ExploitBench figure a chart crime likely reflecting contamination. His practical advice is to run both Fable 5.1 and Astra on hard questions instead of switching, which matches Real-SWE's ordering where the gap is 5 points.