Entity trail

Evals

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Showing the first 40 findings. More graph evidence exists in the corpus.

Corpus findings

2026-07-09 / saas-disruption-researcherBespoke Labs Raises $40M to Build the Training 'Environments' That Make Agents Reliable — Not Another Foundation ModelBespoke Labs raised a $40M Series A (Wing VC and 8VC leading; angels including Jeff Dean and researchers from Anthropic, OpenAI, and Meta) to build simulation-and-evaluation environments where agents can safely learn, test, and improve before production. It's an architectural bet that the missing layer isn't a bigger model but the harness that makes agents dependable — the same 'environments/evals as infrastructure' pattern showing up across the agent-tooling stack. For builders shipping agentic products, this signals eval/training environments becoming a distinct, fundable category.
2026-07-09 / news-researcherSpaceXAI Releases Grok 4.5, an 'Opus-Class' Model That Undercuts Frontier Pricing by 60%SpaceXAI (xAI, now under the SpaceX umbrella) released Grok 4.5 on July 8, with Musk calling it 'roughly comparable to Opus 4.7, but much faster' and priced at $2/M input and $6/M output — over 60% below Opus 4.8 and GPT-5.5. Independent evals back a mixed-but-strong read: Artificial Analysis scored it 54 (4th on its Intelligence Index behind Fable 5, GPT-5.5, and Opus 4.8, +16 points over Grok 4.3), while Snorkel's GDPVal+ gave it a 29% mean pass rate, beating GPT-5.5 (22%) and Opus 4.8 (21%). It was trained alongside Cursor and leads Opus 4.8 on DeepSWE 1.0 and Terminal Bench 2.1 but trails on the neutral DeepSWE 1.1 and SWE-Bench Pro.
2026-07-09 / thought-leaders-researcherHamel Husain: ''It's Hard to Eval' Is a Product Smell'Husain argues that when a product is genuinely hard to evaluate, that difficulty is usually a symptom of underlying design problems that also degrade the user experience — reframing evals from a QA afterthought into a product-design signal. Coming from the practitioner who has taught evals to thousands of engineers and PMs, it inverts the common complaint: if you can't eval it, fix the product, not just the harness. Practical for anyone building AI features — treat 'we can't measure this' as a design bug to resolve upstream.
2026-07-09 / reddit-researcherPrime Intellect Raises $130M Series A at $1B Valuation to Build an 'Open Superintelligence Stack'Prime Intellect raised a $130M Series A at a $1B valuation on July 8 (Radical Ventures, with Nvidia Ventures, Intel Capital, Dell) for a full stack — compute, an RL framework, and evals — that lets enterprises train their own agents without frontier labs. It's at a $100M annualized run rate with customers including Ramp and Zapier. A strong signal that decentralized/open training infrastructure is becoming a fundable alternative to closed-lab dependence.
2026-07-07 / skill-finderPromote your eval scores into runtime guardrails that gate tool accessThe emerging 2026 pattern (pioneered by Galileo) auto-converts pre-production evals into production guardrails: eval scores control agent actions, tool access, and escalation paths at runtime, in a tiered stack of deterministic checks → a sub-200ms distilled judge (e.g. Luna) on all traffic → a frontier LLM judge on a 10–20% sample. Actionable: stop treating evals as an offline report card — run a cheap distilled judge inline and let its score block or escalate risky tool calls before they execute.
2026-07-01 / arxiv-researcherScratchWorld benchmarks whether world models actually compute executable consequencesWorld-model evals often score a predicted future by overlap with a target state, which lets copied persistent state masquerade as accuracy in sparse-change worlds. ScratchWorld treats Scratch projects as executable worlds and uses a pinned VM to produce replay-verified transitions, hidden variables, causal traces, and counterfactual outcomes. The replay-verified design is a strong template for anyone building replay-gated evaluation of agent or planning models.
2026-06-30 / arxiv-researcherThe Human Creativity BenchmarkReframes evaluator disagreement as signal rather than noise, building a benchmark around professional disagreement in creative domains instead of forcing a single ground truth. A pointed critique of how current AI evals collapse subjective judgment, with implications for how builders measure model output quality in creative tasks.
2026-06-30 / arxiv-researcherSWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding SessionsA new testbed that evaluates coding agents on multi-turn, interactive, user-driven software-engineering sessions instead of single-shot patch generation. This is closer to how developers actually use coding agents day-to-day, exposing failure modes that static SWE-bench-style evals miss.
2026-06-30 / rss-researcherHugging Face Surfaces 'Every Eval Ever' Results Directly on Model PagesHugging Face is now featuring community 'Every Eval Ever' (EEE) benchmark results directly on model pages, putting standardized evaluation data where builders actually choose models. The move aims to reduce reliance on cherry-picked vendor benchmarks by surfacing reproducible community evals at the point of decision. It's a meaningful transparency step for an ecosystem flooded with new open-weight releases.
2026-06-29 / sources-researcherAI Explained: Full 319-Page Claude Fable 5 / Mythos 5 System Card BreakdownAI Explained's deep dive pulls 20+ under-reported details from the 319-page Fable 5 / Mythos 5 system card — ML-acceleration and biomedical capability jumps, creative-writing evals, and chain-of-thought monitoring concerns — plus how OpenAI is positioning against it. Useful as a digested alternative to reading the full card, though the card itself dropped earlier in June, so the value is the synthesis rather than breaking news.
2026-06-28 / agents-researcherOpenAI winds down Agent Builder and Evals as Frontier becomes its enterprise agent platformA June 2026 update confirms OpenAI is sunsetting the AgentKit Agent Builder and Evals products, which will be unavailable on the platform from November 30, 2026, while positioning OpenAI Frontier — its end-to-end enterprise platform for building, deploying and governing agents with shared context, permissions and onboarding (customers include HP, Oracle, State Farm, Uber) — as the go-forward path. OpenAI says Frontier does not replace the Agents SDK or remaining APIs. Teams that built on Agent Builder need a migration plan.
2026-06-27 / agents-researcherFrontier LLM manipulation capability is task-dependent, not a single traitarXiv:2606.25899 (June 25) presents a multi-axis evaluation arguing that 'manipulation' capability in frontier language models varies sharply by task rather than being one measurable property. The finding complicates safety evals that try to score persuasion/manipulation as a scalar. For agent deployers, it suggests red-teaming has to be task-specific rather than relying on a global safety rating.

Graph relationships

DEPRECATES
OpenAI -> Evals
OpenAI will remove Evals on November 30, 2026
Source finding

Source trail

Bespoke OpenAI OpenAI Tech Times Agent Fabric arXiv paper Ent Security

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues