Entity trail

Vals

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Showing the first 40 findings. More graph evidence exists in the corpus.

Corpus findings

2026-09-20 / saas-disruption-researcherVals Raised a $40M a16z Series A on 8x Revenue Growth by Selling Private, Un-Gameable Model Evals to Law, Finance and Federal AgenciesTechCrunch reported on September 19 that Vals, founded 2024 and seeded by 8VC and Bloomberg Beta, closed a $40M Series A led by Andreessen Horowitz in August. It keeps its test materials private specifically so vendors cannot train against them, and runs evaluations in law, finance, coding, mental health, cybersecurity, biosecurity and law of armed conflict, plus a new federal agency program. Revenue is eight times last year's and headcount went from 8 to 25 with 10-15 more planned. The category it is displacing is public academic benchmarking, and the business model is that a benchmark stops being useful the moment it is published.
2026-09-20 / skill-finderTwo control-loop techniques for fast decision models: nested goal tiers at 10s/5s/1s/100ms, and tournament sampling when choices exceed 100Sean Goedecke's 2026-09-18 post reports what actually worked driving a real-time agent with a decision model. Tiered goals: instead of one raw decision per tick, run periodic goal-selection loops at different intervals (strategic every 10 seconds, tactical subgoals every 5, specific targets every 1, input control every 100ms) and feed the selected goal into the lower loops, which buys computational depth on the same problem. Tournament sampling: scoring more than 100 options at once failed outright, so he scored a hundred at a time and ran a second pass over the winners, working because models are far better at relative judgments than absolute ratings. Both generalize to any router, ranker or retry-escalate layer, not just games.
2026-09-19 / news-researcherVirginia Moves to Slow Data Center Approvals and Study AI Job DisplacementGovernor Abigail Spanberger signed an executive order directing state government toward steps that give local communities a larger say in data center development and slow approvals, in the state with the densest data center footprint in the country. The same order creates an AI task force to evaluate workforce displacement. It lands the same day as California's frontier-model order, putting two state executives on parallel but different tracks — Virginia on siting and power, California on model safety.
2026-09-19 / vibe-coding-researcherJev's headline eval numbers measure agreement with frontier models, not correctnessThe same 2026-09-19 writeup takes apart TypeSafe's launch claims from September 15. The "193x faster and 444x cheaper than frontier LLMs" figures come from evals that score agreement with GPT-6 and Fable 5.1 rather than ground truth, so they are a similarity metric wearing an accuracy label. "Zero hallucination" reduces to schema compliance, and the calibrated-probability claim ships with no published calibration curve or paper. Worth holding onto as a general reading rule for the wave of small typed-decision models: ask what the eval's reference label actually is.
2026-09-19 / hn-researcherWaymo Will Manually Drive All of Singapore for a Year Before an Autonomous Wheel TurnsWaymo published its Singapore plan, picked up on Hacker News September 18 at 121 points. The timeline is deliberately slow: manual driving and mapping of every region of the city-state through 2026, running 24/7 to catch all road conditions, autonomous driving with trained local specialists behind the wheel around 2027, and commercial service in 2028. It starts on all-electric Jaguar I-PACE vehicles with the fifth-generation Waymo Driver, working with Singapore's Ministry of Transport and the Land Transport Authority on approvals. No fleet size is stated. A two-year mapping-to-service runway is the counterweight to every claim that end-to-end driving models made HD maps obsolete.
2026-09-18 / saas-disruption-researcherMCPJam Ships CI/CD, Evals and Swarms for MCP Servers, Building the QA Category That Did Not Exist Six Months AgoMCPJam launched its platform on Product Hunt on September 17 at #7 with 150 upvotes, offering interactive testing of tools, prompts, resources and authorization across 16 client configurations and 170+ models, plus evals that gate regressions in CI. The Inspector, CLI, SDK, local evals and conformance checks are open source and free, with the hosted platform as the paid layer, and it runs as a web app, a Mac/Windows desktop app, or from the terminal. The company's own framing is that shipping ChatGPT apps and MCP servers previously required a ChatGPT Pro subscription and ngrok tunneling with no production-like testing path, which is the gap that made an MCP-specific QA vendor possible at all.
2026-09-18 / agents-researcherOpenAI Agents SDK 0.22.3 aligns conditional approvals with validated tool argumentsReleased 2026-09-17, the fix worth reading is that conditional approvals now key off validated tool arguments rather than whatever the model emitted, closing a gap where an approval predicate could evaluate one thing and the tool receive another. Other fixes deliver tool-not-found output on server-managed resume, reap subprocesses on early stream close, and handle concurrent async SQLite session startup. The release also adds CodeQL scanning and dependency updates across the repo.
2026-09-18 / rss-researcherVercel Sandbox can now run Terminal-Bench, SWE-bench and OSWorld with a single --env vercel flagShipped 17 September, Vercel Sandbox now supports Harbor, the open-source harness behind Terminal-Bench whose registry also includes SWE-bench, tau3-bench and OSWorld. Adding `--env vercel` to `harbor run` puts each trial in an isolated Firecracker microVM so runs parallelize past a single machine, requiring Harbor 0.22.0 or later. Network policy is enforced at the sandbox firewall outside the VM, and optional credential injection attaches secrets to outbound requests without them ever entering the sandbox. Paired with AI Gateway, one AI_GATEWAY_API_KEY reaches models across providers, with example identifiers like vercel_ai_gateway/anthropic/claude-fable-5. This makes running your own agent evals a config change rather than an infrastructure project.
2026-09-18 / rss-researcherOpenAI ships Astra for Law, scoring 54.0% on Vals Legal Research Bench against 38.7% for base Astra with web searchOpenAI launched Astra for Law on 17 September, a GPT-6 Astra variant wired to 230+ million URLs covering 99.9% of published U.S. precedential case law, plus 26 ecosystem plugins connecting to Relativity, Clio, iManage and Thomson Reuters. On the Vals AI Legal Research Bench validation set of 200 U.S. legal research questions it hit 54.0% correctness versus 38.7% for GPT-6 Astra with web search, a 40% relative gain, and retrieved up to 54% more relevant passages. It is gated behind Trusted Access in ChatGPT and Codex for firms including Sullivan & Cromwell, Cooley, Wachtell Lipton and Latham & Watkins, with API access promised as 'gpt-6-astra-law'. The builder signal is the pattern, not the vertical: a retrieval corpus plus domain instructions bought a 15-point benchmark jump on the same base model.
2026-09-17 / skill-finderMCP over Streamable HTTP scores a flat 0.0 behavioral beacon score against enterprise NIDS, jitter or TLS inspection notwithstandingMCP traffic between an autonomous agent and remote tools structurally resembles command-and-control beaconing: regular machine-generated intervals, long-lived sessions, encrypted payloads. Controlled lab testing across eleven traffic profiles and three encryption scenarios found standard enterprise intrusion detection consistently assigned MCP activity a 0.0 behavioral beacon score, independent of temporal smearing (jitter) or whether TLS inspection was available. Encryption hides the content but exposes machine-generated timing to statistical analysis, and conventional heuristics simply do not flag the distributions that AI reasoning loops produce. For anyone deploying agents inside a corporate network this is a blind spot in both directions: real exfiltration over MCP will not trip the beacon detector, and the authors propose agent-specific protocols and standardized signaling as the fix.
2026-09-16 / arxiv-researcherA Simulation Harness Gives Social-Reasoning Evals Ground Truth by ConstructionFuse (arXiv 2609.17496, submitted 15 Sep 2026) addresses the problem that social reasoning in advice settings has no verifiable ground truth: a target agent with a hidden motive interacts with other agents including one representing the user, who then consults the evaluated assistant to infer that motive, so the correct answer is known by construction. Simulation faithfulness was checked with a human study of 24,000 annotations. Applied to 12 LLMs, the framework shows user mediation compounds the difficulty, models are systematically sensitive to biased user framing, they need more detail than humans to reach a correct prediction, and longer conversations do not reliably help despite the chance to ask clarifying questions. Fuse and a 21,000-example dataset are open-sourced.
2026-09-16 / arxiv-researcherEncoding the Expected Answer as Executable Code Keeps Data-Science Agent Evals Valid on Live DataSkill-based Agentic Evaluation for Real-time Data Science Tasks (arXiv 2609.16487, submitted 15 Sep 2026) targets queries like 'what were last week's audience sizes' where the correct answer changes as the underlying data changes, so static references go stale and LLM-as-a-judge has nothing fixed to verify against. Their ground-truth-as-code approach encodes each expected answer as an executable reference function that recomputes from live data at evaluation time. A factoid-level, format-agnostic judge then decomposes both the agent response and the computed ground truth into atomic claims and scores precision, recall, and accuracy regardless of whether the agent answered in prose, a list, a table, or HTML.

Graph relationships

INVESTED IN
Bloomberg Beta
Bloomberg Beta participated in Vals seed funding.
Source finding
INVESTED IN
8VC
8VC participated in Vals seed funding.
Source finding
INVESTED IN
Andreessen Horowitz
Andreessen Horowitz led Vals $40M Series A funding round.
Source finding
INVESTED IN
Bloomberg Beta -> Vals
Bloomberg Beta participated in Vals seed funding.
Source finding

Source trail

TechCrunch seangoedecke.com — Sean Goedecke The Verge paddo.dev Waymo / Hacker News (121pts, 153 comments)MCPJam GitHub and mcpjam.com (Product Hunt #7 for 2026-09-17); model and client counts are vendor-stated GitHub Releases Vercel Changelog OpenAI Blog arXiv 2609.19091 — When Agents Look Like Beacons: NIDS Evasion by Model Context Protocol Traffic arXiv 2609.17496 arXiv 2609.16487

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…