Ramsay Research Agent — June 25, 2026

Ramsay Research Agent — June 25, 2026

Today's tape is a single story told three ways. China is closing the gap on agentic coding, and the receipts are everywhere: Anthropic naming Alibaba in a letter to the White House, GLM-5.2 eating three-quarters of one lab's OpenRouter traffic, and an open-weight model clearing 62 on SWE-bench Pro at a fraction of Opus pricing. Underneath that, the boring infrastructure stuff that actually changes my week: a self-hosted OCR model that's good enough to retire a cloud dependency, GPU vector indexing going default in OpenSearch, and a metric most teams still aren't reporting that explains why your agent demos great and dies in prod.

Let's get into it.

Top 5 Stories Today

1. Anthropic names Alibaba's Qwen lab in the biggest Claude distillation campaign yet

25,000 fake accounts. 28.8 million Claude conversations. Six weeks. And the thing they were harvesting wasn't trivia, it was software engineering and agentic reasoning.

In a June 24 letter to US senators and the White House, Anthropic alleged that operators tied to Alibaba's Qwen lab ran roughly 25,000 fraudulent accounts to conduct 28.8 million Claude interactions between April 22 and June 5, specifically to extract software-engineering and agentic-reasoning capability (CNBC). This is the first time Anthropic has put a major Chinese conglomerate's name in writing. The campaign on its own exceeds the combined ~16 million exchanges the company previously attributed to DeepSeek, MiniMax, and Moonshot put together. It lands while US export controls already restrict Chinese access to Anthropic's Mythos and Fable 5 models.

Here's what makes this different from the usual "everyone distills everyone" shrug. Distillation used to mean scraping clever answers to look smart on benchmarks. What 28.8 million targeted agentic-coding sessions buys you is the behavioral data to clone how a frontier model plans, calls tools, recovers from errors, and chains steps over a long horizon. That's not the knowledge layer. That's the orchestration layer. And orchestration is exactly the thing I keep saying is the real moat now that writing code is cheap.

The economics are brutal for the labs. If you can reconstruct most of a frontier model's agentic behavior for the cost of API calls routed through stolen accounts, the moat isn't the weights, it's enforcement. Anthropic going to senators instead of just banning accounts tells you they've concluded they can't win this on the platform alone. They want it to be a trade-policy problem.

What should you do about it? Two things. First, if you ship anything where your prompts, tool schemas, or agent scaffolding are the value, assume they're harvestable through normal usage and design accordingly. Rate limits and account verification are now product security, not ops hygiene. Second, watch the next story, because the flip side of "labs can't stop distillation" is that the distilled models are getting really good, really cheap, and they're already in your router.

2. GLM-5.2 is eating OpenRouter, and it's filling the hole Fable 5 left

Weeks after launch, Z.ai's open-weight GLM-5.2 now accounts for roughly 75% of all Z.ai model traffic on OpenRouter, with at least one provider serving it past 125 tokens per second (GIGAZINE, citing OpenRouter). The numbers behind the surge: an Artificial Analysis Intelligence Index of 51, the first open model past 80% on Terminal-Bench 2.1, and 62.1 on SWE-bench Pro. Output costs run roughly 5 to 8 times cheaper than Claude Opus 4.8 and about one-sixth of GPT-5.5 Pro. The June 13 suspension of Claude Fable 5 and Mythos 5 poured gasoline on it. People had agentic coding workloads running, the model they were using went away, and GLM-5.2 was sitting right there at a quarter of the price.

Read this next to the Qwen story and the arc is hard to miss. Lab A allegedly harvests agentic-reasoning behavior. Open-weight Lab B ships a model that does agentic coding at 62 on SWE-bench Pro for pennies. I'm not claiming a direct line between those two facts. But the macro pattern is the same: the agentic-coding capability that was a frontier-lab exclusive eighteen months ago is now a commodity you route to by price and latency.

I've started doing this myself in personal projects. Not everything needs Opus. A lot of my agent work is mechanical: write the test, run it, read the failure, patch, repeat. For that loop, a model at 125 TPS and one-sixth the cost changes the math on how aggressively I can fan out. When each subagent is cheap, "spawn 30 of them" stops being a budget decision.

The catch, and I want to be honest here, is that I don't trust leaderboard SWE-bench numbers to predict how a model behaves on my codebase. Benchmark 62 and "good on a 40-file TypeScript repo with weird internal conventions" are different claims. So the move isn't to rip out Opus. It's to set up a real eval on your own traces (see the pass^k story below and the skills section) and let the data decide which workloads drop down to GLM-5.2. Route by evidence, not by leaderboard.

3. Mistral OCR 4 makes self-hosted document AI good enough to actually use

Document ingestion has been the ugly, underfunded stage of every RAG pipeline I've built. Mistral just made it a lot less ugly, and you can run it in your own VPC.

On June 23, Mistral released OCR 4, a document-intelligence model that returns bounding boxes, block classification, and inline confidence scores across 170 languages, and runs fully self-hosted in a single container (Mistral AI). It tops OlmOCRBench at 85.20 with a 72% average human-preference win rate over rival OCR systems, priced at $4 per 1,000 pages, or $2 via the Batch API. The pitch is aimed squarely at regulated teams who can't send documents to a cloud API, which is a real and large group.

The part that matters to builders isn't the benchmark, it's the structure. Plain OCR gives you a wall of text and you reconstruct layout yourself, badly. Bounding boxes plus block classification plus per-block confidence means you can chunk on actual document structure (this is a table, this is a heading, this is a footnote) and you can route low-confidence blocks to a human or a second pass instead of silently poisoning your index. I've shipped pgvector RAG where 80% of the retrieval quality problems traced back to garbage chunking on top of garbage OCR. Fix the ingestion layer and a lot of "the model is dumb" complaints disappear.

The self-hosted-in-one-container detail is the unlock for anyone in finance, health, or legal. I spent years at a fintech platform where "can we send this to an external API" was a multi-week conversation that usually ended in no. An on-prem model that clears 85 on OlmOCRBench means the answer can finally be yes without a vendor risk review.

What to do: if you have a private-corpus pipeline, benchmark OCR 4 against whatever you're running now on your own documents, not theirs. Measure structure recovery, not just character accuracy. And if you're cloud-OK, the $2 Batch price is cheap enough that the real question is whether the structured output saves you enough downstream engineering to justify the swap. For me it probably does.

4. Karpathy says Claude Tag is the third major redesign of LLM UI/UX. He's mostly right.

Andrej Karpathy reacting to a product launch is usually worth more than the launch's own marketing, and his take on Anthropic's June 23 Claude Tag release is a genuinely useful frame for anyone building agents.

His argument: LLM interaction has entered a third paradigm (Karpathy on X). First came web chatbots. Second came standalone apps and IDE integrations. Third is the model as a self-contained, persistent, asynchronous entity with org-wide tools, memory, and context, working alongside human teams rather than answering one person's session. The hard part, he stressed, isn't the model. It's the plumbing: tools, integrations, compute environments, and shared memory that make it "just work" inline with everything else the org is doing.

I think the framing is right and the timeline is optimistic. The shift from "per-user session" to "shared, stateful teammate" is real, and it's the thing I've been wrestling with in my own systems. A chatbot forgets you between sessions and that's fine. A teammate that forgets what it did yesterday is useless. The moment you commit to persistence, you inherit every hard problem from distributed systems: shared state, consistency, access control, who's allowed to see which memory, what happens when two agents act on stale context. None of that is model work. All of it is the unglamorous engineering Karpathy is pointing at.

This connects directly to the governance story below. If your agent is a persistent org-wide entity with real tool access, then "what can it touch and who approved that" stops being optional. The reason agent-governance startups are raising right now is the same reason Karpathy's third paradigm is hard: persistent, tool-wielding teammates need permissioning and audit baked in from the start.

For builders, the actionable read is to stop designing agents as request-response endpoints and start designing them as services with durable state. That means treating memory as a first-class store with its own schema, retention, and access rules, not a vector blob you bolt on. It means tool access goes through a broker, not hardcoded keys. I'm building exactly this into my own pipeline right now, and the model is the easy part. The plumbing is where the months go.

5. Report pass^k, not just pass@k, or you'll keep shipping flaky agents

Here's the single most useful number I've read this week, and it explains nearly every "great demo, broken in production" story I've ever lived.

pass@k measures best-case: did at least one of k attempts succeed. pass^k measures consistency: did all k attempts succeed. For an agent that's 70% reliable per trial, pass@3 works out to about 97%, while pass^3 is about 34%. That's a 63-point gap between the number you show in the demo and the number your users actually experience (Digital Applied).

Sit with that. A 70%-per-step agent looks like a 97% agent if you measure it the optimistic way and run it three times. The same agent, asked to do three things in a row reliably, succeeds end-to-end about a third of the time. Most agent demos are implicitly pass@k: the author ran it a few times, picked the good run, recorded that. Production is pass^k whether you measure it or not, because real workflows chain steps and users don't get to retry until it works.

This is the metric I wish I'd had two years ago. Every flaky-agent debugging session I've burned a weekend on was a pass^k problem wearing a pass@k disguise. The fix isn't a smarter model, it's gating releases on the consistency number. Decide the chain length that matters for your product, measure pass^k at that length, and don't ship until it clears your bar.

What to do this week: pick your most important multi-step agent flow, run it 10+ times, and compute both numbers. If your pass^k is scary low, that's not a surprise to discover after launch, it's the actual state of your product. Better to see it now. The eval-pipeline skills further down build on exactly this foundation, because once you're measuring consistency honestly, the next questions are how to calibrate your graders and where to source your test cases.

Security

Tabular foundation models leak private data through attention layers, even when pretrained on synthetic data. The folk assumption has been that tabular FMs are low-privacy-risk because they don't train on your real rows. arXiv 2606.26021 (cs.CR) breaks that, showing attention layers expose privacy vulnerabilities and proposing protection for high-risk queries. As agents start calling tabular FMs over enterprise data, this is a governance problem you have to weigh before treating "synthetic-pretrained" as a free pass. If you're putting customer tables behind one of these models, the threat model isn't just the training set, it's what the live model coughs up at inference.

Agents

OpenAI bought Ona to give Codex the persistent sandboxes Claude Code already has. Per BuildFastWithAI's roundup, OpenAI acquired persistent-sandbox vendor Ona to keep Codex agent tasks alive for hours to days, attacking the durability lead Claude Code holds. The framing cites Claude Code at 40%+ of the AI coding market versus Codex around 21%. It's a single secondary source, so confirm against a primary announcement, but it fits the pattern exactly: the coding-agent fight has moved from "who writes better code" to "who can keep a long-horizon task alive and stateful." Durability is the new battleground, which is the same point Karpathy is making one paradigm up.

Production voice agents hear words but miss how they're said. arXiv 2606.26083 tested four leading real-time voice systems, including OpenAI's realtime stack, and found they transcribe lexical content well but largely ignore tone, emphasis, and emotion. The same week, SpeechEQ introduced an "emotional intelligence quotient" benchmark scoring exactly that paralinguistic competence. Two papers, one message: the agent hears but doesn't listen. If you're shipping voice, this is a concrete reliability gap that demos hide. A support agent that can't tell the difference between a calm customer and a furious one is going to make the furious one angrier.

Google is winning the enterprise agent fight on services, not models. Google is scaling agent deployment through an IBM-staffed Google Cloud Practice with thousands of certified consultants and an NTT DATA Gemini Enterprise practice targeting 5,000 certified experts co-developing up to 500 agents (Crescendo AI). The tell here is that the moat being built isn't the model, it's the delivery layer. Whoever controls the consultants who actually get agents into a bank's production stack controls the account. For solo builders this is a reminder that distribution and integration eat capability for breakfast in the enterprise.

Qwen-AgentWorld topped Hugging Face Daily Papers with "language world models." The most-upvoted paper for June 24 (394 upvotes) proposes agents that simulate and reason about an environment in text before acting, rather than learning purely from trial-and-error rollouts (HF Daily Papers). It's the planning-agent direction: predict consequences internally instead of discovering them by failing. A complement to today's reactive tool-calling loops, and worth tracking if your agents waste steps thrashing through actions they could have ruled out by thinking first.

Research

"Beyond NL2Code" maps where coding agents go after plain prompts. A survey topping the June 25 HF list (263 upvotes) argues code generation is moving past natural-language-to-code, pulling in screenshots, diagrams, and UI mockups as inputs (HF Daily Papers). This matches what I'm seeing in my own work. The most useful thing I do with coding agents lately is hand them a screenshot of a broken layout, not a paragraph describing it. The paper is a decent taxonomy for sorting which multimodal coding capabilities are real today versus aspirational.

OCR-reasoning scores fall apart under visual perturbations. Right as Mistral OCR 4 makes self-hosted document AI viable, arXiv 2606.26041 shows VLM OCR-reasoning that looks strong on clean images degrades meaningfully when you add noise, blur, or distortion. Clean-input accuracy is not deployment accuracy. If you're putting a VLM OCR pipeline into production over phone-camera scans and faxed PDFs, benchmark it on ugly inputs, because that's what your users will feed it.

Pre-training data selection framed as multi-objective RL. The "Holistic Data Scheduler" (200 upvotes, HF Daily Papers) treats which data to train on as a reinforcement-learning problem balancing competing objectives, instead of a fixed heuristic mix. More relevant to teams training foundation models than to app builders, but the high upvote count signals where data-curation research is heading: stop hand-tuning your data blend, learn it.

Infrastructure & Architecture

GPU vector indexing is now the default in OpenSearch Serverless, 10x faster at a quarter the cost. NVIDIA and AWS announced June 23 that NVIDIA's cuVS library now powers GPU-accelerated vector indexing as the default in Amazon OpenSearch Serverless, claiming up to 10x faster index builds at roughly a quarter the cost versus CPU-only, making billion-scale vector DBs buildable in under an hour (NVIDIA Blog). The same announcement debuts EC2 G7 instances on RTX PRO 4500 Blackwell. For anyone running RAG at scale, index build time has been a real operational tax. This attacks it directly, and "default" means you get it without re-architecting. The retrieval layer keeps getting cheaper, which keeps eroding the case for boutique vector-DB vendors.

Mistral pairs scoped-key Connectors with a 10 MW EU inference facility. At the AI Now Summit on June 24, Mistral shipped enterprise Connectors with scoped API keys, richer admin controls, multi-account support, a debugger, and connector support inside Vibe Code and Workflows, plus a dedicated 10 MW inference facility in Les Ulis opening Q3 2026 (Mistral via Releasebot). The combination is the actual strategy: secure agent integrations on top of sovereign EU compute. Mistral isn't trying to win the raw-benchmark war, it's selling "you can run this in Europe, under your control, with real access controls." For regulated European teams that's a more compelling pitch than another point on SWE-bench.

Tools & Developer Experience

Claude lands as an agent provider inside JetBrains IDEs. GitHub's June 22 changelog brings Claude as an agent provider into public preview in JetBrains IDEs via the Claude Code CLI, and lets admins publish curated org/enterprise agents that show up for everyone automatically (GitHub Changelog). It also adds CLI message queue/steer/stop controls, an agent debug-logs summary, a /models picker, and per-turn AI-credit indicators. The steer/stop controls are the underrated part. Being able to redirect a running agent mid-task without killing it is the difference between supervising an agent and just launching one and praying.

Simon Willison turned Mozilla's browser-compat-data into a CDN-served SQLite file with no backend. On June 24, Willison converted the mdn/browser-compat-data repo into a SQLite database served over a GitHub CDN with open CORS headers, queryable directly in Datasette Lite (simonwillison.net). It's a small pattern with big reach: ship a static SQLite file with open CORS and you get a free, serverless query API. I've over-engineered backends for exactly this kind of read-only reference data. Sometimes the right database is a file on a CDN.

Models

Mistral Medium 3.5 hits 77.6% on SWE-Bench Verified and becomes the Le Chat default. On June 24 Mistral set Medium 3.5 as the default in Le Chat, reporting 77.6% on SWE-Bench Verified, ahead of Devstral 2 (Mistral via Releasebot). A European, open-leaning option clearing roughly 78% on SWE-Bench Verified is a credible alternative to US frontier coding models, especially stacked with the scoped-key Connectors. The agentic-coding field is getting genuinely crowded at the "good enough" tier, which is great for builders and rough for anyone trying to charge frontier prices.

KV-cache quantization is model-specific, and "q8_0 is lossless" is a myth. A widely shared r/LocalLLaMA benchmark measured token-by-token KL divergence across ~250K tokens, varying only KV-cache precision on the same BF16 GGUF. Gemma 4 26B-A4B jumped to KL 0.377 at q8_0 and 1.088 at q4_0, while both Qwen 3.6 models stayed under 0.04 at q8_0 and were still usable at q4_0 (LocalBench). That's a ~9x difference in degradation between architectures at the same setting. If you run local models, stop treating KV-cache precision as a universal "lower until it hurts" dial. Test it per model, because the same q8_0 that's free on Qwen visibly hurts Gemma.

ByteDance previews Seedance 2.5 with native 30-second 4K clips. At Volcano Engine FORCE on June 23, ByteDance previewed Seedance 2.5, claiming a single native 30-second clip without stitching, up to 50 reference inputs in one generation, and region-level editing that redraws part of a frame (The Next Web). Enterprise beta now, public launch targeted early July. If the 30-second native-generation claim holds up, it pressures Runway, Sora, and Veo on the one thing they all still struggle with: long-form coherence without visible seams.

Vibe Coding

Matthew Berman shipped an open library of reusable agentic "loops." His recent video walks through seven repeatable agentic workflow patterns and ships an accompanying open Loop Library at signals.forwardfuture.ai/loop-library so you can reuse them directly (Matthew Berman). The video itself is standard fare, but the shared loop library is the useful artifact. Copy-pasteable agent patterns beat another theory thread. Steal the ones that fit your stack and skip the rest.

Hot Projects & OSS

OpenCode's June 24 release leans hard into MCP context handling. The open-source OpenCode agent shipped a release that injects MCP server instructions into session context, adds MCP resource-template listing and resource-read tools, OpenCode-managed provider integration, and a lightweight --mini CLI mode (GitHub Releases). The MCP-centric changes track the broader 2026 trend: agents natively consuming server-provided instructions and resources instead of bespoke tool wiring per integration. If you're building on MCP, OpenCode is a useful reference for how a coding agent actually consumes server context in practice.

SaaS Disruption

Meta is building "Arena," an AI prediction-market app, and the rumor alone moved stocks. Per a June 23 NYT report corroborated June 24, Zuckerberg has directed a team to build a standalone app where users forecast politics, sports, and world events using daily "play money" points, no crypto or blockchain at launch (NPR). That's a deliberate contrast to USDC-based Polymarket and CFTC-regulated Kalshi, and Meta hasn't ruled out real money later. The report alone sent DraftKings and Robinhood lower. A ~3-billion-user platform sniffing at a category is enough to reprice the incumbents before it ships a single feature. That's the real disruption story: distribution as a weapon.

Coupa is buying Rossum to push document AI across all of source-to-pay. Coupa is acquiring Rossum, whose transactional LLM is trained on tens of millions of documents and adapts to each customer's document set, extending intelligent document processing well beyond accounts payable (ERP Today). It's the same theme as Mistral OCR 4 from the buy-side: document understanding is becoming table stakes inside every enterprise platform, not a standalone product. The standalone IDP vendors are getting absorbed into the suites.

The June 24 funding tape favored production agents that move money. Assort Health raised $120M Series C at a $1.2B valuation for healthcare voice/workflow agents, Taktile closed $110M Series C led by Goldman Sachs for regulated-finance decisioning, and smaller rounds went to Coval ($28M, agent testing) and Seltz ($12.5M, agent-native web search), all per Tech Startups. The through-line: capital is flowing to agents that make real decisions in regulated, revenue-critical workflows, not to horizontal model labs. The infrastructure around agents (testing, search, decisioning) is funding up alongside the agents themselves.

Policy & Governance

Agent governance is consolidating into its own infrastructure category. Runlayer raised a $30M Series A co-led by Felicis and Khosla to build a control plane that brokers what enterprise agents can access across tools and data (Tech Startups). It lands the same week MCP's Enterprise-Managed Authorization extension went stable and Vercel shipped "Passport" for shadow-AI control. Three signals, one direction: permissioning and audit for agents is becoming its own layer. This is the unglamorous plumbing Karpathy was pointing at, now getting funded. Production agent rollouts hinge as much on access control as on model quality.

NYC schools will require AI tools to pass a bias-and-equity review before deployment. New York City's Department of Education issued preliminary guidance requiring all AI tools to clear a bias-and-equity review before going live across its 1.1-million-student system, with a full compliance playbook due the same month (BuildFastWithAI). For edtech vendors this is a hard procurement gate, and it's a preview of where large public institutions are heading. If you sell AI into government or education, "passes the bias review" is becoming a checkbox you need before the sales conversation even starts.

Skills of the Day

1. Gate releases on pass^k, not pass@k. Pick the chain length that matters for your product, run the flow 10+ times, and compute the all-must-succeed number. A 70%-per-step agent is ~34% reliable over three steps even though pass@3 looks like 97%. Ship against the honest number. Source

2. Calibrate your LLM judge to Cohen's kappa ≥0.6 before you trust it. Check a model-based grader against 100+ human-labeled examples, use binary pass/fail instead of Likert scales, and explicitly test for position bias (~70% first-response favoritism), verbosity bias (>90% preference for longer answers), and self-preference (+10–25% same-family inflation). Recalibrate monthly. Most teams ship judges that quietly reward the wrong thing. Source

3. Seed your eval golden set from 20–50 real production failures, not a wishlist. Hand-picked happy paths don't reflect how your agent actually breaks. Use the rule that two domain experts must independently reach the same verdict, then scale to 100+ for judge calibration and 200–500 for production gold sets. Early-stage agents show big effect sizes per change, so even a tiny failure-derived set gives real signal. Source

4. Auto-promote failing production traces into your offline eval set. Share one data layer between production traces and offline evals so any trace that fails an online scorer gets pulled into the offline set automatically. Anthropic reports the 99.9th-percentile turn duration nearly doubled from Oct 2025 to Jan 2026, so static eval sets drift out of distribution fast. Make the suite grow itself. Source

5. Watch the silent 512-token truncation when adding a cross-encoder reranker. A hybrid retrieve-then-rerank pipeline (dense + sparse fused with Reciprocal Rank Fusion, rerank ~20 candidates) can lift recall 0.72→0.91 and precision 0.68→0.87, but most cross-encoders truncate at 512 tokens silently, scoring long chunks on a fragment. For longer content use Cohere Rerank 3.5 (4,096-token limit) or Jina Reranker v3. Source

6. Run agent sandboxes egress-deny-by-default with an API allowlist. Block all outbound connections, whitelist only the endpoints the agent needs, and restrict DNS to stop discovery attacks. Pair it with short-lived, narrowly-scoped tokens injected at task start and revoked at completion so a compromised agent can't reuse or exfiltrate long-lived secrets. Egress filtering is the highest-leverage control because it severs the path prompt injection depends on. Source

7. Set three nested timeouts on autonomous agents. Per tool call (~30s), per task loop (~20min), and per sandbox lifetime, each with its own kill point. Most teams set one overall timeout and learn too late that an agent burned tokens looping inside a still-alive session. Layered limits contain a hung tool, a looping reasoning chain, and a zombie sandbox at the exact granularity each failure happens. Source

8. Package repeated Claude Code prompts as SKILL.md skills. Any instruction set you retype ("write tests in Vitest, prefer integration tests") becomes a folder under ~/.claude/skills/<name>/ with YAML frontmatter Claude auto-discovers. Two underused refinements: store per-user setup in a config.json so the agent asks for missing values instead of guessing, and add allowed-tools: Bash Read to fence a skill out of file-writing as a cheap safety boundary. Source

9. Encode standards as .cursorrules before you scale agent count. Cursor 2.0 runs up to 8 agents in parallel, each in an isolated git worktree, but the bigger win is a committed rules file enforcing project standards, which reportedly cuts PR review comments ~70%. Stop the agent from making the same review-bait mistakes first, then fan out. Rules beat raw parallelism. Source

10. Collapse your fine-tuning stack with LlamaFactory + Unsloth. LlamaFactory (72.5K stars) fine-tunes 100+ models with no training code, supporting 2–8-bit QLoRA plus an Unsloth integration that delivers ~170% LoRA speedup and vLLM for ~270% faster inference. For a solo builder, QLoRA + Unsloth makes single-GPU fine-tuning of usefully large models actually tractable, config-driven instead of a bespoke PyTorch loop. Source