Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-09-12 · high
The search-based approach cut attack success from 45.6% to 10% while barely touching task performance.
Why now: The paper posted its results as of September 12.
A new paper argues that safety guardrails written once by a person and applied to every model are the wrong shape. EvoSafeHarness instead searches for a custom harness per model, combining natural-language policy text with executable code checks, and scores each candidate against behavior analysis, domain specs, and adversarial review.
The stakes are concrete. A harness strict enough to stop a jailbreak-prone model tends to over-block a more cautious one, so teams either ship a harness tuned for their worst model and eat false refusals everywhere else, or tune per-model by hand and fall behind.
On DecodingTrust-Agent, the searched harness cut attack success from 45.6% to 10.0%, for a 3.3-point drop in task utility, and won 14 of the 15 test cells in that benchmark. On AgentDojo it held 82.8% utility while pushing attack success to 0%, according to the paper.
I don't build agent safety layers day to day, but I've spent enough time tuning prompts and guardrails in my own projects to recognize the pattern. A hand-written rule that works great on one model version breaks on the next one, and you're back to manual retuning. Treating the harness itself as a search target instead of a document someone edits is the kind of thing that collapses that cycle.
The paper doesn't say how long the search takes to converge for a new model, or what happens when a model gets updated after a harness has already been fit to it. Those are the questions that decide whether this stays a research result or turns into a real workflow.
My take: this becomes a checkbox in agent platforms within a quarter, the way prompt caching and structured outputs did, because the alternative (a security team hand-editing policy text per model release) doesn't scale past a handful of models. Watch for whether any of the major agent frameworks ship a harness-search step before the end of the year.
Each link below shares sources, entities, or timing with this story.
arXiv 2608.11878 replaces the handful of manually implemented injection-testing environments with an Environment Simulator, Attacker Agent, and User Simulator that generate executable stateful environments and discover viable injection points automatically. Injection timing an...
arXiv 2608.27234 calls the planner exactly once per query to emit a full plan in a declarative DSL, then applies dual-lattice information-flow control over confidentiality and integrity across explicit data flows and control dependencies, storing results as labeled artifacts a...
First defense modeling multi-turn indirect prompt injection as temporal causal takeover. Uses counterfactual re-executions at tool-return boundaries to detect when tool outputs steer agent behavior. Evaluated on AgentDojo across four task suites. Builder-ready pattern for tool...
arXiv 2609.09875 argues existing frameworks measure completion (AgentBench) or robustness (AgentDojo, ASB) but never attribute a failure to a stage. It scores instruction integrity, planner, memory, tool selection, invocation, correctness, alignment, faithfulness, security and...
arXiv 2608.05108 skips the RL-trained attacker models that dominate red teaming and generalize poorly, instead accumulating a strategy library across a sequence of (dataset, target) pairs that transfers to unseen targets with no retraining. AgentDojo: 86.7% ASR against Gemini-...
A July 21 paper pairs two near-identical agents: an Explore Agent that inspects untrusted input but holds no tools, and a Safe Agent that takes privileged actions using its own context plus length-constrained hints from the explorer (arXiv 2607.19595). Borrowing from residual...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.