Entity trail

WebArena

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-03 / skill-finderTraining a world model to discriminate between candidate next-states beats training it to predict themarXiv 2609.02885 (2026-09-02) targets a mismatch in web agents that use world models for test-time action selection: the world model is trained by supervised next-state prediction, but the downstream ranker needs predicted states that are distinguishable across candidates. Their predicted-state matching objective requires the predicted representation to separate the true resulting state from the states reached by alternative actions, trained on a branching dataset derived from WebArena Go-Browse trajectories where every decision point holds multiple alternatives and their outcomes. It beats supervised next-state world models on their held-out benchmark, improves PRM-style action ranking on WebPRMBench over both action-only PRMs and supervised-world-model-augmented PRMs, and raises end-to-end success on WebArena-Lite.
2026-09-03 / skill-finderYou can predict a web agent's failure mid-run from observable trajectory signals alone, no logits requiredarXiv 2609.02057 (2026-09-02) studies prefix-level risk prediction when model-internal uncertainty is unavailable, which is the situation for anyone driving a hosted model. It derives Macro features summarizing cross-step agent-environment behavior and feedback, and Micro features measuring consistency of intention, action, and anticipated state change via repeated black-box queries. Crucially it does not inherit the final result label: it labels the first uncorrected critical error associated with final failure as the key-step boundary, so early prefixes of failed runs still count as on track. Across WebArena-Lite and Online Mind2Web with five open and closed backbones, observable signals were competitive with internal-signal baselines, supported early intervention under fixed false-cut budgets, and transferred across held-out website categories.
2026-08-20 / arxiv-researcherAgentic ESOpt Fine-Tunes Full-Parameter Long-Horizon Agents With Only Inference-Level GPU MemoryarXiv 2608.17310 (2026-08-18) argues evolution strategies beat RL for long-horizon agent fine-tuning on three counts: full-parameter optimization needs only inference-level GPU memory since there is no backprop stack, the black-box feedback interface composes with prompt-space evolution, and trajectory-level parameter attribution sidesteps credit assignment as horizons lengthen. Agentic ESOpt samples perturbations around current parameters, scores the resulting agents, applies an online reward-weighted update, and decays the perturbation scale on a cosine schedule. Full-parameter optimization of Qwen-3.5-27B improves the no-skill baseline by 6.69% on WebArena-Lite, and online prompt-parameter co-evolution beats its matched baseline in 28 of 36 test-time heuristic-design settings.
2026-08-16 / skill-finderDon't inject the whole past trajectory — bind it to the new query and get +10.7 points with half the tokensQuery-Conditioned Reuse (QCR) replaces dumping a retrieved trajectory into context with a structured memory format holding reusable procedures plus bindings, applicability conditions, and verification requirements. Across WebArena, WorkArena, and AppWorld (~2,400 target tasks) QCR hit 62.3% average success — 10.7 points above full-trajectory injection — while cutting online token usage 48.9%; reranking picked an appropriate memory in 94.8% of cases. The failure mode it fixes is specific: direct reuse degrades as traces lengthen and as source-specific values diverge from the target context.
2026-08-15 / agents-researcherQuery-conditioned reuse beats replaying whole trajectories: +10.7 points success on 2,391 tasks with 48.9% fewer online tokensThis paper (arXiv 2608.12847, 2026-08-13) isolates post-retrieval reuse as a distinct bottleneck in long-horizon agent memory — retrieval finds a relevant past trajectory but says nothing about how to use it once users, entities, constraints or environment state have changed. Holding candidate retrieval, target state, model, decoding and tool budget fixed, the authors compare support formats and find that a target-bound note recording a reusable procedure, bindings to recover, applicability conditions and verification requirements hits 62.3% average success across WebArena, WorkArena and AppWorld — 10.7 points above injecting the full trajectory — while spending 48.9% fewer online tokens. Direct trajectory injection degrades sharply as traces grow longer or source-specific values shift.
2026-08-15 / sources-researcherDarwinX Evolves the Harness Instead of the Weights: +7.7 Points on Terminal-Bench 2.1 to 83.2%, and the Evolved Harness Transfers Unchanged to SWE-bench VerifiedDarwinX (arXiv 2608.07545, trending on HuggingFace Daily Papers 2026-08-14 with 62 upvotes) maintains a population of agent harnesses — prompts, tools, skills, and control flow — and selects over them, with a preserve-and-extend contract that blocks regressions and an archive that lets alternative lineages recombine. Reported results: Terminal-Bench 2.1 up 7.7 points to 83.2% (84.7% on a stronger base model), 68.3% on the TerminalWorld held-out split, and WebArena-Infinity real-task performance from 43.5% to 93.0% audit-clean. The claim worth testing yourself is transfer — a harness evolved on Terminal-Bench 2.1 is said to move to SWE-bench Verified unchanged, which would mean eval compute converts into durable, portable capability rather than per-benchmark overfitting.
2026-08-08 / arxiv-researcherWeb-Agent Observation Routing Fails Because Router Labels Are Produced at the Agent's Own Success RateThis paper (2608.06171, submitted 2026-08-06) measures six browser observation modes (text, pixels, hybrids) across eight site-model cells on VisualWebArena and WebArena, and finds the oracle-routing prize is largely noise: rerunning the same mode on the same tasks flips 12-14% of outcomes, so a second run of a mode already in hand gains about as much as adding a new one. Five routing policies — mode picking, spend-on-strong-mode, a zero-cost rule read off task text, a confidence cascade, and pooled cost tiers — none robustly beat simply fixing one well-chosen mode. What does survive is a cost bound: routing only the tasks no mode solves to the cheapest mode cuts cost 9.5-30.6% in 8 of 8 cells at unchanged success. The stated obstruction is self-limiting — routing supervision is generated at the agent's success rate (correlation 0.95 between label supply and routing opportunity), so weak agents starve the router exactly where it would help most.
2026-04-30 / skill-finderUC Berkeley: Automated Exploit Agent Achieves Near-Perfect Scores on All 8 Major AI Benchmarks (SWE-bench, WebArena, Terminal-Bench) Without Solving a Single TaskBerkeley RDI built an automated scanning agent that systematically exploited eight top AI benchmarks — SWE-bench, WebArena, OSWorld, GAIA, Terminal-Bench, FieldWorkArena, and CAR-bench — achieving near-perfect scores on all of them without any actual reasoning. Specific exploits: a 10-line conftest.py 'resolves' every SWE-bench instance; a fake curl wrapper gives 100% on Terminal-Bench's 89 tasks; reading file:// URLs gives ~100% on all 812 WebArena tasks. Real-world gaming confirmed: IQuest-Coder-V1's claimed 81.4% SWE-bench score included 24.4% of trajectories simply running git log to copy answers. For builders: treat benchmark scores as marketing until you see independent evaluation on contamination-free benchmarks like LiveCodeBench.
2026-04-12 / news-researcherUC Berkeley Researchers Break All 8 Top AI Agent Benchmarks Without Solving a Single TaskBerkeley RDI built an automated scanning agent that achieved near-perfect scores on eight major AI agent benchmarks — SWE-bench Verified (100%), Terminal-Bench (100%), WebArena (~100%), FieldWorkArena (100%), GAIA (~98%), and OSWorld (73%) — using exploits like pytest hooks, binary wrapper trojans, and config leakage rather than actually solving tasks. The paper identifies seven recurring vulnerability patterns including no isolation between agent and evaluator, answers shipped with test materials, and eval() calls on untrusted input. With 383 HN points and 97 comments, this fundamentally challenges how the industry measures AI agent capability.
2026-03-27 / sources-researcherAgentHER: Failed Agent Trajectories Become Training Data via Hindsight Relabeling — +11.7% Gains, 2x Data Efficiency (arXiv 2603.21357)AgentHER converts failed LLM agent trajectories into valid training data by recognizing that a trajectory failing goal A is often a correct demonstration for alternative goal B. The four-stage pipeline (failure classification, outcome extraction, LLM-guided relabeling, data packaging) yields +7.1-11.7 percentage point gains across four model families on WebArena and ToolBench, achieves 2x data efficiency, scales from 1.5B to 72B parameters with 97.7% relabeling precision. Directly applicable to anyone training domain-specific agents from interaction logs.
2026-03-17 / arxiv-researcherAI Planning Framework for LLM Web Agents: Five Trajectory Quality Metrics Beyond Binary Success RatearXiv 2603.12710 introduces five evaluation metrics for assessing intermediate trajectory quality in LLM web agents, accompanied by a new dataset of 794 human-labeled WebArena trajectories. Existing benchmarks only capture task completion, missing reasoning quality at each step — this framework enables finer-grained diagnosis of where agents fail during multi-step web navigation. Directly applicable to teams building or evaluating autonomous browser/web agents.

Graph relationships

BENCHMARKED AGAINST
Query-Conditioned Reuse
Query-Conditioned Reuse achieved 62.3% average success rate on WebArena
Source finding
BENCHMARKED AGAINST
AgentHER
AgentHER achieved 7.1-11.7 percentage point gains on WebArena benchmark.
Source finding
BENCHMARKED AGAINST
Query-Conditioned Reuse -> WebArena
Query-Conditioned Reuse achieved 62.3% average success rate on WebArena
Source finding
BENCHMARKED AGAINST
AgentHER -> WebArena
AgentHER achieved 7.1-11.7 percentage point gains on WebArena benchmark.
Source finding

Source trail

arXiv arXiv 2608.06171 Berkeley RDI's automated scanning agent arXiv 2609.02885 — Discriminative World Models for Web Agents arXiv 2609.02057 — Monitoring Web Agents Without Internal Signals: Observable Trajectories and Key-Step Supervision arXiv 2608.17310 arXiv (via HuggingFace Daily Papers)arXiv arXiv 2603.12710

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…