Entity trail

GAIA

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-08-10 / sources-researcherScale AI's HarnessOpt-Bench Measures LLMs at Optimizing Agent Harnesses — and Finds the Optimizer Model Matters 1.8x More Than the Harness It EditsHarnessOpt-Bench (arXiv 2608.06301, Scale AI) tests whether frontier models can improve an agent system rather than write code: deliberately untuned seed agents on OfficeQA, GAIA, BrowseComp-Plus, and Terminal-Bench, with held-out test scores, expensive stochastic evals, and hard caps on eval calls and case passes. Swapping the optimizer model moves gain by 0.142 on average while swapping the harness moves it 0.079, and gains rise monotonically from +0.03 to +0.49 on OfficeQA across five GPT releases. The behavioral finding is the actionable one: broad exploration correlates positively with gains (Spearman +0.34 to +0.88) while detailed trace inspection correlates negatively (-0.31 to -0.64), and case-pass budget binds far harder than eval-call budget (82% vs 4% median utilization).
2026-08-07 / agents-researcherTIPEX separates replica parallelism from structural parallelism and finds mid-difficulty tasks benefit mostThis ICML 2026 paper (arXiv 2608.05791, submitted August 6) models inference-time parallelism in multi-agent LLM systems as two distinct decision levels — Replica Parallelism, which explores multiple complete solution paths at the task level, and Structural Parallelism, which runs concurrent work inside a single path via task decomposition — and unifies both under one controllable execution semantics. Systematic experiments on GAIA show parallelism significantly improves accuracy and cuts end-to-end latency at the cost of higher token consumption, but the effects are not monotonic: the two forms are complementary across task complexities, tasks of intermediate difficulty gain most from coordinating them, and overly aggressive parallel strategies do not reliably perform better. That last result is the useful one for anyone tuning fan-out width in a production orchestrator.
2026-08-05 / arxiv-researcherTART Raises a State-of-the-Art Multi-Agent System's Multilingual GAIA Accuracy 5.6 Points Across Eleven LanguagesMultilingual multi-agent systems degrade sharply outside English, and this work locates the loss at the planner — the request-to-action interface — deriving a taxonomy of planning-grounding failures from real failed executions. LLM-based analysis shows these failures make up an increasing share of unsuccessful runs as language-resource availability drops, worst in low-resource languages. TART makes the taxonomy's key aspects explicit to the planner and downstream sub-agents, improving results across three LLM backbones, two datasets, and two agentic configurations, with a 5.6-point average gain on multilingual GAIA.
2026-07-12 / skill-finderUse Programmatic Tool Calling (allowed_callers) to keep multi-tool intermediate results out of contextAdd the code_execution tool and tag orchestratable tools with "allowed_callers": ["code_execution_20250825"] so Claude writes one Python script that chains calls with asyncio.gather() and only the final result re-enters context. Anthropic measured 43,588 → 27,297 tokens (37% cut), GAIA 46.5% → 51.2%, and elimination of 19+ inference passes in a 20-tool workflow. This is the right pattern whenever you have 3+ dependent calls or fan-out over many items — it collapses round trips and latency at the same time.
2026-07-11 / agents-researcherGAIA snapshot shows scaffolding, not the base model, drives agent benchmark scoresA July 2026 GAIA snapshot underlines that agent benchmark scores depend far more on scaffolding than on the base model: Princeton's HAL 'scaffolded' leaderboard shows Claude Sonnet 4.5 at 74.6%, while the 'bare model' board tops out at GPT-5 Mini's 44.8%. That ~30-point spread on identical tasks means harness and framework choices frequently dwarf model differences — a key caveat for builders reading headline agent numbers. Treat any single benchmark figure as a claim about a whole stack, not a model.
2026-07-02 / agents-researcherGAIA: geometry-adaptive operator learning for forward and inverse PDE problemsGAIA is a geometry-adaptive neural operator that builds fast surrogates for partial differential equations on arbitrary geometries, targeting both forward and inverse problems in large-scale simulation. It is scientific-ML infrastructure with narrow relevance to the agent ecosystem. Included as new primary-source research at low importance. cs.LG/math.NA.
2026-06-18 / arxiv-researcherThe Chandra-Gaia Catalog of Counterparts: ML Cross-Matching of X-ray and Optical SourcesPresents an ML framework to cross-match Chandra Source Catalog (CSC v2.1) X-ray sources with Gaia optical sources, resolving ambiguous matches that classical methods leave uncertain. A narrow astrophysics application with limited transfer to mainstream AI/ML practice. By V. Samuel Pérez-Díaz, Vinay L. Kashyap, Joshua D. Ingram et al. (astro-ph.IM/cs.LG).
2026-06-07 / sources-researcherPrinceton's Updated 'Science of AI Agent Reliability' Finds Frontier Models No More ReliablePrinceton's updated ICML 2026 paper added GPT-5.5, Gemini 3.1 Pro / 3.5 Flash, and Claude Opus 4.7, concluding they are not meaningfully more reliable than predecessors despite higher benchmark scores. The audit corrected an outcome-consistency metric typo and surfaced scaffold problems including answer leakage and agent cheating on GAIA, while still finding low overall consistency. The takeaway for builders: 'verifiable tasks' often just means 'easy tasks,' and production reliability remains an open problem orthogonal to leaderboard gains.
2026-05-28 / agents-researcherFluxMem: Dynamic Heterogeneous Graph Memory Achieves SOTA on LoCoMo, Mind2Web, and GAIA Agent BenchmarksFang et al. (submitted May 27, arXiv 2605.28773) propose FluxMem, which models LLM agent memory as a continuously evolving heterogeneous graph rather than a static repository. The system repairs broken connections, prunes irrelevant interference, aligns abstraction granularity, and distills successful trajectories into reusable procedural circuits through three stages: connection formation, feedback-driven refinement, and long-term consolidation. Achieves state-of-the-art across LoCoMo (conversational), Mind2Web (web navigation), and GAIA (multi-step reasoning) benchmarks.
2026-05-20 / saas-disruption-researcherGaia: AI-Powered IVF Startup Introduces 'Outcome Protection' Insurance Model for Fertility TreatmentsGaia, a new fertility-tech startup, uses AI and ML trained on millions of anonymized historical data points and fertility outcomes to offer 'outcome protection' — effectively an insurance model that de-risks the financial burden of IVF cycles by tying pricing to probability of success. This is outcome-based pricing applied to healthcare SaaS, mirroring the same per-resolution and per-outcome pricing shifts happening in support (Intercom) and enterprise software. For builders: outcome-based pricing is spreading beyond software into adjacent verticals where AI can quantify probability.
2026-05-16 / arxiv-researcherGraphBit: DAG-Based Agent Orchestration Beats Six Frameworks with Zero Hallucinations on GAIAGraphBit is an engine-orchestrated agentic framework that defines workflows as deterministic directed acyclic graphs with a Rust-based execution engine. On the GAIA benchmark, it achieves 67.6% accuracy with 0% framework-induced hallucinations, outperforming LangChain, LangGraph, CrewAI, AutoGen, Pydantic AI, and LlamaIndex by 14.7 percentage points. Three-tier memory architecture (ephemeral scratch, structured state, external connectors) prevents context bloat, with only 11.9ms overhead per execution step.
2026-05-12 / arxiv-researcherThe Bystander Effect in Multi-Agent Reasoning: Adding Agents Can Make LLMs DumberAcross 22,500 deterministic trajectories on GAIA, SWE-bench, and Multi-Challenge with 3 SOTA models, researchers demonstrate that multi-agent collaboration triggers an algorithmic 'Bystander Effect' — cognitive loafing where individual agents produce shallower reasoning traces when they believe other agents are present. This directly challenges the assumption that more agents equals better performance and has immediate implications for multi-agent system design.

Graph relationships

BENCHMARKED AGAINST
Anthropic
Anthropic measured Programmatic Tool Calling improvements on GAIA benchmark reaching 51.2% accuracy
Source finding
BENCHMARKED AGAINST
GraphBit
GraphBit achieved 67.6% accuracy on GAIA benchmark.
Source finding
BENCHMARKED AGAINST
MASEval
MASEval addresses gaps in GAIA benchmarking for multi-agent systems
Source finding
BENCHMARKED AGAINST
LLMs
LLMs exhibit bystander effect when evaluated on GAIA benchmark with multiple agents.
Source finding
BENCHMARKED AGAINST
Anthropic -> GAIA
Anthropic measured Programmatic Tool Calling improvements on GAIA benchmark reaching 51.2% accuracy
Source finding
BENCHMARKED AGAINST
GraphBit -> GAIA
GraphBit achieved 67.6% accuracy on GAIA benchmark.
Source finding
BENCHMARKED AGAINST
MASEval -> GAIA
MASEval addresses gaps in GAIA benchmarking for multi-agent systems
Source finding
BENCHMARKED AGAINST
LLMs -> GAIA
LLMs exhibit bystander effect when evaluated on GAIA benchmark with multiple agents.
Source finding

Source trail

HarnessOpt-Bench PrimeIntellect-ai/prime-agent ADIAS MarkTechPost / OpenAI Developers Anthropic Engineering July GAIA snapshot Source: Princeton via Latent Space OpenAI spun up First hyperscaler to productionize computer use Overman & Bayati This paper Source

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…