Skills2026-08-10 · source-backed

Score skill retrieval per field instead of concatenating name, description, and body.

Issue

2026-08-10

Confidence

source-backed

Sources

Redaction

passed

Story

Field Aware Agent Skill Retrieval computes sparse and dense similarity separately for each field and combines the scores, hitting 77.95 Recall@10 on SkillRet and 83.78 on SRA-Bench, with the margin widening as the skill bank grows. If your library crossed a few hundred entries and the right skill stopped surfacing, that's a retrieval-layer bug, not a prompt bug.

Source trail

Field Aware Agent Skill Retrieval More from arxiv.org

Entities

Bench Field Aware Agent Skill Retrieval Recall Score SkillRet SRA

Provenance

Canonical issue: Ramsay Research Agent — August 10, 2026
AI generated: no
Story unit: 2026-08-10-score-skill-retrieval-per-field-instead-of-concatenating-name-description-and-body
Labels: source-backed, canonical briefing excerpt

Fetching from the wire…

Score skill retrieval per field instead of concatenating name, description, and body.

Story

Related stories

Score every skill edit by rolling the task back and re-running it.

MCR-Bench shows LLM code reviewers degrade as review rounds accumulate.

93.2% of individual checks pass, 55.3% of tasks actually succeed.

TRAIL runs a translator against a challenger and gains 23.1% relative syntax accuracy on C-to-Rust.

Score a SKILL.md by reverted merged PRs, because synthetic tasks are too easy to measure anything.

A fine-tuned 4B Qwen in 2.6 GB beats GPT-5.6 on a transit-kiosk agent benchmark, and the PEFT gain vanishes by 27B.

Source trail

Entities

Provenance

Score skill retrieval per field instead of concatenating name, description, and body.

Story

Related stories

Score every skill edit by rolling the task back and re-running it.

MCR-Bench shows LLM code reviewers degrade as review rounds accumulate.

93.2% of individual checks pass, 55.3% of tasks actually succeed.

TRAIL runs a translator against a challenger and gains 23.1% relative syntax accuracy on C-to-Rust.

Score a SKILL.md by reverted merged PRs, because synthetic tasks are too easy to measure anything.

A fine-tuned 4B Qwen in 2.6 GB beats GPT-5.6 on a transit-kiosk agent benchmark, and the PEFT gain vanishes by 27B.

Source trail

Entities

Provenance

Story

Related stories

Score every skill edit by rolling the task back and re-running it.

MCR-Bench shows LLM code reviewers degrade as review rounds accumulate.

93.2% of individual checks pass, 55.3% of tasks actually succeed.

TRAIL runs a translator against a challenger and gains 23.1% relative syntax accuracy on C-to-Rust.

Score a SKILL.md by reverted merged PRs, because synthetic tasks are too easy to measure anything.

A fine-tuned 4B Qwen in 2.6 GB beats GPT-5.6 on a transit-kiosk agent benchmark, and the PEFT gain vanishes by 27B.

Tomorrow's issue, in your inbox.

Source trail

Entities

Provenance

Story

Related stories

Score every skill edit by rolling the task back and re-running it.

MCR-Bench shows LLM code reviewers degrade as review rounds accumulate.

93.2% of individual checks pass, 55.3% of tasks actually succeed.

TRAIL runs a translator against a challenger and gains 23.1% relative syntax accuracy on C-to-Rust.

Score a SKILL.md by reverted merged PRs, because synthetic tasks are too easy to measure anything.

A fine-tuned 4B Qwen in 2.6 GB beats GPT-5.6 on a transit-kiosk agent benchmark, and the PEFT gain vanishes by 27B.

Tomorrow's issue, in your inbox.

Source trail

Entities

Provenance