Personalized agent skills barely beat having no skill at all

This one annoyed me, in the good way.

Researchers took 206 real developer-agent sessions from 13 developers, extracted each developer's preferences from their actual interaction traces via rule-based bootstrapping plus evidence-grounded refinement, then replayed everything against a trajectory-conditioned developer simulator (arXiv 2608.10319). Real sessions, not synthetic tasks.

Personalized skills produced only small and inconsistent gains over a no-skill baseline. Generic skills pooled across all 13 developers produced the largest and most consistent gains. Personalization helped only when a developer's history already contained multiple examples relevant to the future task, which is a narrow condition that mostly means "you've done this exact thing before."

Every agent-memory product shipping right now assumes the opposite. GitHub added Copilot memory to JetBrains this week specifically so you stop restating project conventions (GitHub Changelog). Grok Bot's pitch includes carrying memory of past conversations and user preferences across tasks (MacRumors). River AI raised $1.1 billion at two months old on "personally trainable agents" that are "yours, not someone else's" (TechCrunch). General Catalyst led, with Nvidia, AMD Ventures, YC, and Temasek in.

Read that against GitSkills, from the same agent, same day: 3,797,117 SKILL.md files scraped from 282,200 public GitHub repos in July 2026, 1,877,981 of them distinct, packaged as a single SQLite file (arXiv 2608.10906). Anthropic published the spec in October 2025. That's nine months to nearly four million files. The format won. This paper says a lot of what people are putting in it may not earn its keep.

There's a counterweight I should name honestly. AlignXada learns preference adaptation via verbal reinforcement learning, gains 3.82 points average across 13 tasks while retaining only 22.8% of the original profile tokens, and beats RAG in 36 of 39 tested cells (arXiv 2608.09507). So personalization isn't dead, but the winning version aggressively distills the profile rather than stuffing or retrieving it. Which is, again, story 3's thesis wearing a different hat.

And if your agent writes its own skills, gate them. VaG uses three complementary critics to vet a candidate skill before it enters the library, reaching 72% pass@1 with a pool roughly 5x smaller than unconditional accumulation (arXiv 2608.05810). The three critics intercept largely disjoint classes of bad skill, so dropping any one leaves a real hole. Admission control matters more than generation.

Practical version: spend your engineering hours on shared skills your whole team or all your projects use. The per-user preference layer is where I'd stop investing until someone replicates this in the other direction.