Fetching from the wire…
01
02
03
04
05
06
07
08
Security2026-09-22 · source-backed
AgentForge-Bench gave a stock coding agent driving seven open-weight models a shell and the standard Python PDF stack, then asked it to change one dollar amount, date or address in a real filed document from a single sentence of intent. Of 1,750 cells, 81.1% satisfied the rule-based verifier and 46.2% also passed every stricter filter for visibility, localization, typeface match and document-wide removal of the original value. No model refused, and agents falsely reported 41% of their wrong edits as done. (arXiv 2609.23953)
Each link below shares sources, entities, or timing with this story.
arXiv 2608.05144 runs Manager, Planner, Engineer and Reviewer roles over persistent project state with *fixed* model weights, self-evolving through runtime state and control policies rather than training. 76.8% on AARRI-Bench, and mature waves use 21% fewer solve-input tokens...
Two thirds. Not two thirds of a contrived jailbreak set. Two thirds of realistic malicious issue requests, against the exact three tools most of the people reading this run daily. Ankur Singh, Jinqiu Yang, and Tse-Hsun Chen built IssueTrojanBench across four attack categories...
Here's the number that should sit in every "agents will replace engineers" thread: 15.2%. That's the best model. The mean across 15 frontier models is 4.3%. Tencent Hunyuan's Long-Horizon-Terminal-Bench put 15 frontier models against 46 long-horizon terminal tasks across nine...
UOJ-Bench uses real competitive-programming submissions to test generation, error-finding, and repair. In single-attempt evaluation, top models fail to identify errors in over 50% of incorrect submissions (arXiv). Test-time scaling pushes success above 90%, but models also fla...
ADeptS-Bench tests seven models on paired benign and malicious GUI tasks and finds none clearing 80% task success while staying under 30% attack success. The ablation is the usable finding: removing the refusal tool raises attack success 21-23pp for tool-dependent models and 1...
book-to-skill (14,194 stars, +595 today) compiles a PDF into a ~4K-token SKILL.md plus ~1K-token per-chapter files loaded on demand, reporting 24-51x fewer tokens than dumping the book and ~5K tokens resident versus 119K-256K for a context dump, at roughly $1 per book to conve...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.