Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-31 · high
LongPIBench tests attacks across four realistic tasks at production-scale context, and simple attacks still get through.
Why now: The paper covering peer review, resume screening, code review, and email summary went up as new evidence on the problem.
Most prompt-injection benchmarks test short prompts. Real agents don't run on short prompts. They run on peer review packets, resume stacks, full code diffs, email threads, tens of thousands of tokens deep. LongPIBench tests injection attacks at that scale, across four realistic scenarios: peer review, resume screening, code review, and email summary. The finding: even simple heuristic attacks reach high success rates at these lengths, and they frequently get past defenses that hold up fine in short-context tests.
That gap matters because it's exactly where the industry has been grading its own homework. A defense that scores well on a benchmark built from short prompts tells you almost nothing about what happens when an attacker buries an instruction on page 40 of a resume batch or line 2,000 of a diff. Production agents summarizing email, screening candidates, or reviewing code operate at those lengths by default. If your threat model comes from a short-context eval, you're defending the wrong perimeter.
I don't build agents that ingest attacker-controlled documents at scale, so I can't tell you which specific defense mechanism breaks first. The paper doesn't say whether the drop-off is gradual as context grows or a cliff past some threshold. Worth checking before you trust a vendor's injection-defense claim: what context length did they test at, and does it match what your agent actually sees in production.
The take here is simple. Any defense number that doesn't specify context length should be treated as untested for your use case, because the evidence now says length is where these defenses fail.
Each link below shares sources, entities, or timing with this story.
Shared entity: Production / Same source domain / Shared topic / Earlier coverage / Tension
Both cover Production; reported by the same outlet (arxiv.org); overlapping topics (agent, benchmark).
Shared entity: Prompt / Same source domain / Shared topic / Earlier coverage
Both cover Prompt; reported by the same outlet (arxiv.org); overlapping topics (agent, context).
Shared entity: Production / Same source domain / Shared topic / Earlier coverage
Both cover Production; reported by the same outlet (arxiv.org); overlapping topics (agent, attack).
Both cover Production; reported by the same outlet (arxiv.org); overlapping topics (agent, code).
Same source domain / Shared topic / Tension
Reported by the same outlet (arxiv.org); overlapping topics (agent, attack, benchmark, current); pushes against this story (against).
Reported by the same outlet (arxiv.org); overlapping topics (agent, benchmark, code, context); pushes against this story (versus).
Reported by the same outlet (arxiv.org); overlapping topics (agent, benchmark, code, current); pushes against this story (against).
Reported by the same outlet (arxiv.org); overlapping topics (agent, attack, benchmark, defens); pushes against this story (against).