Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-25 · high
The architecture funnels local features into proxy states per layer, running attention there instead of over every token.
Why now: The trade-off numbers come from a single arXiv posting, with no comparison yet against other long-context compression methods on the same hardware.
ProxyFormer trains sequences of about 700,000 tokens on a single 16GB GPU, where a standard transformer decoder manages about 20,000 tokens on the same card, according to the ProxyFormer paper. For anyone training long-context models on one consumer or workstation GPU, this closes the gap between a short document and a full codebase or long transcript.
The architecture squeezes local features into a small set of proxy states per layer, working bottom-up. Global attention runs across that compressed set, not over every token directly. The results decompress and feed back into a local stream that persists across layers, so a detail one compression pass misses stays recoverable later.
The full trade-off happens at a compression ratio of 64, batch size 1. A model trained with a 64,000-token window held 92 to 95% accuracy on a multi-needle retrieval test, and the design adds a proxy-only key-value cache scheme for inference. The paper doesn't say whether pushing the compression ratio past 64 keeps that accuracy intact.
Each link below shares sources, entities, or timing with this story.
Shared entity: GPU / Same source domain / Shared topic / Earlier coverage
Both cover GPU; reported by the same outlet (arxiv.org); overlapping topics (accuracy, attention).
Both cover GPU; reported by the same outlet (arxiv.org); overlapping topics (accuracy, attention).
Both cover GPU; reported by the same outlet (arxiv.org); overlapping topics (accuracy, compression).
Shared entity: GPU / Same source domain / Earlier coverage / Tension
Both cover GPU; reported by the same outlet (arxiv.org); earlier GPU coverage from 2026-08-12.
Both cover GPU; reported by the same outlet (arxiv.org); earlier GPU coverage from 2026-06-20.
Both cover GPU; reported by the same outlet (arxiv.org); earlier GPU coverage from 2026-03-20.
Shared entity: GPU / Same source domain / What happened next
Both cover GPU; reported by the same outlet (arxiv.org); picks up the GPU thread on 2026-08-26.
Shared entity: GPU / Same source domain / Earlier coverage
Both cover GPU; reported by the same outlet (arxiv.org); earlier GPU coverage from 2026-08-16.