A 264MB adapter beats GPT-5 at compressing agent context

Researchers behind Paritok-4B built a LoRA adapter on Qwen3-4B, distilled from a gpt-4.1-mini teacher over 67,074 real OpenHands agent trajectories. The job is context compression: shrinking the log of an AI agent's actions so a downstream model can keep working without re-reading everything it already did.

The numbers are the story. Across all 300 SWE-bench Lite instances, Paritok-4B compresses trajectories to 25.7% of their original size while keeping 86.5% of the solve quality you'd get uncompressed. A gpt-4.1-mini compressor manages 50.2% retention on the same task. gpt-5 does better at 61.9%, still well short of a fine-tuned 4B model a fraction of its size.

The adapter is extractive, not generative. 96.0% of the identifiers, file paths and numbers it emits already appear in its input, and it's conditioned on the agent's current task rather than summarizing blind. That constraint seems to be why it beats larger, more general models at a narrow job.

It also runs on one 24GB GPU, so there's no per-token bill. The paper states that gpt-5, priced at list rates as a compressor, costs more than the tokens it would save downstream. That's a real number from a real workload, not a hypothetical.

Weights, training data and eval scripts are released under Apache 2.0. For anyone running agents that accumulate long trajectories, this is a case where a small, task-specific model outperforms a frontier one, and you can self-host it instead of paying per call to trim your own context.