Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-09-16 · high
JustFit swaps model pieces in and out of memory, giving a single request 6.93x more context than before.
Why now: The paper went up on arXiv on September 15, 2026.
A new inference runtime called JustFit let a 24 GiB M4 Pro MacBook hold 196,608 input tokens and 16,384 output tokens in one request, per the JustFit paper. The jump takes single-request context to 212,992 tokens total, 6.93 times what the mlx-vlm baseline handled on the same hardware. This gives the model enough room to hold a full codebase in memory instead of chunking it for a cloud API.
JustFit combines three techniques: compressed KV execution, component residency swapping, and state-preserving transitions between serving requests.
A 32K-input probe reached 19.11 tokens/s, with a median peak process footprint of 16,374 MiB, comfortably inside the 24 GiB budget.
The runtime also answered 29 of 30 AIME 2026 problems correctly. The paper doesn't say how that score compares to the same model running on server hardware.
Three runs is a thin sample for numbers this exact, and the paper doesn't test hardware beyond one M4 Pro machine. For builders with large codebases or long documents, a 24 GiB laptop holding 200K-plus tokens changes what's possible without a cloud API bill.
Each link below shares sources, entities, or timing with this story.
Weights and the full training stack are public, and the 35B agent beats GPT-5.5 plus Codex while capped at 12GB of VRAM.
A blinded judge checks root cause and impact against 95 real CVEs, and no frontier model made the ten-model lineup.
JustFit combines compressed KV execution, component residency swapping and state-preserving serving transitions. On an M4 Pro running Qwen3.8-27B MXFP4, three independent runs completed 196,608 input and 16,384 output tokens, lifting single-request context from 30,720 position...
An ablation that skipped the router entirely tied the full system's score, per the paper.
A training-free fix called ChannelGuard held steady across three model backends, filter or no filter, blocking every tool-poisoning attempt.
The same researchers warn prompt wording matters almost as much as the retrieval backend, undercutting an easy graph-retrieval fix.
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.