Fetching from the wire…
01
02
03
04
05
06
07
08
Models2026-08-02 · source-backed
Chimera processes text, image and video tokens as one raster-ordered stream with no positional embeddings, combining Kimi Delta Attention for O(N) state tracking, interleaved Multi-head Latent Attention, modality-aware short convolutions, and sparse MoE. The real contribution is HeteroP, a module-wise hyperparameter transfer scheme keyed to each tensor's functional fan-in and depth, which produces a family consistent enough to actually fit compute-optimal laws. The 11B/2B-active model is 1.7x more compute-efficient than a full-attention dense backbone and extrapolates zero-shot from 5-second training clips to 30-second video with 6.5% FID degradation in the final five seconds.
Each link below shares sources, entities, or timing with this story.
Most looped-transformer results compare at fixed model size, conflating architecture with extra compute. SMELT matches per-token FLOPs, non-embedding parameters and KV cache against an unlooped baseline, scaling to 54B non-embedding parameters with a separate Chinchilla-style...
Alagappan Valliappan shows built-in Multi-Token-Prediction draft heads run full attention over the entire KV cache at every draft step, so at million-token context the "negligibly cheap" draft dominates cost and deep native drafts can go net-negative. Applying a StreamingLLM-s...
claude_codex_bridge (3,165★, Python) routes subtasks across heterogeneous coding agents and, more importantly, makes the cross-agent collaboration observable instead of a black box. Early-stage, but it's aimed at the emerging practice of routing different subtasks to different...
The architecture report describes a 125B sparse MoE activating 6B parameters per token, plus 51B of n-gram embedding tables held off the accelerator in host memory with prefetching (arXiv 2608.30320). Against the prior 397B-A17B model it leads on 8 of 14 pre-training benchmark...
The authors model strategic bidding as a repeated game with imperfect public monitoring, then run multi-agent RL over it, and build a criteria set for judging collusion that goes beyond comparing profit against Nash equilibria. Agents sustained supra-competitive outcomes match...
Load Hijack modifies nothing but router weights in a checkpoint. When a private trigger appears, token-to-expert assignment concentrates on experts co-located on a single GPU, making it a straggler while peers idle (arXiv 2608.10614). Across three MoE families and four corpora...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.