Fetching from the wire…
Public story · 2026-09-10 · high
The wrapping missed the transformer layer and checkpointed each child module instead, so activations piled up.
Why now: Hugging Face released the fix on September 9.
Hugging Face released Accelerate 1.15.0 on September 9, fixing a bug in FSDP2's activation checkpointing.
The bug kept memory allocated that checkpointing exists to free. That cuts into the savings teams rely on when training large models across multiple GPUs.
The wrapping logic checkpointed self_attn, mlp and the norm layers on their own instead of the matched transformer layer that contains them. Every activation passed between those child modules stayed alive through the backward pass rather than being freed and recomputed. PR #4172 moves the wrap up to the full layer.
The same Accelerate 1.15.0 release notes list a second fix that's worse in a quieter way. Combining FSDP2, PEFT and FULL_STATE_DICT dropped every rank's adapter shard except rank 0's when saving a checkpoint.
The file writes without error, it loads without error, and nothing in that path tells you weights are missing. Anyone running that combination ends up training or serving with a fraction of the adapter they saved.
Each link below shares sources, entities, or timing with this story.
First-of-its-kind open cloud solution for electronics digital twins. Enables 90% of software validation before hardware availability. Volvo Cars is early adopter. Built on open-source SIL Kit with cloud deployment. Significant for physical AI system builders who need to test o...
1. AWS News Blog — AgentCore Policy GA 2. Synaptics — Coral Dev Board 3. Synopsys — eDT Platform 4. Dataconomy — Grok UK 5. EQS News — Axelera Europa 6. Adversa AI — March Security Digest
Oldenburg, de Kam and Zuijdam note in arXiv 2607.18130 that essentially every parameter-efficient finetuning method (LoRA, adapters, prefix tuning) modifies weights or activations while leaving residual connections untouched. Their method places manifold constraints on hyper-c...
MetroLLM-Bench is 955 cases across six real metro systems of 37 to 414 stations, requiring structured tool calls and a machine-renderable terminal state. On the 238-case held-out split the 4B student scores 91.3 on Tier 1 against GPT-5.6's 90.6 and 90.0, at Q4_K_M. The gain ov...
Released September 6 with 8 merged PRs. PR #3960 puts approval gates and workspace checkpoint/rollback into the graph runtime, so a long-running agent team pauses for a human decision and rewinds to a known workspace state. That pairs directly with the geometric-degradation fi...
Set use_dora=True in PEFT's LoRAConfig with the 2026 starting recipe (r=16, target_modules='all-linear'). DoRA decomposes weights into magnitude and direction and applies LoRA only to direction, yielding +3.7% on LLaMA-7B and +1 to 4.4% on larger models with zero added inferen...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.