Fetching from the wire…
01
02
03
04
05
06
07
08
Research2026-08-06 · source-backed
arXiv 2608.05136 proves gauge-equivariance is necessary (not sufficient) for the transfer on a factored model W = UVᵀ. GD, momentum, shared-scalar Adam, Muon and Shampoo satisfy it. Adam, RMSProp and other coordinate-wise methods don't. A one-parameter family sweeping from coordinate-wise to shared-scalar preconditioning restores the bias monotonically, isolating anisotropy as the cause. In transformers Adam separates two gauge-equivalent initializations at the first step, ending with per-head WQᵀWK invariants 56% apart. Directly relevant if you're tuning LoRA adapters.
Each link below shares sources, entities, or timing with this story.
LoRA benchmarked against Mistral / Shared entity: LoRA / Same source domain / Earlier coverage / Tension
Linked by a graph relationship (LoRA benchmarked against Mistral); both cover LoRA; reported by the same outlet (arxiv.org).
Adam uses Anthropic / Shared entity: Directly / Same source domain / Earlier coverage / Tension
Linked by a graph relationship (Adam uses Anthropic); both cover Directly; reported by the same outlet (arxiv.org).
DoRA competes with LoRA / Shared entity: LoRA / Earlier coverage
Linked by a graph relationship (DoRA competes with LoRA); both cover LoRA; earlier LoRA coverage from 2026-06-28.
Adam uses Anthropic
Linked by a graph relationship (Adam uses Anthropic).
Linked by a graph relationship (Adam uses Anthropic).
Linked by a graph relationship (Adam uses Anthropic).
Linked by a graph relationship (Adam uses Anthropic).
Linked by a graph relationship (Adam uses Anthropic).