Templar simulates pipeline-parallel training that routes around offline stages with little loss penalty
Templar (via r/MachineLearning)·low signal
Templar's Crucible platform combines data-parallel replicas, pipeline compression and SparseLoCo. When an inner stage goes offline, activations and gradients bypass it for several steps. In a 178M-model simulation with 8 replicas, 4 stages per replica, a 1% per-replica failure chance per global step and six-step outages, validation loss stayed close to the no-failure baseline. Fixed projections shared across layers made the bypass more robust. The authors say this simulates learning effects only, not real worker replacement or cost savings (r/MachineLearning).