Latent Space — 'Inside the Model Factory': Poolside Trains a 118B MoE With Fewer Than 70 Researchers and 10,000–20,000 Experiments a Month

Latent Space (swyx + Alessio)·high signal

In the 23 July Latent Space episode, Poolside co-CEO Eiso Kant details the 'Model Factory' behind Laguna S 2.1 — 118B total parameters with 8B active per token, 1M context, trained in FP8, eight weeks from training start to launch on a 10,000 H200 cluster with zero on-call incidents. The operational numbers are the story: under 70 researchers run roughly 10,000–20,000 experiments per month, checkpoints are evaluable within 30 minutes of completion, and training data is streamed rather than pre-materialized to kill re-staging cycles. Kant's thesis — 'model building is ultimately 90% engineering' and '95% of model building' reduces to improving data or compute efficiency — reframes the moat as pipeline throughput rather than architecture; Laguna S 2.1 shipped as open weights on Hugging Face under OpenMDW-1.1 and fits on a single DGX Spark.