Microsoft distilled its pathology foundation models to 22M parameters at 50x less compute, Apache 2.0.

GigaPath-Flash pairs a 22M-parameter ViT-S tile encoder distilled from the billion-parameter GigaPath teacher with a 21M-parameter LongNet slide encoder, retaining 97% of predictive performance (Microsoft Research). GigaTIME-Flash is about 6x faster with 8x less memory, cutting a 1-million-slide job on a single A100 from about 300 GPU-days to about 70, while matching or beating the original across brain, breast, colon and lung cohorts including out-of-distribution ones.