Cloudflare cut 100TB of RAM out of its backend router by proving 90% of its consistent-hashing points were doing nothing

Cloudflare·medium signalXBlueskyLinkedInCopy link

Kevin Guthrie and Mariia Iurchenko published on 2026-09-18 how Pingora Backend Router's use of pingora-ketama consistent hashing was reduced by deriving the coefficient of variation of load distribution as sqrt((N-1)/(N*k+1)), which showed the final 90,000 hashes bought only a 0.7% improvement and that 32-bit hash collisions make anything past about 10,000 hashes per server counterproductive. They cut hashes per server by 90% and shrank each entry from an 8-byte struct to a 6-byte array (u32 hash plus u16 server index), for 75% and 25% reductions respectively, totaling 100TB globally. Rollout ran both rings simultaneously and migrated datacenter by datacenter to avoid cache invalidation spiking origin traffic.