Show HN: OpenLake Offloads KV Cache to a Shared Petabyte Pool, Claiming 66× Faster Time-to-First-Token

GitHub / Hacker News·medium signal

OpenLake, a Rust/io_uring storage engine posted to Show HN today, moves LLM KV cache off the GPU into a petabyte-scale store co-located on GPU hosts so a prefix computed on one GPU can be served to others, claiming million-plus IOPS within 1ms and a 66× time-to-first-token speedup on cache hits at 128K context. The demo serves Gemma4-31B on an H100 at 256K context, and it plugs into vLLM through an `openlake-vllm` connector configured via `--kv-transfer-config`. Apache-2.0, 2.3K stars, 423 forks, 164 commits, with 91 open issues — early but the most concretely useful long-context cost lever posted this week.