ovg-project/kvcached: Virtualized Elastic KV Cache for Dynamic GPU Sharing in LLM Inference — 809 Stars

GitHub·low signal

kvcached introduces a virtualized elastic KV cache layer that enables dynamic GPU memory sharing and reallocation across concurrent LLM inference sessions, eliminating fixed memory partitioning and allowing agent workloads with large context windows to share GPU resources efficiently. The project targets the serving-side bottleneck where high-concurrency agent deployments exhaust GPU memory with separate per-session KV allocations. At 809 stars for a kernel-level infrastructure project, the traction signals real infrastructure operator adoption.