kvcached Adds Prefix Caching: Elastic KV Cache Enables Cross-Request Reuse on Shared GPUs — 881 Stars

GitHub·medium signal

ovg-project/kvcached shipped prefix caching support in April 2026, enabling automatic prefix caching (APC) for vLLM and RadixCache for SGLang with cross-request prefix reuse while maintaining elastic memory management. The system virtualizes GPU KV cache with OS-style memory abstraction so multiple LLMs share a GPU without rigid partitioning. Red Hat's Sardeenz is now building on kvcached for dynamic multi-model serving with Kubernetes/OpenShift.