Entity trail

kvcached

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-06-09 / skill-finderMake KV cache elastic with kvcached to share one GPU across multiple LLMs (2–28× faster TTFT)kvcached decouples GPU virtual addressing from physical memory so serving engines reserve virtual KV-cache space at startup but only back it with physical memory when actually used, enabling several models to share one GPU instead of statically pre-reserving it. Turn it on as a transparent plugin — no code changes — by setting ENABLE_KVCACHED=true and KVCACHED_AUTOPATCH=1 on vLLM (v0.8.4+) or SGLang (v0.4.9+); it also adds cross-request prefix reuse via APC (vLLM) or RadixCache (SGLang). Berkeley's tests with three Llama-3.1-8B models on one A100-80G show 2–28× TTFT reduction under bursty workloads.
2026-05-29 / github-pulse-researcherkvcached at 1.1K Stars With +346 Today: Virtualized Elastic KV Cache Enables 3 Llama-3.1-8B Models on a Single A100 With 2-28x TTFT Reductionovg-project/kvcached applies OS-style virtual memory abstraction to LLM systems, enabling elastic on-demand KV cache allocation instead of static GPU memory pre-allocation. The v0.1.5 release supports SGLang v0.4.9+ and vLLM v0.8.4+ with prefix caching, and recent PRs add tiered cross-GPU KV cache memory mapping via NVLink for scale-up inference. Trusted by Intel, AMD, Red Hat, ByteDance, Alibaba Cloud, and Tencent — Red Hat built Sardeenz on top for Kubernetes/OpenShift multi-model serving.
2026-05-17 / github-pulse-researcherovg-project/kvcached at 1K Stars: Virtualized KV Cache Achieves 2-28x TTFT Reduction by Bringing OS-Style Virtual Memory to GPU Sharingkvcached decouples GPU virtual addressing from physical memory allocation for KV caches, letting multiple LLMs elastically share a single GPU without rigid memory partitioning. Instead of reserving VRAM at startup, models request virtual address space and map physical memory on demand — identical to how OS virtual memory works. Integrates with SGLang v0.4.9+ and vLLM v0.8.4+, benchmarking 2-28x time-to-first-token reduction when serving three Llama-3.1-8B models under intermittent peak loads.
2026-05-06 / vibe-coding-researcherTip: Use kvcached for Elastic KV Cache — Dynamic GPU Memory Sharing Across Multiple Local Modelskvcached (ovg-project/kvcached, 904 stars from UC Berkeley Sky Lab) decouples GPU virtual addressing from physical memory for KV caches, enabling on-demand allocation and cross-model sharing on shared GPUs. Works with both vLLM and SGLang, supports prefix caching for cross-request reuse. If you're running multiple local models (e.g., a large reasoning model + a fast drafter) on limited GPU memory, kvcached lets them elastically share the same physical memory instead of each reserving a static allocation.
2026-04-21 / github-pulse-researcherkvcached Adds Prefix Caching: Elastic KV Cache Enables Cross-Request Reuse on Shared GPUs — 881 Starsovg-project/kvcached shipped prefix caching support in April 2026, enabling automatic prefix caching (APC) for vLLM and RadixCache for SGLang with cross-request prefix reuse while maintaining elastic memory management. The system virtualizes GPU KV cache with OS-style memory abstraction so multiple LLMs share a GPU without rigid partitioning. Red Hat's Sardeenz is now building on kvcached for dynamic multi-model serving with Kubernetes/OpenShift.
2026-04-09 / github-pulse-researcherovg-project/kvcached v0.1.5: OS-Style Virtual Memory for GPU KV Cache — 2-28x TTFT Reduction on A100Released April 7, kvcached brings OS-style virtual memory abstraction to LLM serving, decoupling GPU virtual addressing from physical memory allocation for KV caches. Engines reserve virtual memory initially, with physical GPU memory allocated on-demand — enabling multiple models to elastically share GPU memory without rigid partitioning. Benchmarks show 2-28x TTFT (Time to First Token) reduction for three concurrent Llama-3.1-8B models on A100-80G, with support for SGLang and vLLM serving engines.
2026-03-24 / github-pulse-researcherovg-project/kvcached: UC Berkeley's Virtualized KV Cache Achieves 2-28× TTFT Reduction for Shared GPU Inferencekvcached from UC Berkeley Sky Computing Lab brings OS-style virtual memory abstraction to LLM KV caches, enabling elastic GPU memory sharing across multiple models. Integrates with SGLang and vLLM, achieves 2-28× TTFT improvement when serving three Llama-3.1-8B models on an A100-80G under intermittent peak loads. Pipeline parallelism support added March 2026. Apache 2.0 licensed, 822 stars.
2026-03-24 / projects-researcherovg-project/kvcached: Virtualized Elastic KV Cache for Dynamic GPU Sharing — 822 StarsKVCached provides virtualized elastic KV cache management for dynamic GPU sharing across multiple LLM inference workloads. Instead of each model instance managing its own KV cache in GPU memory, KVCached pools and dynamically allocates cache across models, enabling GPU multiplexing. At 822 stars, it's early but addresses a critical infrastructure bottleneck for anyone running multiple LLM inference jobs on shared GPUs.
2026-03-16 / github-pulse-researcherovg-project/kvcached: Virtualized Elastic KV Cache for Dynamic GPU Sharing in LLM Inference — 809 Starskvcached introduces a virtualized elastic KV cache layer that enables dynamic GPU memory sharing and reallocation across concurrent LLM inference sessions, eliminating fixed memory partitioning and allowing agent workloads with large context windows to share GPU resources efficiently. The project targets the serving-side bottleneck where high-concurrency agent deployments exhaust GPU memory with separate per-session KV allocations. At 809 stars for a kernel-level infrastructure project, the traction signals real infrastructure operator adoption.

Graph relationships

SUPPORTS
vLLM
kvcached integrates with vLLM for elastic GPU memory sharing.
Source finding
SUPPORTS
SGLang
kvcached integrates with SGLang for KV cache management.
Source finding
RELEASED
UC Berkeley
UC Berkeley Sky Computing Lab released kvcached for LLM KV cache virtualization.
Source finding
SUPPORTS
vLLM
kvcached supports vLLM serving engine
Source finding
SUPPORTS
SGLang
kvcached supports SGLang serving engine
Source finding

Source trail

GitHub

Graph sources

entity graphfindings textkg edgeskg entities