Entity trail

SGLang

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-08-16 / arxiv-researcherTEMPO Shows MoE Expert-Parallel Load Balancers Optimize the Wrong Quantity Below ~160 TokensarXiv 2608.13057 (2026-08-13) measures expert time on two datacenter GPU generations and finds it is linear in neither token count (EPLB, LPLB, UltraEP) nor activated-expert count (METRO): below roughly 156-168 tokens HBM weight streaming dominates so cost attaches to activated replicas, and above it grouped GEMM rounds tokens to 128-tile M-tiles so splitting an expert adds padded compute. Real decode batches hold hot and cold experts in both regimes at once, and recorded batches show proxy dispatchers differ by 1.4-1.6x in modeled block time. TEMPO's makespan-aware dispatcher, integrated out-of-process with SGLang, gains 4-6% throughput and cuts p99 latency ~15.6% on Qwen3-235B — while showing only mechanism cost on communication-dominated DeepSeek-V3. The authors explicitly claim a phase diagram, not a universal win.
2026-08-13 / reddit-researcherAlibaba Releases Qwen3.8-2.4T-A95B Open Weights — 2.4 Trillion Parameters, 512 Experts, With Day-0 vLLM and SGLang SupportAlibaba published open weights for Qwen3.8-2.4T-A95B, a fine-grained MoE with 2.4T total parameters, 95B active per token, 512 experts and a 92-layer hybrid full/linear attention backbone — one of the largest open-weight models ever released. vLLM shipped day-0 support verified on both NVIDIA and AMD with ready-made 4-bit checkpoints (NVFP4 at 1.32 TiB for an 8xB300 node, MXFP4 at 1.45 TiB for 8xMI355X), and NVIDIA measured over 4K tokens/sec per GPU on GB300 NVL72. The top r/LocalLLaMA post of the day at 1,507 upvotes, though the HuggingFace discussion tab is dominated by complaints that the open release is text-only and stripped of the 1M context and vision that Qwen3.8-Max has behind the API.
2026-08-10 / reddit-researcherUnsloth Ships Day-One GGUF Quants for Muse Glimmer 30B Before Ollama and LM Studio Support ExistsWithin hours of Meta's release, unsloth/Muse-Glimmer-30B-GGUF appeared on Hugging Face with a dedicated guide at unsloth.ai/docs/models/muse-glimmer (115 upvotes on r/LocalLLaMA). Meta's own post lists Ollama, LM Studio, llama.cpp, MLX, ExecuTorch, vLLM and SGLang integrations as still "coming soon," so the quant community got runnable weights out ahead of the official toolchain — Meta also shipped its own GGUF at meta-models/Muse-Glimmer-30B-GGUF. The practical caveat from the thread is a recurring one: a commenter warned "I really, really hope you tested the chat_template — that has been the bane of recent releases across the board."
2026-08-06 / rss-researcherLiquid AI's LFM2.5-2.6B beats an 8B Gemma on tool use while running in under 2.5 GBLiquid AI released LFM2.5-2.6B, a 2.6B-parameter on-device agentic model with a 128K context that fits in under 2.5 GB of memory. It posts 51.87 on AIME25, 59.17 on IFBench, 56.88 on BFCLv4 and 62.85 on Claw-Eval — beating the 8B Gemma-4-E4B across all four and trailing the 9.7B Qwen3.5-9B narrowly. Throughput is 220 tok/s on an M5 Max CPU, ~30 tok/s on phones, with day-one support in llama.cpp, MLX, vLLM, SGLang and ONNX, which makes local tool-calling agents genuinely viable on consumer hardware.
2026-08-03 / reddit-researcherMiniMax-H3 Weights Finally Land on Hugging Face — a 33B Dense Video Model, Not the MoE People ExpectedMiniMax pushed the MiniMax-H3 repository live on Hugging Face within the last several hours (984 downloads at time of check), closing a promise that was still unfulfilled on August 1 when no repo or model card existed. The model card shows 33 billion parameters in a dense single-stream transformer — three modules (H3-Context-IR, H3-Base at 768p, H3-Regenerate-2K) producing 4–15 second clips at up to 2K/24fps with native 32kHz stereo audio, recommended on 4 GPUs via SGLang. It ships under the bespoke 'MiniMax H3 Community License,' not the MIT license MiniMax used on its earlier M-series checkpoints, so read the terms before self-hosting commercially.
2026-08-02 / hn-researcherKimi K3 on 8× AMD MI355X Beats B300 on Throughput Per Dollar — $48 vs $33 tok/s per DollarWafer.ai's July 31 writeup runs the 2.8T-parameter Kimi K3 at TP8 on a single MI355X node, hitting 952 tok/s aggregate, 118 tok/s single-stream decode and roughly 13k tok/s steady-state prefill after optimization work on missing PyTorch sampling functions and AITER MLA prefill kernels via SGLang and ROCm. Against a two-node TP16 B200 deployment that's 3.8× the aggregate throughput per node and over 1.3× single-stream decode; B300 delivers 1.65× higher aggregate throughput but costs 2.4× more per GPU, landing MI355X at $48 tok/s/$ versus B300's $33 at $2.50/GPU-hr. The caveat worth flagging: the numbers come from a vendor-adjacent blog, not an independent benchmark house.
2026-07-31 / github-pulse-researcherXYZ AI Lab Drops a Matched Pair of Long-Horizon Agent Repos on the Same Day — a Runtime and Its RL TrainerXYZ-AI-Lab published AxisAgentic (855 stars) and axrl (718 stars) both on 2026-07-23, and neither README mentions the other despite being two halves of one loop. AxisAgentic is a runtime that emits append-only traces reconstructing exactly what the model observed at any point, with explicit runtime markers for rollback, context compaction, and discard events — which is what makes state-faithful SFT export possible without leaking hidden history into the training example. axrl is the post-training side: SGLang rollout plus Megatron training, with PPO, GRPO/GRPO2, GSPO, TOPR and TIS objectives, black-box harness integration via an OpenAI-compatible proxy, and trajectories running 300+ turns.
2026-07-25 / arxiv-researcherWindowed-MTP: Sliding-Window Draft Attention Cuts Speculative Decoding Cost 28-44% at 1M-Token ContextAlagappan Valliappan shows that built-in Multi-Token-Prediction draft heads run full attention over the entire KV cache at every draft step, so at million-token context the 'negligibly cheap' draft dominates cost and can make deep native drafts net-negative. Applying a StreamingLLM-style sliding window plus attention sink to the draft's attention only — training-free, drop-in, lossless because the full-attention target still verifies every token — drops ~99% of draft KV entries at 1M and cuts per-decode-step cost 28% to 44% across Qwen GDN-MoE 35B/122B and a Mamba2-hybrid NoPE 120B in SGLang on a single GPU. The unread draft KV (7.7-11% of total KV at 1M) is reclaimed via a ring buffer at no acceptance or quality cost.
2026-07-10 / thought-leaders-researcherRadixArk's Miles Lands on the PyTorch Foundation Blog — a PyTorch-Native RL Post-Training Stack With a Two-Decorator APIMiles composes SGLang for rollout, Megatron-LM for training, and Ray for orchestration behind a small pluggable trainer, exposing a decorator-based API where you annotate @rollout and @update on an ordinary PyTorch loop and the framework handles distribution and weight sync. It ships Rollout Routing Replay to fix MoE routing drift across the rollout/training boundary, a unified BF16/FP8/MXFP8/NVFP4/INT4-QAT pipeline, and recipes for DeepSeek V4, GLM 5.2, Qwen3.6, Kimi K2.6 and Nemotron 3 Ultra. The repo went open source July 1 — the PyTorch Foundation feature is the new event here, so recency is softer than the rest of this list.
2026-06-30 / skill-finderSpeculative decoding is now a production default — and it now extends to reasoning stepsSpeculative decoding (a fast draft model proposes tokens, the large target model verifies them in parallel) went from research to production standard in 2026, built into vLLM, SGLang, and TensorRT-LLM, cutting time-to-first-token 2-3x at unchanged quality for single-user serving. The newer angle: techniques like SpecReason push speculation from token-level to multi-step *reasoning*, hitting 1.5-2.5x speedups while improving answer accuracy up to 9.9%. If you self-host and have tight latency SLAs, enabling it is close to free throughput.
2026-06-27 / reddit-researcherAlibaba Open-Sources Qwen-AgentWorld — a 'Language World Model' That Simulates 7 Agent EnvironmentsQwen released Qwen-AgentWorld-35B-A3B (35B total / 3B active MoE, 256K context, Apache 2.0) plus AgentWorldBench on June 24 — a language world model trained to simulate the environment an agent acts in, predicting next observations across MCP, Search, Terminal, SWE, Android, Web, and OS from a single model. A flagship 397B-A17B variant scores 58.71 on AgentWorldBench, edging out GPT-5.4 (58.25). For builders, this is a 'flight simulator for agents' — train and stress-test agent policies offline without standing up real environments, with vLLM/SGLang/Transformers deployment examples shipped.
2026-06-24 / skill-finderEnable speculative decoding only at low batch sizes — it's net-negative when the GPU is already saturatedSpeculative decoding (a small draft model proposes tokens the target verifies in parallel) delivers 2–5x latency wins, but only in memory-bound, low-batch regimes; at large batch sizes the GPU is already compute-bound and the extra draft and verification work makes inference slower than plain autoregressive decoding. The actionable rule: gate speculative decoding on batch size rather than leaving it always-on. SGLang's EAGLE implementation handles this adaptively — it tracks an EMA of accepted draft length and switches between predefined speculative-length tiers (e.g., [1,3,7]), each with a pre-captured CUDA graph so tier changes are cheap.

Graph relationships

SUPPORTS
Qwen3.8-2.4T-A95B
SGLang shipped day-0 support for Qwen3.8-2.4T-A95B.
Source finding
USES
Miles
Miles composes SGLang for rollout.
Source finding
USES
XGrammar
XGrammar is default structured output backend in SGLang
Source finding
COMPETES WITH
vLLM
SGLang serves as a drop-in replacement for vLLM
Source finding
SUPPORTS
kvcached
kvcached integrates with SGLang for KV cache management.
Source finding
SUPPORTS
kvcached
kvcached supports SGLang serving engine
Source finding
SUPPORTS
kvcached
kvcached enables RadixCache support for SGLang.
Source finding
SUPPORTS
kvcached
kvcached compatible with SGLang for prefix caching
Source finding
USES
kvcached
kvcached integrates with SGLang v0.4.9+.
Source finding
SUPPORTS
kvcached
kvcached v0.1.5 supports SGLang v0.4.9+ with prefix caching.
Source finding
USES
RadixArk
RadixArk commercializes SGLang, the open-source inference engine.
Source finding
USES
axrl
axrl uses SGLang for rollout
Source finding

Source trail

Muse Glimmer unsloth/Muse-Glimmer-30B-GGUF Meta's technical blog posted on X independent public harness 128K context, 2.6B parameters The repo went live Wafer.ai AxisAgentic Alagappan Valliappan cutting time-to-first-token 2-3x GitHub

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…