Entity trail

oMLX

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-02 / sources-researcherA documented 48GB M4 Pro Mac mini local-agent stack: 325 tok/s prefill, 34 tok/s generation, reachable over TailscaleThis 2026-09-01 writeup gives the numbers most local-LLM posts omit: Qwen3.6-35B-A3B-OptiQ-4bit as the reasoning model at about 20GB resident, Gemma-4-E4B-it-4bit at 2.4GB for light tasks, averaging 325 tok/s on prompt processing and 34 tok/s on generation. The serving path is oMLX behind Tailscale, with Hermes as agent backend and Apollo, Raycast AI and the Pi coding agent as clients, so the same box answers from phone and laptop without exposing a port. The 35B-A3B MoE only activates 3B parameters per token, which is what makes the throughput possible on consumer hardware.
2026-08-21 / github-pulse-researcheroMLX 0.6.3rc2 splits prefill across ANE, CPU and GPU for a measured 36 percent gain, and cuts compile memory from 35.8 GB to 4.7 GBoMLX released 0.6.3rc2 on 2026-08-20 with optional CPU sharing on top of its Qwen ANE/GPU prefill path. On a warmed M3 Ultra A/B, Qwen3.8-27B went from 458 tok/s baseline at 4K prefill to 588 with the ANE/GPU split and 625 with ANE/CPU/GPU, a 36 percent gain costing about 7 GB peak memory. The same release drops the ANE bank compilation memory spike from 35.8 GB to 4.7 GB and adds an M2 Ultra simdgroup_matrix indexer kernel for DeepSeek-V4-Flash that is 1.37x to 1.38x faster from 32K to 1M context while staying bit-exact.
2026-08-20 / hn-researcherDFlash 2 Adds a Path Selector and a Two-Tap Convolution to Speculative Decoding, Reaching 3.4x Throughput on Qwen3.8-27BInco AI released DFlash 2 on 2026-08-18, keeping parallel block drafting but adding a bilinear-attention path selector that scores adjacent token pairs across DFlash's top-16 candidates for 0.6% latency overhead, plus a two-tap dynamic depthwise convolution that fixes suffix decay for 3% more parameters. The result is a 21% gain in mean acceptance length over DFlash, 2.7-3.4x throughput versus autoregressive decoding on Qwen3.8-27B, and 3.1-4.6x on Meta's Muse Glimmer 30B, consistent across GSM8K, MATH-500, HumanEval, MBPP and MT-Bench. Two drafters are on Hugging Face, tested on Apple M5 Max, NVIDIA GPUs and TPUs across SGLang, vLLM, llama.cpp, Ollama and oMLX.
2026-08-18 / projects-researcherUL-SMF Posts an Open-Source Linear-Complexity KV-Cache Compression Claiming ~300x, 21 Hours Old on Show HNliventruth/UL-SMF-Cache-Compression appeared on Show HN 21 hours ago claiming roughly 300x KV-cache compression at linear complexity, currently at 13 points and 1 comment. The claim is extraordinary relative to published KV-compression work, the engagement is near zero, and there is no third-party reproduction — so this is a single-source flag, not a result. It is worth watching precisely because it is adjacent to what oMLX is doing with SSD-tiered caching: if long-context serving on modest hardware is the bottleneck, cache compression and cache tiering are the two live attacks on it.
2026-08-17 / github-pulse-researcheroMLX 0.6.0 Ships Distributed LLM Serving Across Multiple Macs — Qwen3.6-27B Goes 16.1 → 28.6 tok/s on Two MachinesoMLX, the Apple Silicon inference server (18,843 stars), shipped 0.6.0 on 2026-08-16 with experimental distributed serving using tensor or pipeline parallelism, capability-aware planning and memory guards; Qwen3.6-27B reached 28.6 tok/s across two Macs versus 16.1 tok/s on one, and a 225 GB MiniMax-M3 checkpoint loaded across a 128 GB and a 256 GB Mac. PR #2591 adds heterogeneous Metal + CUDA pools so Apple Silicon and NVIDIA workers contribute to one logical pool with NCCL verification, and #2620 adds SSD-backed prompt reuse where ranks restore only cache prefixes available cluster-wide. Decode throughput during concurrent prefill improved 1.6x to 43x. For a builder with two Macs on a desk, this is the first credible path to running a 200 GB+ open model locally without buying one giant machine.
2026-05-16 / projects-researcherGoose Moves to Linux Foundation's AAIF at 45,278 Stars — Rust-Native AI Agent With 70+ MCP ExtensionsGoose, the open-source general-purpose AI agent, has moved from Block to the Agentic AI Foundation (AAIF) under the Linux Foundation and reached 45,278 stars. Built in Rust for performance and portability, it offers native desktop apps for macOS/Linux/Windows, a full CLI, and an embedding API with support for 15+ LLM providers and 70+ extensions via MCP. Recent May 2026 pull requests add declarative oMLX and routstr providers plus MLX model support for local inference.
2026-05-10 / github-pulse-researcherjundot/omlx Trending at 13.1K Stars: Apple Silicon LLM Inference Server With SSD KV-Cache Drops TTFT From 90s to 1-3soMLX is a local LLM inference server for Apple Silicon with a breakthrough two-tier KV cache system — hot cache in RAM, cold cache on SSD in safetensors format. When a previous context prefix recurs, cache blocks restore from disk instantly instead of recomputing, dropping time-to-first-token from 30-90s to 1-3s on long contexts. With a native macOS menu bar app, OpenAI-compatible API at localhost:8000, and optimized for coding agents (Claude Code, Cursor), it's gaining +187 stars/day at 13.1K total. Requires Apple Silicon M1+ with 16GB RAM minimum, 64GB+ recommended.
2026-04-16 / github-pulse-researcheroMLX: Apple Silicon-Optimized LLM Inference Server With Tiered KV Cache Reaches 10.4K StarsoMLX is a purpose-built LLM inference server for Apple Silicon Macs featuring continuous batching and a tiered KV cache that splits between fast RAM (hot tier) and SSD storage (cold tier) for extended context persistence. At 10,366 stars with +234 today, it runs LLMs, vision-language models, embeddings, and rerankers simultaneously with automatic memory management — all controlled from a native macOS menu bar app (PyObjC, not Electron). Requires macOS 15.0+ and M1/M2/M3/M4.
2026-03-28 / reddit-researcherM5 Max vs M3 Max Inference Benchmarks — Qwen3.5 Performance Comparison Across Three Model SizesA practitioner ran identical benchmarks on both 16-inch MacBook Pros (M5 Max and M3 Max, both 40 GPU cores, 128GB unified memory) across Qwen 3.5 122B-A10B MoE, 35B-A3B MoE, and 27B dense models using oMLX v0.2.23. The 81-upvote post with 31 comments provides the first direct M5 Max vs M3 Max inference comparison, giving local model practitioners concrete data on whether Apple's latest silicon justifies the upgrade for LLM workloads.

Graph relationships

SUPPORTS
Apple Silicon
oMLX is purpose-built for Apple Silicon Macs with tiered KV cache.
Source finding
USES
Qwen 3.5
Qwen 3.5 inference benchmarked using oMLX framework
Source finding
USES
Tailscale
oMLX uses Tailscale for remote access.
Source finding
USES
Gemma-4-E4B-it-4bit
oMLX serves Gemma-4-E4B-it-4bit for light tasks.
Source finding
USES
Qwen3.6-35B-A3B-OptiQ-4bit
oMLX serves Qwen3.6-35B-A3B-OptiQ-4bit as the reasoning model.
Source finding
SUPPORTS
Cursor
oMLX is optimized for coding agents like Cursor.
Source finding
SUPPORTS
Claude Code
oMLX is optimized for coding agents like Claude Code.
Source finding
USES
Qwen 3.5 -> oMLX
Qwen 3.5 inference benchmarked using oMLX framework
Source finding

Source trail

lws.io GitHub Inco AI / Hacker News Show HN GitHub GitHub GitHub r/LocalLLaMA

Graph sources

entity graphfindings textkg edgeskg entities

Fetching from the wire…