Entity trail

GGUF

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Showing the first 40 findings. More graph evidence exists in the corpus.

Corpus findings

2026-07-11 / sources-researcherKivarro: A New Open-Source Rust/Tauri 'Command Center' for Local LLM InferenceTrending on r/LocalLLaMA this week, Kivarro (AKMessi/kivarro) is a cross-platform Tauri v2 + SvelteKit desktop app that wraps llama.cpp and mistral.rs behind a dense instrumentation-style UI — model registry, hardware-fit, GGUF metadata indexing, live NVIDIA VRAM telemetry, RAG bases, and a local OpenAI-compatible API. It's early (baseline shell, single-maintainer) but notable as a design-forward alternative to the usual local-inference UIs. For anyone running GGUF models locally, it's a fresh option to watch.
2026-07-01 / thought-leaders-researcherSimon Willison Runs Ornith-1.0 Locally — an Open Coding Model That Writes Its Own RL Training ScaffoldIn a June 29 hands-on, Simon Willison ran DeepReinforce's new MIT-licensed Ornith-1.0 (the 35B MoE GGUF, ~20GB) in LM Studio wired to his Pi harness and reported it drives an agent loop across many tool calls proficiently. The model's novelty is self-scaffolding: it writes the training harness that guides its own reinforcement-learning improvement, and it's built on pretrained Gemma 4 and Qwen 3.5. For builders, it's a credible local, open alternative for agentic coding that runs on a single high-memory machine.
2026-06-25 / reddit-researcherr/LocalLLaMA Benchmark Breaks the 'q8_0 KV Cache Is Lossless' Folk Wisdom — Gemma 4 Degrades ~9x More Than Qwen 3.6A widely-shared r/LocalLLaMA benchmark measured token-by-token KL divergence across ~250K tokens while varying only KV-cache precision (f16/q8_0/q4_0) on the same BF16 GGUF. Gemma 4 26B-A4B jumped to KL 0.377 at q8_0 (1.088 at q4_0), while both Qwen 3.6 models stayed under 0.04 at q8_0 and remained usable even at q4_0. The takeaway for local builders: KV-cache quantization is model-specific, not a universal 'always lower precision until it hurts' dial.
2026-06-22 / sources-researcherUnsloth Squeezes GLM-5.2's 1.51TB Down to 217GB at 1-Bit — Plus ~2x-Faster Gemma 4 MTPUnsloth released GGUF quantizations compressing GLM-5.2's 1.51TB BF16 weights to 217GB at 1-bit, bringing the strongest text-only open model into reach of a single high-memory local rig. The same update adds Gemma 4 MTP support with auto speculative decoding for ~2x faster inference. For builders wanting frontier-open quality without API dependence, this is the practical 'can I actually run it locally' answer.
2026-06-15 / hn-researcherAI Coding at Home Without Going Broke Hits #1 on Hacker NewsA builder's guide to running capable local coding agents on consumer hardware drew 346 points and 282 comments, topping HN. The discussion centered on quantized open-weight models, GGUF setups, and avoiding $200+/mo API bills — a recurring practitioner anxiety as frontier API pricing climbs.
2026-06-07 / sources-researcherGoogle Ships Gemma 4 QAT Checkpoints — Smallest Model Drops From 11.4GB to ~1.1GBGoogle DeepMind released Quantization-Aware Training checkpoints for all Gemma 4 sizes, shrinking the smallest model from 11.4GB to 1.1GB (0.84GB text-only) with up to ~72% lower VRAM and 2x faster inference on mobile NPUs while preserving quality. The release includes Q4_0 GGUF for llama.cpp, a new mobile-specialized quant format, and compressed tensors for vLLM, with same-day Ollama and vLLM support. Notably, Unsloth's Daniel Han flagged that naive QAT→Q4_0 conversion loses accuracy while dynamic GGUF recovers it — a practical gotcha for anyone deploying these locally.
2026-05-27 / reddit-researcherPrismML Releases Bonsai Image 4B: 1-Bit Text-to-Image Diffusion Transformers Running in Browser at 0.93GBPrismML released Bonsai Image 4B, binary (1-bit) and ternary text-to-image diffusion transformers that achieve 8.3x size reduction (0.93GB vs ~16GB for FLUX.2 Klein 4B) while retaining ~95% of full-precision image quality. The models run entirely locally — 9.4 seconds for 512x512 on iPhone 17 Pro Max, 6 seconds on Mac M4 Pro — and even support in-browser inference via WebGPU. Released under Apache-2.0 on HuggingFace with GGUF, MLX-1bit, and safetensors formats; 514 upvotes and 68 comments on r/LocalLLaMA.
2026-05-26 / vibe-coding-researcherNuExtract3: Open-Weight 4B Vision-Language Model for Structured Extraction and OCR Under Apache 2.0NuMind released NuExtract3, a 4B-parameter VLM based on Qwen3.5-4B that handles both structured information extraction (JSON template-driven) and image-to-Markdown conversion for documents, scans, receipts, forms, and tables. Available in GGUF and MLX quantized formats for local deployment, it's self-hostable and directly useful in agentic coding pipelines for RAG preprocessing and document understanding. The model supports multilingual documents and offers both reasoning and non-reasoning inference modes.
2026-05-26 / reddit-researcherQwen3.5 35B A3B Heretic Uncensored Release Preserves All 785 MTPs — Immediate Community ResponseWithin hours of the Financial Times investigation into Heretic, the r/LocalLLaMA community (120 upvotes) shared a new Qwen3.5-35B-A3B uncensored model built with Heretic that preserves all 785 native Multi-Token Predictions. Available in Safetensors, GGUF, NVFP4, and GPTQ-Int4 formats on Hugging Face, this release demonstrates the speed at which decensored variants appear for new models — directly illustrating the FT's policy concerns about guardrail enforcement in open-weight AI.
2026-05-17 / reddit-researcherGemma 4 Ortenzya: Writing-Focused Finetune Aims to Fix Prose Quality and TranslationA second Gemma 4 31B creative finetune dropped on HuggingFace targeting natural English prose, creative writing, translations, and roleplay. Available in Safetensors and GGUF, the 55-upvote release (22 comments) focuses specifically on improving writing quality where base Gemma 4 falls short, rather than just removing refusals like other uncensored variants.
2026-05-17 / reddit-researcherG4-Meromero-31B: Gemma 4 Uncensored Creative Finetune Drops with 15/100 Refusal RateA new finetune of Google's Gemma 4 31B designed for creative tasks launched on HuggingFace, achieving a KL divergence of just 0.0100 from the base model with only 15/100 refusals. Available in both Safetensors and GGUF formats, the 87-upvote post (38 comments) signals the Gemma 4 ecosystem is rapidly developing uncensored creative variants alongside the Qwen ecosystem.
2026-05-16 / arxiv-researcherForgetting That Sticks: Quantization-Permanent Unlearning via Circuit AttributionStandard unlearning evaluations measure behavioral suppression at full precision immediately after training, but real deployments quantize models — where supposedly deleted knowledge can resurface. This paper introduces circuit attribution to identify and surgically ablate knowledge-encoding subnetworks, producing unlearning that persists through GPTQ, AWQ, and GGUF quantization pipelines. Critical for compliance with data deletion mandates (GDPR right-to-be-forgotten) in production-quantized LLM deployments.

Graph relationships

SUPPORTS
Pi
Pi supports GGUF model format.
Source finding
SUPPORTS
Pi -> GGUF
Pi supports GGUF model format.
Source finding

Source trail

AKMessi/kivarro simonwillison.net Unsloth's changelog stephen.bochinski.dev Source: Google / MarkTechPost Five days of independent benchmarks MoonshotAI's 1T-parameter open-weight model NuMind released NuExtract3 The research preview MTP layers OpenBMB's release The DeepSeek-native terminal agent

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…