Entity trail

Sebastian Raschka

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-07-09 / thought-leaders-researcherSebastian Raschka Ports 'LLM From Scratch' to JAX and Preps 'Build a Reasoning Model (From Scratch)'On July 8 Raschka published part 34b of his from-scratch series — rebuilding the path from bigrams to GPT-2 one component at a time in JAX — while opening pre-orders for a follow-up book, 'Build a Reasoning Model (From Scratch),' teaching how reasoning LLMs actually work under the hood. The JAX port is a signal that his canonical PyTorch-first educational stack is broadening to the JAX ecosystem. For builders who learn by re-implementing, it's a fresh, dependency-free reference for both base and reasoning-model internals.
2026-06-21 / sources-researcherRaschka: VibeThinker-3B Shows Post-Training, Not Parameter Count, Drives Small-Model ReasoningSebastian Raschka's June 17 note covers VibeThinker-3B, a 3B model built on Qwen2.5-Coder-3B that reportedly punches well above its size thanks to strong post-training. The takeaway for builders: post-training increasingly determines small-model capability, not just parameter count, so a well-trained 3B can handle agent subtasks on modest hardware. Relevant for cost-sensitive pipelines that want to offload routine steps to a cheap local model.
2026-06-21 / sources-researcherSebastian Raschka Breaks Down GLM-5.2's IndexShare for Cheaper 1M-Token Sparse AttentionIn a June 18 'Ahead of AI' model note, Sebastian Raschka dissects GLM-5.2's IndexShare mechanism for cheaper 1M-token sparse (DSA) attention — the key efficiency lever behind the open-weight model's long-context inference. For builders weighing self-hosting a frontier-class coding model, the analysis explains how IndexShare keeps long-context costs down, i.e., the architectural 'why' behind GLM-5.2's pricing, from a high-signal source rather than vendor marketing. Useful before committing GPU budget to long-context self-hosting.
2026-06-18 / sources-researcherSebastian Raschka Publishes His Jan–May 2026 LLM Research Paper ListRaschka's 'Ahead of AI' reference list compiles the LLM papers he bookmarked across the first half of 2026, spanning new models, training methods, agents, reasoning, and efficiency. His standout observation: architecture work has moved beyond simply scaling transformers toward hybrid architectures and representation geometry, with Nemotron 3 Super flagged as a must-read because it documents techniques already in production. A high-signal curated entry point for builders catching up on the year's research without trawling arXiv themselves.
2026-05-29 / projects-researcherLLMs-from-scratch Crosses 96K Stars — Sebastian Raschka's PyTorch LLM Tutorial Becomes Canonical AI Education Resourcerasbt/LLMs-from-scratch has grown to 96,229 GitHub stars, cementing its position as the most popular hands-on LLM implementation guide. The Jupyter Notebook-based repository walks developers through implementing a ChatGPT-like LLM in PyTorch step by step, covering attention mechanisms, deep learning fundamentals, and finetuning. Currently trending on GitHub, reflecting sustained demand for foundational AI understanding as the field accelerates.
2026-05-17 / rss-researcherSebastian Raschka: KV Sharing in Gemma 4, Compressed Attention in DeepSeek V4 — New LLM Architecture Deep DiveSebastian Raschka published a technical analysis of the latest LLM architecture innovations: Gemma 4's cross-layer KV sharing with per-layer embeddings to reduce KV-cache memory, DeepSeek V4's Multi-Head Compressed attention, Laguna XS.2's layer-wise attention budgeting, and ZAYA1-8B's compressed convolutional attention. The piece establishes that GQA and MLA are now the norm for efficient long-context inference. Essential reading for anyone optimizing inference infrastructure.
2026-05-13 / projects-researcherrasbt/LLMs-from-scratch Crosses 94,163 Stars — Step-by-Step PyTorch LLM Implementation Becomes One of the Highest-Starred Educational AI RepositoriesSebastian Raschka's LLMs-from-scratch has reached 94,163 GitHub stars, establishing itself as one of the most popular educational repositories in the AI space. The project provides a complete step-by-step implementation of a ChatGPT-like LLM in PyTorch from scratch, serving as the de facto curriculum for engineers who want to understand transformer internals rather than just use APIs.
2026-04-18 / projects-researcherrasbt/LLMs-from-scratch Reaches 91K Stars — The Definitive PyTorch LLM Tutorial Outgrows Most Production FrameworksSebastian Raschka's LLMs-from-scratch repository has grown to 91,002 stars, implementing a ChatGPT-like LLM in PyTorch step by step from tokenization through training and inference. At 91K stars, this educational repository is now larger than many production ML frameworks, reflecting massive demand for hands-on LLM understanding beyond API-level usage. For developers who want to understand what's happening inside the models they're building on, this is the canonical resource.
2026-04-05 / news-researcherSebastian Raschka Deep-Dive: The Six Building Blocks of a Coding AgentSebastian Raschka published a detailed breakdown on April 4 of how coding agents work in practice, identifying six core building blocks: model selection, repo context management, tool design, prompt-cache stability, memory, and long-session continuity. He released a companion minimal Python harness (rasbt/mini-coding-agent) and argues that in real-world applications, the surrounding system plays as much of a role as the model itself. 205 HN points and 67 comments.
2026-04-01 / thought-leaders-researcherSebastian Raschka: Claude Code's Real Secret Sauce Is the Harness, Not the ModelML researcher Sebastian Raschka published a detailed analysis of the leaked Claude Code source, concluding that the surrounding software harness — not the underlying Claude model — is the primary driver of coding performance. Key architectural patterns: file-read deduplication, static/dynamic content caching with boundary markers, dedicated Grep/Glob/LSP tools instead of bash, subagent parallelization with shared cache, and context optimization that writes oversized tool results to disk with preview references. Raschka argues that DeepSeek, MiniMax, or Kimi could achieve similar coding performance with equivalent harness engineering.
2026-03-23 / news-researcherSebastian Raschka Publishes Visual Guide to Attention Variants in Modern LLMs: MHA, GQA, MLA, Sparse, and HybridMachine learning educator Sebastian Raschka published a comprehensive visual guide covering the full taxonomy of attention mechanisms in production LLMs — from Multi-Head Attention and Grouped-Query Attention to DeepSeek's Multi-Latent Attention, sparse attention patterns, and hybrid architectures. The piece is essential reference material for builders choosing or fine-tuning models, with visual diagrams mapping the trade-offs between memory, compute, and quality across each variant. Published in Ahead of AI newsletter.
2026-03-22 / rss-researcherSebastian Raschka: Visual Guide to Attention Variants — MHA, GQA, MLA, Sparse, and Hybrid Architectures ComparedSebastian Raschka published a comprehensive visual guide covering the full spectrum of attention mechanisms in production LLMs: Multi-Head Attention (MHA), Grouped-Query Attention (GQA), Multi-Latent Attention (MLA), sparse attention, and hybrid architectures. The guide explains the engineering rationale behind each variant — GQA reduces KV cache memory, MLA further compresses KV while preserving quality, sparse attention enables extended context — with architectural diagrams throughout. Essential reference for builders fine-tuning or selecting base models.

Graph relationships

BUILT BY
Build a Reasoning Model (From Scratch)
Sebastian Raschka is writing Build a Reasoning Model (From Scratch).
Source finding
BUILT BY
LLM From Scratch
Sebastian Raschka ported LLM From Scratch to JAX.
Source finding
MENTIONS
GQA
Sebastian Raschka published visual guide covering GQA, MHA, MLA, sparse and hybrid attention architectures
Source finding
BUILT BY
LLM Architecture Gallery
Sebastian Raschka published the LLM Architecture Gallery
Source finding
RELEASED
mini-coding-agent
Sebastian Raschka released the mini-coding-agent Python harness.

Source trail

Tom's Hardware June 18 Ahead of AI note Sebastian Raschka Alex Kim's independent analysis ccunpacked.dev sebastianraschka.com Sebastian Raschka

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues