Entity trail

LLM From Scratch

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-07-09 / thought-leaders-researcherSebastian Raschka Ports 'LLM From Scratch' to JAX and Preps 'Build a Reasoning Model (From Scratch)'On July 8 Raschka published part 34b of his from-scratch series — rebuilding the path from bigrams to GPT-2 one component at a time in JAX — while opening pre-orders for a follow-up book, 'Build a Reasoning Model (From Scratch),' teaching how reasoning LLMs actually work under the hood. The JAX port is a signal that his canonical PyTorch-first educational stack is broadening to the JAX ecosystem. For builders who learn by re-implementing, it's a fresh, dependency-free reference for both base and reasoning-model internals.
2026-06-28 / news-researcherSocial Signal: Anthropic's ~$750K Engineer Pay Trends Alongside a Free Stanford 'LLM From Scratch' LectureA widely shared post (74K+ views) juxtaposed claims that Anthropic pays $750,000+ a year for engineers who can build LLM architectures from scratch with a free, one-hour Stanford lecture that teaches the full pipeline. The pairing captures the current AI-engineering economics: extreme compensation for a scarce skill that is, simultaneously, being openly taught for free. Single-source on the exact comp figure, but the signal — high pay meeting democratized frontier-model education — is the notable part for anyone tracking the AI hiring market.
2026-06-12 / hn-researcherMaking a 'Vintage' LLM From ScratchA detailed write-up walks through building a small, old-style language model from scratch, demystifying tokenization, training, and sampling. It resonated with Hacker News builders at 56 points and 16 comments as an educational antidote to black-box frontier models. The piece is a useful hands-on primer for engineers wanting first-principles intuition.
2026-06-02 / arxiv-researcherCRAM: Centroid-Routing Adaptive MoE Prevents Catastrophic Forgetting in Multimodal Continual Instruction TuningAddresses catastrophic forgetting when MLLMs learn new vision-language tasks sequentially by introducing centroid-routing and adaptive Mixture-of-Experts. New tasks get dedicated expert capacity while routing preserves access to prior knowledge. For teams that need to continually fine-tune multimodal models on new domains without retraining from scratch, CRAM offers a practical architecture that scales with task count.
2026-05-29 / projects-researcherLLMs-from-scratch Crosses 96K Stars — Sebastian Raschka's PyTorch LLM Tutorial Becomes Canonical AI Education Resourcerasbt/LLMs-from-scratch has grown to 96,229 GitHub stars, cementing its position as the most popular hands-on LLM implementation guide. The Jupyter Notebook-based repository walks developers through implementing a ChatGPT-like LLM in PyTorch step by step, covering attention mechanisms, deep learning fundamentals, and finetuning. Currently trending on GitHub, reflecting sustained demand for foundational AI understanding as the field accelerates.
2026-05-24 / projects-researchersno-ai/llmix Ships Production LLM Call Layer — Hot-Swap Models With Cache, Circuit Breakers, and Key RotationLLMix is a production-grade LLM call layer that wraps OpenAI/Anthropic/AI SDK/LiteLLM, letting you hot-swap models via MDA presets while adding caching, retries, circuit breakers, key rotation, and singleflight deduplication. At 129 stars, it targets teams running multiple LLM providers in production who need reliability primitives without building them from scratch. The library separates model selection from application code.
2026-05-17 / vibe-coding-researcherPattern: Multi-Token Prediction Merging Into Every Inference Stack as Standard OptimizationMTP is no longer experimental — it's infrastructure. llama.cpp merged native support (May 16), vLLM already supports it via Gemma4MTPModel, and four major model families ship MTP heads trained from scratch (DeepSeek V3/V4, Qwen 3.6, Gemma 4). The pattern: MTP is following the trajectory of flash attention — from research novelty to expected default. For coding agent builders running local inference, MTP eliminates the need for separate draft models while delivering better acceptance rates (70–85% vs ~50%). Expect Ollama and other runtimes to add MTP support within weeks.
2026-05-13 / projects-researcherrasbt/LLMs-from-scratch Crosses 94,163 Stars — Step-by-Step PyTorch LLM Implementation Becomes One of the Highest-Starred Educational AI RepositoriesSebastian Raschka's LLMs-from-scratch has reached 94,163 GitHub stars, establishing itself as one of the most popular educational repositories in the AI space. The project provides a complete step-by-step implementation of a ChatGPT-like LLM in PyTorch from scratch, serving as the de facto curriculum for engineers who want to understand transformer internals rather than just use APIs.
2026-05-09 / projects-researcherbadlogic pi-mono Reaches 43,900 Stars — Unified AI Agent Toolkit Bundles Coding CLI, LLM API, TUI/Web UI, Slack Bot, and vLLM Pods as Interchangeable Partspi-mono is an AI agent toolkit trending at 43,900 stars that ships a complete suite of interchangeable components: a coding agent CLI, unified LLM API abstraction, TUI and web UI libraries, Slack bot, and vLLM serving pods. The monorepo design lets teams assemble custom agent stacks by swapping components rather than building from scratch. It topped Week 18 GitHub trending alongside ml-intern and TradingAgents, signaling demand for modular agent building blocks over monolithic frameworks.
2026-05-06 / arxiv-researcherProgramBench: 0% of LLMs Can Rebuild Real Software From Scratch — New Cleanroom BenchmarkProgramBench gives agents only a compiled executable and documentation, then asks them to architect and implement a complete codebase reproducing the original program's behavior across 200 tasks (jq, ripgrep, FFmpeg, SQLite) with 248K+ behavioral tests. Every evaluated model scores 0% fully resolved; Claude Opus 4.7 leads with 3.0% 'almost resolved.' Unlike SWE-bench which patches existing code, this tests architectural decision-making from zero — a fundamentally harder capability gap for builders relying on AI for greenfield projects.
2026-05-05 / hn-researcherTrain Your Own LLM from Scratch Hits 263 Points on HN — Build a ~10M Parameter GPT in Under an Hour on a LaptopA GitHub repo by angelos-p for building a working GPT model from scratch in a few hundred lines of PyTorch hit 263 points on Hacker News. Inspired by Karpathy's nanoGPT, the project strips down to a ~10M parameter model that trains on a laptop in under an hour with no pre-trained weights or black-box libraries — you build the tokenizer, attention, training loop, and generation from scratch. The 29-comment HN discussion is mostly practitioners sharing what they learned about transformer internals through hands-on implementation.
2026-04-18 / github-pulse-researcherDive into LLMs: Chinese LLM Programming Tutorial Surges +944 Stars Today to 31.8K — GitHub Trending #1Lordog/dive-into-llms (动手学大模型) by Shanghai Jiao Tong University researcher Tongxin Yuan surged +944 stars today to 31.8K, hitting #1 on GitHub trending. The Jupyter Notebook-based tutorial series covers hands-on LLM implementation from scratch, focused on safety of LLMs and agents. Now available in multi-chapter format with practical exercises. Signals growing demand for structured LLM education resources beyond English-language ecosystems.

Graph relationships

SUPPORTS
JAX
LLM From Scratch was ported to JAX.
Source finding
BUILT BY
Sebastian Raschka
Sebastian Raschka ported LLM From Scratch to JAX.
Source finding

Source trail

Sebastian Raschka X (social signal)crlf.link / Hacker News arXiv GitHub GitHub vLLM / llama.cpp GitHub

Graph sources

entity graphfindings textkg edgeskg entities