Karpathy's LLM Wiki Pattern: Stateful Knowledge That Compounds, Not Stateless RAG

Andrej Karpathy published a GitHub Gist describing a pattern I've been using without having a name for it. He calls it the "idea file" pattern, and it flips the RAG orthodoxy on its head.

The concept: instead of retrieving documents at query time (the standard RAG approach), use an LLM to pre-compile knowledge into an interlinked wiki. The LLM reads sources, extracts concepts, creates backlinks between related ideas, and updates existing pages when new information arrives. Karpathy's personal research wiki has grown to roughly 100 articles and 400,000 words.

Why this matters more than it seems. RAG is stateless. Every query starts from scratch. You embed your documents, retrieve the top-k chunks, and hope the model stitches together the right context. The wiki pattern is stateful. Knowledge compounds. When article 47 references concepts from articles 12 and 31, the interconnections are already materialized. You don't re-derive them every time.

I've been doing a version of this with my own CLAUDE.md files and the MindPattern knowledge graph. Thirty-one markdown files with wiki-link syntax creating edges between concepts. It works because the LLM can traverse pre-built relationships instead of discovering them at query time. The retrieval step becomes a graph walk, not a vector similarity search.

The timing here isn't coincidental. The Karpathy-derived CLAUDE.md skills repo hit 56,000 stars in under two weeks, gaining 42,000 stars in a single week. That's the fastest-growing developer tool artifact I've tracked this year. A single markdown file that constrains agent failure modes (wrong assumptions, over-engineering, scope creep) is outpacing actual software projects. Config-as-guardrails is becoming standard practice, and Karpathy gave it intellectual legitimacy.

What builders should do: start building your project's knowledge wiki now. Not a docs site. A living set of interlinked markdown files that an LLM maintains. Feed it your architecture decisions, your domain constraints, your "why we did it this way" context. Let the model update cross-references as your project evolves. You'll spend less time re-explaining context to your agent, and the knowledge will compound instead of evaporating between sessions.