Edward Yang Proposes PyTorch as a "Reference Language": LLM-Written Kernels Verified Against PyTorch Autograd

PyTorch / Hacker News·medium signalXBlueskyLinkedInCopy link

In a July 25 PyTorch compiler devlog, Edward Z. Yang argues PyTorch should serve a dual role — reference specification and implementation language — so that hand- or model-optimized kernels can be checked for equivalence rather than generated by a compiler that must guarantee peak performance on matmuls and attention itself. Concretely: PyTorch autograd code stays the reference, language models emit explicit forward-backward variants, and verifiers (bitwise tests or structural equivalence) confirm the two match, with fusions required to carry reference implementations. No version or timeline is committed, but it is a first-party statement that LLM-generated kernels are now part of PyTorch's compiler roadmap thinking.