Hacker News
A 41,321-Parameter WebGPU Model Highlights 75 Languages in 27.4KB, 73x Faster Than Shiki on a Stress Test
Shu Ding at Vercel Labs published gpu-lexer on September 9, 2026: a syntax highlighter with no per-language grammars, splitting source into words, whitespace, newlines and symbols and labeling each with a 41,321-parameter WebGPU model trained on 4,688,781 tokens. Minified and Brotli-compressed it is 27.4KB and covers 75 languages including mixed-language embedded script and style regions. On a warmed browser run over 10x-concatenated three.min.js it finished in 402ms against Shiki's 29.6 seconds and Highlight.js's 1.29 seconds, with 88.02% token agreement with Shiki on held-out files and 90.35% weighted across popular languages.
↳ Follow the thread