Mr. Chatterbox: Language Model Trained Entirely on 28K Out-of-Copyright Victorian British Texts
Simon Willison·low signal
Trip Venturella released Mr. Chatterbox — a 340M-parameter model trained from scratch on 2.93 billion tokens from 28,000 Victorian-era British Library texts (1837-1899) using Andrej Karpathy's nanochat framework. Simon Willison calls it 'pretty terrible' as a conversational partner (Chinchilla scaling suggests it needs ~7B tokens, more than double available data), but the experiment demonstrates a complete pipeline for training on exclusively public domain data. Fine-tuning used synthetic conversation pairs from Claude Haiku and GPT-4o-mini, creating an interesting hybrid of historical corpus and modern alignment.