Autoresearch with Claude Code on Production Codebase: 60 Experiments, 93% Failure Rate as Feature

r/ClaudeAI·medium signal

A practitioner applied Karpathy's autoresearch pattern to a production Django/pgvector hybrid search system (not ML training), running 60 experiments with Claude Code autonomously. 93% failure rate, but the 7% that worked produced genuine optimizations. Key insight: autoresearch on real codebases requires tighter guard rails than ML training loops because production code has irreversible side effects. Uses Claude Code's /loop with checkpoint/revert pattern.