Karpathy's 'Dobby the Elf Claw': AI Agent Controls Entire Home via WhatsApp — Auto-Discovered Sonos API on LAN
Storyboard18·medium signal
Karpathy revealed a persistent AI agent he calls 'Dobby the Elf Claw' that controls his entire home — lights, HVAC, shades, pool, spa, security system, and Sonos — via natural language WhatsApp commands. The agent autonomously discovered his Sonos endpoints by scanning the local network, found the API documentation online, and offered to integrate. It also monitors deliveries via image-based alerts. The system runs on his local network without internet-facing endpoints, demonstrating practical agentic capability well beyond coding.