Verbalized Sampling: Zero-Training Fix for LLM Mode Collapse — Prompt 'Generate 5 Responses with Probabilities, Select Proportionally' Increases Output Diversity 1.6-2.1×

Context Studios·medium signalXBlueskyLinkedInCopy link

Verbalized Sampling is a prompt-only technique that combats mode collapse (reduced output diversity from alignment training) without any additional training or infrastructure. You simply change your prompt to ask the model to 'generate 5 different responses with estimated probabilities and then select proportionally.' This increases output diversity by 1.6-2.1× without quality loss, and works with all modern LLMs. The technique is immediately applicable to any pipeline where you need diverse outputs — code generation, creative writing, test case generation, or adversarial prompt testing.