How Do Instructions Shape Speech? Cross-Attention Attribution for Style-Captioned TTS

arXiv·low signal

Mathur, Sayed, Madha et al. adapt the DAAM attribution framework to speech diffusion, analyzing 3,600 style-caption/transcript pairs across 25 layers and 24 ODE steps in CapSpeech-TTS — the first cross-attention attribution study for style-captioned TTS. They find style tokens have lower temporal variance than content tokens (confirming global conditioning), correlate with F0 and energy, peak in early generation steps and deeper layers, with attention entropy bottoming at layer 17 where style importance peaks. Useful for interpretability and controllability work in instruction-driven TTS, though narrow in immediate deployment impact.