Exploding and Vanishing Gradients in Deep Networks: The Effect of Residual Connections

arXiv·low signal

Analyzes the classic exploding/vanishing gradient problem through multiplicative ergodic theory and characterizes how residual connections alter gradient behavior. A theoretical lens that helps explain why residual architectures train stably, of marginal day-to-day use but useful background for those designing or debugging very deep networks.