Click to add famous image networks in order, from LeNet-5 to ResNet-152, and compare their layer counts. Depth grew slowly, then shot up.
Depth crept up slowly, then jumped once skip connections made it trainable.