Incognition logo
Home
Learn
Practice
Leaderboard
0
VisualizationsNeural Networks & How They Learn

Exploding Gradients

Slide to stack more layers, each multiplying the gradient by 2.5. Since 2.5 is above 1, the total snowballs into the hundreds of thousands.

2.5
2.5
2.5
2.5
multiplied together
39
4 layers

Rates above 1 compound the other way — the gradient grows without limit.

Customize