Incognition logo
Home
Learn
Practice
Leaderboard
0
VisualizationsNeural Networks & How They Learn

RMSProp

Bars show the gradient, or slope, for five weights, from 4.2 down to 0.05. One shared learning rate is too big for some and too small for others.

w1
4.20
w2
0.12
w3
2.80
w4
0.05
w5
1.10

One learning rate for all of these is a problem. It is far too big for w1 and far too small for w4.

Customize