Capped at Three

A model has been training calmly until one weight, $w$, lands somewhere very steep. For this problem, treat the loss as depending on that weight alone:

$L(w) = 5w^2, \qquad \frac{dL}{dw} = 10w$

The lowest loss is at $w = 0$. The weight is currently $w = 1$, and it is trained by gradient descent with learning rate $0.3$:

$w \leftarrow w - 0.3\, g$

where $g$ is the gradient used for the update.

With plain gradient descent, $g = 10w$ and the weight goes $1, -2, 4, -8, 16, \dots$ Each update overshoots further than the last, flipping sign and growing, and the run is heading for NaN.

The team switches on gradient clipping with a limit of $3$. Before every update, if $|g| > 3$ the gradient is replaced by $3$ with the same sign, so a gradient of $10$ becomes $3$ and a gradient of $-7$ becomes $-3$. If $|g| \le 3$ it is left exactly as it is. The loss, the learning rate and the starting point $w = 1$ all stay the same.

With clipping on, what does $w$ do in the long run?