Frozen at Exactly One

A tiny model has two weights and the loss

$L(w_1, w_2) = w_1^2 + (w_2^2 - 1)^2$

The lowest possible loss is $0$, reached at $(w_1, w_2) = (0,\ 1)$ and at $(0,\ -1)$.

Because of a bug in the initialisation code, $w_2$ starts at exactly $0$. $w_1$ starts at $3$. The model is trained with plain full-batch gradient descent at a learning rate of $0.05$, with no momentum. The log:

By step 200, both components of the gradient read $0.0000$.

The run is restarted from the same starting point with one change. Which change brings the loss down near $0$?