The Same Valley, Turned

A model with two weights $w_1, w_2$ is trained on one of two loss surfaces:

$L_A(w_1, w_2) = 50\,w_1^2 + 0.5\,w_2^2$

$L_B(w_1, w_2) = 25\,(w_1 - w_2)^2 + 0.25\,(w_1 + w_2)^2$

Both are the same narrow valley: steep across, shallow along, with its bottom at $(0, 0)$ where the loss is $0$. In A the valley floor runs along the $w_2$ axis. B is exactly A turned by 45° about the bottom, so its floor runs along the diagonal $w_1 = w_2$.

Each run starts at the matching point: A at $(1,\ 7)$, and B at $(4\sqrt2,\ 3\sqrt2) \approx (5.66,\ 4.24)$. In both cases that is 1 unit up the steep wall and 7 units along the floor from the bottom, with a starting loss of $74.5$.

For reference, the gradients are

$\nabla L_A = \big(100\,w_1,\ \ w_2\big)$

$\nabla L_B = \big(50(w_1 - w_2) + 0.5(w_1 + w_2),\ \ -50(w_1 - w_2) + 0.5(w_1 + w_2)\big)$

Three optimizers were raced on valley A. The table shows the number of steps until the loss fell below $0.01$ and stayed there:

Momentum keeps a velocity $v$ that starts at $0$. Each step it updates $v \leftarrow 0.9\,v - 0.01\,\nabla L$ and then moves the weights by $v$. Adam is the standard version, with bias correction on.

The same three optimizers, with identical settings, are now raced on valley B from its matching start.

Which prediction for valley B is right?