Clip the Gradient, Then Find Its Biggest Step

A training run keeps exploding, so the team switches on gradient clipping.

At one particular step, the gradient over the model's five parameters is

$g = (7,\; -5,\; 4,\; -3,\; 1)$

Three different sizes of the same vector show up in this problem, so here they are side by side:

The clipping rule the team uses is the standard clip-by-norm rule, and it is triggered by the Euclidean norm:

If $\lVert g \rVert_2 \le c$, leave $g$ exactly as it is.

If $\lVert g \rVert_2 > c$, replace $g$ by the rescaled vector $\hat{g} \;=\; g \cdot \frac{c}{\lVert g \rVert_2}$

They set the clipping threshold to $c = 6$.

The engineer on call wants to know how far the single most-affected parameter moves after clipping — that is, the largest absolute entry of the clipped gradient $\hat{g}$, which is exactly its Chebyshev norm $\lVert \hat{g} \rVert_\infty$.

Compute $\lVert \hat{g} \rVert_\infty$.

Give your answer rounded to two decimal places — for example 1.23.