The Parabola's Promise

A model with a single weight $w$ has the loss

$$L(w) = w^2 + 4e^{-w}$$

and currently sits at $w_0 = 0$.

Instead of guessing a learning rate, you replace $L$ near $w_0$ by the parabola that matches it there: same height, same slope and same curvature at $w_0$.

$$q(\Delta) = L(w_0) + L'(w_0)\,\Delta + \tfrac{1}{2}\,L''(w_0)\,\Delta^2$$

Here $\Delta$ is the size of a step away from $w_0$, so $q(\Delta)$ is the parabola's prediction for $L(w_0 + \Delta)$. (This parabola is called the second-order Taylor approximation of $L$ at $w_0$.)

You then step straight to the lowest point of the parabola. Call that step $\Delta^*$, so the new weight is $w_1 = w_0 + \Delta^*$.

The parabola promised a loss of $q(\Delta^*)$ at $w_1$. The loss you actually get there is $L(w_1)$.

What is $L(w_1) - q(\Delta^*)$, the actual loss minus the promised one?

Round your answer to 3 decimal places. Carry at least five decimals through the intermediate steps.