Four Places to Run the Check

A tiny network reads one number $x$ and predicts one number. It has two hidden ReLU neurons and an output neuron with no activation:

$z_1 = w_1 x + b_1, \qquad z_2 = w_2 x + b_2$

$\text{prediction} = v_1\,\text{ReLU}(z_1) + v_2\,\text{ReLU}(z_2) + c, \qquad L = (\text{prediction} - \text{target})^2$

Its parameters are

A teammate has hand-written the backward pass, which returns a gradient for all seven parameters. It has exactly one bug. For each hidden neuron's bias, the code reuses the line it has for that neuron's weight: it multiplies the neuron's error signal $\dfrac{\partial L}{\partial z}$ by the input $x$. Everything else in the code is correct.

The teammate doesn't know about the bug yet. They plan to test the code with a gradient check on one example. For each of the seven parameters in turn, they nudge it up by $0.01$ and down by $0.01$, run the forward pass both times, and compare

$\frac{L(p + 0.01) - L(p - 0.01)}{0.02}$

with the code's gradient for that parameter. On these examples no weighted sum gets anywhere near zero during the nudges, so the check measures every true slope exactly. It disagrees with the code precisely where the code's number is wrong.

They have four candidate examples:

On which of these examples would the check show at least one disagreement?