Two Outputs, Three Hidden Neurons

A network reads two numbers, $x_1 = 1$ and $x_2 = 2$ . It has three hidden neurons with ReLU and two output neurons with no activation. Every neuron first forms its weighted sum $z$ : each input times its weight, plus the bias.

Input → hidden

hidden neuron	weight from $x_1$	weight from $x_2$	bias
1	$1$	$-0.5$	$2$
2	$3$	$-3$	$1$
3	$2$	$0$	$-0.5$

Hidden → output

output neuron	weight from hidden 1	weight from hidden 2	weight from hidden 3	bias
1	$2$	$-1$	$0$	$-3$
2	$2$	$3$	$-1$	$2$

The targets are $3$ for output 1 and $3.5$ for output 2. The loss is the MSE over the two outputs:

$L = \frac{1}{2}\Big[(\text{output}_1 - 3)^2 + (\text{output}_2 - 3.5)^2\Big]$

The backward pass has to carry this error from the outputs into the hidden layer. What each hidden neuron needs is its error signal

$\frac{\partial L}{\partial z}$

which says how fast the loss changes as that neuron's weighted sum is nudged. The gradients of every weight and bias feeding the neuron are built from this one number.

What error signals reach hidden neurons 1, 2 and 3?

Two Outputs, Three Hidden Neurons

Hints