A hidden layer in the middle of a network has three inputs (the outputs of the layer below) and two neurons, and . On one training example the inputs are
and the layer's weights are
| neuron | weight on | weight on | weight on |
|---|---|---|---|
Each neuron computes a weighted sum and passes it through its activation. Write for the weight connecting to neuron .
The backward pass has already worked its way back to this layer. Taking each neuron's activation into account, it has found the slope of the loss with respect to each neuron's weighted sum:
The layer now has two jobs. It gives each of its six weights a gradient, and it hands a share of the blame back to each of its three inputs, so that the layer below can carry the backward pass on. The blame handed to an input is the slope of the loss with respect to that input.
For input : what is the gradient of , and how much blame does this layer hand back to ?
Select all that apply.