A two-layer linear network runs the forward pass x→W1→h→W2→yx \rightarrow W_1 \rightarrow h \rightarrow W_2 \rightarrow yx→W1→h→W2→y, with
W1=[2013]W2=[1−121]W_1 = \begin{bmatrix} 2 & 0 \\ 1 & 3 \end{bmatrix} \qquad W_2 = \begin{bmatrix} 1 & -1 \\ 2 & 1 \end{bmatrix}W1=[2103]W2=[12−11]
So h=W1xh = W_1 xh=W1x and y=W2hy = W_2 hy=W2h.
Backpropagation reaches the output carrying the gradient
∂L∂y=[12]\frac{\partial L}{\partial y} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}∂y∂L=[12]
Propagate it all the way back to the input. What is ∂L∂x\dfrac{\partial L}{\partial x}∂x∂L?
Select all that apply.
[−211]\begin{bmatrix} -2 \\ 11 \end{bmatrix}[−211]
[113]\begin{bmatrix} 11 \\ 3 \end{bmatrix}[113]
[162]\begin{bmatrix} 16 \\ 2 \end{bmatrix}[162]
[51]\begin{bmatrix} 5 \\ 1 \end{bmatrix}[51]