A two-layer network with a ReLU in the middle computes
with
so the input is 2-dimensional, the hidden layer has 3 units, and the output is 2-dimensional.
acts entrywise, and its derivative is wherever its input is positive and wherever its input is negative.
Evaluate at . What is the Jacobian at this point — a matrix?
Select all that apply.