A network stacks six identical layers. For k=1,…,6, layer k computes
ak=tanh(zk),zk=wak−1,w=1.25
and the same weight w is shared by every layer. During one backward pass it happens that tanh(zk)=0.6 at every layer. Recall that
dzdtanh(z)=1−tanh2(z)
The gradient arriving from the loss at the top of the stack is ∂a6∂L=5.
What is ∂a0∂L?