A deep network is built entirely from sigmoid layers. At the current parameters the stack happens to be perfectly uniform: every layer carries the same weight , and every layer's sigmoid output is
Backpropagation multiplies in one factor per layer as it travels downward. For a sigmoid layer that factor is
and the sigmoid's derivative is given by
The loss hands the top of the stack a gradient of magnitude . After that gradient has passed down through of these layers, the magnitude reaching the weights below them is
The team calls a layer dead once the gradient arriving at it has fallen strictly below — at that size, multiplying by any sane learning rate moves the weights by nothing.
What is the smallest whole number at which the gradient has gone dead?
Give your answer as a whole number of layers.