SELU Activation

SELU is ELU with two very specific numbers bolted on:

\text{SELU}(x) = \lambda \cdot \begin{cases} x & x > 0 \\ \alpha\,(e^{x} - 1) & x \le 0 \end{cases}

with

\lambda = 1.0507009873554805, \qquad \alpha = 1.6732632423543772

Task: write selu(values) returning one output per input, each rounded to 4 decimal places.

Compute the ELU part first with that fixed alpha, then multiply the whole thing by lambda. The scale applies to both branches, positive and negative.
alpha and lambda are constants here — the function takes no extra arguments.
x = 0 maps to exactly 0.0.

Those two ugly constants are not arbitrary: they're solved for, so that activations with mean 0 and variance 1 going into a layer come out with mean 0 and variance 1 again. Stack SELU layers and the signal neither explodes nor fades as it goes deeper — the network is self-normalising, which is where the extra S comes from. Change either constant and that property quietly disappears, which is why you'll always see them written out to full precision.

SELU Activation

Examples

Hints