One Cell, Once Per Token

Unrolling a plain RNN draws one copy of the same cell per time step. Each copy receives two things: its own token, and the hidden state handed over by the copy before it. Every copy uses the same weights.

Here the hidden state has $n$ units, and each token has already been turned into a single number. At step $t$ the cell reads the token $x_t$ and the previous hidden state $h_{t-1}$ :

$h_t = \tanh\!\left(W_h\, h_{t-1} + W_x\, x_t + b\right)$

Task: write rnn_forward(tokens, W_h, W_x, b, h0), returning the hidden state after every step — the whole unrolled chain — with each number rounded to 4 decimal places.

argument	shape	meaning
`tokens`	list of $T$ numbers	one token per time step, in order
`W_h`	$n \times n$ list of lists	`W_h[r][c]` is the weight from unit $c$ of the previous state into unit $r$ of the new one
`W_x`	list of $n$ numbers	`W_x[r]` multiplies the token on its way into unit $r$
`b`	list of $n$ numbers	one bias per unit
`h0`	list of $n$ numbers	the hidden state before any token has been read

Return $T$ hidden states, in order: [[...h_1...], [...h_2...], ...]. h0 is not part of the output.
Round only the returned values. The unrounded hidden state is what gets handed to the next step.
The same W_h, W_x and b are used at every step. Nothing in this function depends on the position.

Two of the test cases use identical weights and the same two tokens in the opposite order — and they disagree. That asymmetry is the whole point of recurrence: what a step remembers depends on what was read before it, so a sequence is not a bag of tokens.

One Cell, Once Per Token

Examples

Hints