Read Everything, Then Write

A sequence-to-sequence model is two recurrent networks in a row. The encoder reads the whole input sequence and keeps only its final hidden state — the context vector. The decoder starts from that context and writes the output one value at a time, feeding each value it writes back in as its next input.

To keep the arithmetic visible, every token here is a single number instead of an embedding vector, so each sequence is just a list of numbers.

Both networks use the same plain RNN update, each with its own weights. With hidden size $H$ , a single input number $x$ and the previous hidden state $h$ , the new hidden state is

h^{\text{new}}_i = \tanh\Big(W^{\text{in}}_i\, x + \sum_{j=0}^{H-1} W^{h}_{ij}\, h_j + b_i\Big), \qquad i = 0, \dots, H-1

Each network arrives as a dict with "W_in" (a list of $H$ numbers), "W_h" (an $H \times H$ list of rows — row $i$ holds the weights feeding unit $i$ ) and "b" (a list of $H$ numbers). The decoder has one extra entry, "v" (a list of $H$ numbers), which turns its hidden state $s$ into the value it writes:

y = \sum_{i=0}^{H-1} v_i\, s_i

The forward pass

The encoder starts from a hidden state of all zeros and applies its update once per input number, in order.
Its final hidden state is the context. The decoder's hidden state $s$ starts equal to the context.
At each output step, update $s$ with the decoder's weights, using as input $x$ the value the decoder wrote at the previous step ( $0$ at the very first step). Then write $y$ from the new $s$ .

Task: write seq2seq(inputs, encoder, decoder, steps) and return a tuple (context, outputs): the context vector, and the list of the steps values the decoder writes. Round every number you return to 4 decimal places, but carry the unrounded values forward through the computation.

Read Everything, Then Write

Examples

Hints