What the Belt Keeps

A single LSTM unit is reading a sentence. At every step its three gates have already produced their values — in a real LSTM each gate is a small network that computes its own value from the current token and the previous hidden state, but here the numbers are handed to you so you can watch what they do to the memory.

Two things travel along the chain: the cell state $c_t$, the long-term memory that runs straight through with only small controlled edits, and the hidden state $h_t$, this step's visible answer.

Each step supplies four numbers:

The memory is edited, then read out:

$c_t = f_t\, c_{t-1} + i_t\, g_t, \qquad h_t = o_t \tanh(c_t)$

Task: write lstm_run(gates, c0). gates is a list of [f, i, g, o], one entry per time step in order, and c0 is the cell state before the first token arrives. Return a list of [c_t, h_t] pairs, one pair per step, with every value rounded to 4 decimal places.

• A gate is only ever multiplied by something. $f_t$ multiplies the old cell state; $i_t$ multiplies the candidate.
• $g_t$ is a candidate, not a gate, so it is not confined to $[0, 1]$ and may be negative.
• The cell state itself is never squashed — it is a running quantity and can grow beyond $1$.
• Round only the numbers you return. The unrounded cell state is what carries into the next step.

Watch the third step of the first case: forget gate at $1$, input gate at $0$, and the cell state crosses that step completely untouched. That is the conveyor belt — and it is the reason an LSTM can hold something for hundreds of steps while a plain RNN's single memory is rewritten from scratch every time.