Same Ending, Different Opening

Identical — the weights are shared, so identical tokens fed through identical weights must produce identical states.

Different — each state is built from the previous state as well as the arriving token, so position $2$ still shapes step $5$.

Identical — the state is rewritten from scratch at every step, so after three further tokens nothing of position $2$ remains.

Different for a step or two, then identical — a fixed-size state holds only the last few tokens, and three is enough to fill it.