What a Wider Vector Bought

The context vector is not the limiting factor: the 1024-wide model still loses 9.1 points from the shortest bucket to the longest, so whatever is causing the fall-off must lie elsewhere in the encoder or the decoder.

How much must be squeezed through one fixed-size vector is what hurts long sentences — the extra room pays off exactly where the load is heaviest, and quality still falls because 1024 is also fixed.

Longer sentences contain rarer words, so the decoder's vocabulary is what limits quality here; the gap between the two columns is a training artefact rather than an effect of the context width.

The wider vector helps by about the same amount at every length, which shows extra capacity produces a generally better model rather than rescuing long inputs specifically.