Two Relations at Once

Softmax forces a head to commit to its single highest-scoring position, so one head can select cat or quietly but never both. Four heads mean four such selections, and two of them can be spent on the two words we want.

A head of width $64$ needs far more projection parameters than four heads of width $16$, so the wide head spends everything it has on one oversized set of projections. Multi-head attention is fundamentally a cost-saving device, and only the cheaper setup has capacity left over to represent a second relationship.

At width $64$ the dot products are large enough that the single head's softmax saturates into a near-one-hot spike, so it cannot spread weight over two positions at all. Each width-$16$ head produces smaller dot products and so retains the ability to spread.

A single head produces one weight distribution over the five positions, applied across all 64 dimensions. Those weights sum to 1, so $0.9$ on cat leaves at most $0.1$ for quietly. Four heads produce four independent distributions: one can put $0.9$ on cat while another puts $0.9$ on quietly, in separate slices of the output.