Incognition logo
Home
Learn
Practice
Leaderboard
0
VisualizationsSequences, Language & Attention

Weighted Context

Click to multiply each word's value vector by its attention weight, one word at a time. Blending these weighted values is how attention builds its output.

the0.15×
0.20
0.10
cat0.70×
0.90
-0.40
sat0.15×
-0.30
0.70

Each value vector is scaled by its attention weight. Note it is the values being blended, not the keys.

Customize