Incognition logo
Home
Learn
Practice
Leaderboard
0
VisualizationsSequences, Language & Attention

Transformer Block

Click the four parts of one transformer block — self-attention, feed-forward and two add & normalize steps — to see what each does.

Tap a piece to see what it does

Four pieces, always in this order. Everything else in a transformer is this block, repeated.

Customize