These patterns are illustrations, not a live model's output. Each head below depicts a specialisation that interpretability research has found in real transformers — coreference, subject–verb syntax, positional, previous/next-token, clause-boundary, noun-phrase (Clark et al. 2019; Voita et al. 2019). Heads only organise this way once a model is trained, and untrained attention is just noise — so showing you the real arithmetic here would teach you less, not more. For the actual computation — softmax(QKᵀ/√d_k)·V evaluated step by step — use the Self-Attention Mechanism lab.
Sentence tokens
Attention heads — click to explore
Head 0
Coreference / pronoun resolution — tracks which nouns pronouns refer to
Query (rows) × Key (columns) — attention weights