Without positional encoding, "dog bites man" and "man bites dog" would look identical to a transformer — the word order would be completely invisible.
Explore the heatmap: why do nearby rows (positions) look similar?
This lab builds on concepts from:
Positional Encoding in Transformers
Positions are rows. Dimensions are columns. Each cell = sin or cos of position/frequency. Click a row to inspect that position's encoding vector.
Encoding Heatmap — click any row (position)
Nearby positions have similar encodings — this is why transformers understand word order.