Token Embeddings
| The | 0.70 | 0.93 | 0.76 | 0.93 |
| cat | 0.70 | 0.93 | 0.57 | 0.76 |
| sat | 0.78 | -0.46 | 0.73 | -0.86 |
| on | 0.82 | -0.45 | -0.34 | 0.33 |
| the | -0.14 | 0.69 | 0.35 | 0.43 |
| mat | 0.96 | -0.74 | -0.04 | 0.54 |
Step 1: Token Embeddings
Each word is converted into a numerical vector (embedding). These are the starting representations that the attention mechanism will transform.
Steps