Every layer of every neural network on Earth is one operation: a matrix multiplied by a vector. A 175-billion-parameter model is just billions of numbers organized into matrices that warp vectors as they flow through. Once you see matrices as transformations — not grids of numbers — you understand what deep learning actually does.
Learning Objectives
After this lesson, you will be able to:
See matrices as machines that stretch, rotate, flip, and reshape space -- like Instagram filters for numbers
Multiply a matrix by a vector and understand what the result means as a visual transformation
Connect matrix operations to how AI models process data, and understand why the order of operations matters
Interpret the determinant as the area-scaling factor of a transformation, and explain why a zero determinant means irreversible information loss
If you understood vectors, you can understand matrices. A vector is a single arrow. A matrix is a machine that bends, stretches, and rotates ALL the arrows at once. You already have the foundation -- now you are just adding one more tool to your belt.
When we multiply a matrix by a vector, we are not just doing arithmetic. We are transforming that vector -- moving it to a new location in space according to rules encoded in the matrix.
The input vector [1, 1] got mapped to [3, 3]. But what happened geometrically? The matrix stretched space unevenly -- more along the y-axis than the x-axis, and it added a shearing effect. To truly understand a matrix, you need to see what it does to the entire space, not just one vector.
Try it! Open the Python REPL (bottom-right of the screen: click Quick Actions, then Python) and type these lines yourself.
Try it: Set the matrix entries and watch space warpInteractive
Loading visualization...
Try this: Start with the identity matrix [[1,0],[0,1]] -- nothing happens. Then change the top-left entry to 2 -- watch the x-axis stretch. Then change the bottom-right to 0 -- watch the y-axis collapse. Each number in the matrix controls a specific aspect of the transformation.
Here is the key insight that makes matrices click: the columns of a matrix tell you where the basis vectors end up.
A=[2013]⇒e^1→[20],e^2→[13]
What Do You Think?
If a matrix has columns [1, 0] and [0, 1] (the identity matrix), what happens to any vector you multiply it by?
The identity matrix sends each basis vector to itself. Since every vector is built from basis vectors, every vector stays put. The identity matrix is the "do nothing" transformation -- the mathematical equivalent of a pass-through layer.
Quick check
A 2×2 matrix has columns [3, 0] and [0, 3]. What does this transformation do?
What if a column is the zero vector? Then that dimension gets collapsed. A projection matrix crushes higher-dimensional data onto a lower-dimensional subspace:
The determinant of a matrix tells you the scaling factor of the transformation. If det(A) = 2, areas double. If det(A) = 0.5, areas halve. If det(A) = 0, the transformation collapses space into a lower dimension -- information is irreversibly lost.
Determinant — Watch the Unit Square Scale (or Collapse)Interactive
If a matrix A transforms space, the inverse A^(-1) undoes that transformation:
A−1A=I
A matrix has an inverse only when its determinant is nonzero -- when the transformation does not collapse any dimension. If information is lost (determinant = 0), there is no way to recover it. This is why singular matrices are problematic in ML: they indicate redundancy or degeneracy in the data or model.
Let's see all of this come alive with real numpy. The playground below applies a 2×2 matrix to a square of points and prints the transformed coordinates. Change the matrix entries and watch how the shape morphs.
Tests · Verify identity leaves vector unchanged, scale doubles components, rotate90 maps [3,4] to [-4,3], and collapse zeroes the y-component.
Visualizing and Understanding Neural Networks
Jason Yosinski, Jeff Clune, Yoshua Bengio, Hod Lipson (2014)
Pioneering work on visualizing what neural network layers actually learn. Shows how each layer progressively transforms representations from raw pixels to abstract concepts.
⚡ Playground:Vectors & Matrices → — multiply matrices and watch vectors rotate, scale, and shear in real time.
Matrices are space-warping machines. A matrix transforms every vector in a space simultaneously through rotation, scaling, shearing, or projection, and every neural network layer is fundamentally a matrix multiplication
Columns reveal the transformation. The columns of a matrix tell you exactly where the basis vectors land after the transformation, which completely determines what happens to every other vector
Determinant measures area scaling. A determinant of zero means the matrix collapses space into a lower dimension, permanently destroying information; a negative determinant means the transformation flips orientation
Order of multiplication matters. Matrix multiplication is not commutative (AB does not equal BA), which is why the order of layers in a neural network profoundly affects its behavior
Non-linearities prevent collapse. Without activation functions between layers, stacking matrix multiplications would reduce to a single matrix; non-linearities enable complex, curved decision boundaries
What does it mean if a matrix has a determinant of zero?
Next up: Eigenvalues and SVD -- we discover the special directions that matrices stretch without rotating, and learn how to decompose any transformation into its fundamental pieces. This is the key to PCA, recommendation systems, and understanding what neural networks learn.