Eigenvalues and Eigenvectors

Eigenvalues and eigenvectors reveal the “hidden structure” of a linear transformation. They describe the special directions where the transformation simply scales, without rotating or distorting.

Let $A$ be a square matrix. A non-zero vector $\mathbf{v}$ is an eigenvector of $A$ if:

$$A\mathbf{v} = \lambda \mathbf{v}$$

where $\lambda$ (lambda) is a scalar called the eigenvalue.

In plain terms: when you apply $A$ to an eigenvector, the result is just a scaled version of that same vector. The direction stays the same (or flips if $\lambda$ is negative). Only the length changes.

Solve the characteristic equation:

$$\det(A - \lambda I) = 0$$

This gives you the possible $\lambda$ values. Then for each $\lambda$, solve $(A - \lambda I)\mathbf{v} = \mathbf{0}$ to find the eigenvectors.

• $\lambda = 1$: the vector is unchanged
• $\lambda = 2$: the vector is stretched to double length
• $\lambda = -1$: the vector is flipped (reversed direction)
• $\lambda = 0$: the vector is squashed to zero (the transformation collapses that direction)

In the diagram: the matrix $A$ scales the x-direction by 2 (blue eigenvector doubles in length) and the y-direction by 0.5 (green eigenvector shrinks to half). The dashed arrows show the original vectors, the solid arrows show the result after applying $A$. Every other vector would get rotated and distorted, but eigenvectors just scale cleanly.

Example 1: Diagonal Matrix

$$A = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}$$

For $\mathbf{v} = \langle 1, 0 \rangle$: $A\mathbf{v} = \langle 2, 0 \rangle = 2\langle 1, 0 \rangle$, so $\lambda = 2$.

For $\mathbf{w} = \langle 0, 1 \rangle$: $A\mathbf{w} = \langle 0, 3 \rangle = 3\langle 0, 1 \rangle$, so $\lambda = 3$.

Diagonal matrices make eigenvalues obvious: they’re just the diagonal entries.

Example 2: Finding Eigenvalues of a 2x2

$$A = \begin{bmatrix} 4 & 1 \\ 2 & 3 \end{bmatrix}$$

Characteristic equation: $\det(A - \lambda I) = (4 - \lambda)(3 - \lambda) - (1)(2) = 0$

$\lambda^2 - 7\lambda + 10 = 0$

$(\lambda - 5)(\lambda - 2) = 0$

Eigenvalues: $\lambda_1 = 5$, $\lambda_2 = 2$.

Example 3: Geometric Meaning

If $\lambda \gt 1$: the eigenvector direction gets stretched.

If $0 \lt \lambda \lt 1$: it gets compressed.

If $\lambda \lt 0$: it gets flipped and scaled.

The three panels show what different eigenvalues do to a vector: $\lambda = 2$ doubles its length, $\lambda = 0.5$ halves it, and $\lambda = -1$ flips it to point the opposite direction.

Eigenvalues and eigenvectors are used everywhere:

• Machine learning: PCA (Principal Component Analysis) finds the eigenvectors of the covariance matrix to identify the most important directions in data
• Google’s PageRank: the dominant eigenvector of the web’s link matrix determines page importance
• Game development: stability analysis in physics simulations, animation compression, vibration modes
• Quantum mechanics: observable quantities are eigenvalues of operators

Example: In character animation, eigen-decomposition compresses large motion capture datasets while preserving the most important movements. The eigenvectors with the largest eigenvalues capture the dominant motion patterns.