Statistics 5101 (Geyer, Spring 2022) Examples: Eigenvalues and Eigenvectors

The Spectral Decomposition

A matrix M is symmetric if M = M^T, that is, if m_ij are the components of M, then m_ij = m_ji for all i and j.

If M is a symmetric matrix, then there exists a spectral decomposition

M = O D O^T

where O is an orthogonal matrix and and D is a diagonal matrix.

A matrix O is orthogonal if

O O^T = O^T O = I

where I is the identity matrix.

A matrix D is diagonal if only the diagonal components are nonzero, that is, if d_ij are the components of D, then d_ij = 0 unless i = j.

An n × n matrix I is the identity matrix of dimension n if it is diagonal and all of its diagonal elements are equal to one.

Eigenvalues and Eigenvectors

Diagonal elements of D in the spectral decomposition are called eigenvalues of M. Columns of O in the spectral decomposition are called eigenvectors of M. Hence the spectral decomposition is also called the eigenvalue-eigenvector decomposition.

Positive Definite and Positive Semidefinite Matrices

A symmetric matrix M is positive semidefinite if

b^T M b ≥ 0, for all vectors b.

A symmetric matrix M is positive definite if

b^T M b > 0, for all nonzero vectors b.

For any symmetric matrix M we can write

b^T M b = b^T O D O^T b = c^T D c

where

c = O^T b
b = O c

Thus we see that M is positive definite (respectively, positive semidefinite) if and only if D has the same property.

Summary: a symmetric matrix is positive definite if and only if all its eigenvalues are strictly positive, and a symmetric matrix is positive semidefinite if and only if all its eigenvalues are nonnegative.

Calculating Eigenvalues and Eigenvectors

Try 1

Since all the eigenvalues are positive, this matrix is positive definite.

Try 2

Since not all the eigenvalues are nonnegative, this matrix is not positive semidefinite.

Try 3

Since all the eigenvalues are nonnegative, this matrix is positive semidefinite. Since one of the eigenvalues is zero, this matrix is not positive definite.

Zero? How is 4.440892e-15 zero? Well, computer arithmetic is not exact, having only about 16 decimal places of precision, so there is no way to tell whether a number this small — relative to the numbers in the problem statement — is zero or not.