3. LINEAR MAPPINGS AND DIAGONALIZATION

In this chapter, we first consider functions that transform elements from one vector space into another one. These functions comply with the fundamental properties of linearity, already considered above, and are therefore called linear mappings. In many instances, we are able to formulate matrices to represent linear mappings. Afterwards, we consider a concept that has many applications in the sciences and engineering: the eigenvector. Such a vector is invariant under a linear mapping (except for a constant factor called eigenvalue) and for certain linear mappings (and associated matrices) it will be possible to find a basis of the vector space such that the associated matrix is diagonal, known as the diagonalization of the matrix.

3.1 Linear mappings and matrices

3.1.1 Introduction and basic properties

Example 3.1

Let us start with an example: Let p = [x₁, y₁, z₁] be a point in the vector space ℝ³ (for simplicity, we use row vectors here). Then, there exists a unique projection to the vector space ℝ² spanned by x and y: such a point is simply q = [x₁, y₁] (in a three-dimensional Cartesian coordinate system, q lies in the plane [x, y, 0]). We regard [x₁, y₁, z₁] → [x₁, y₁] as a mapping T from ℝ³ to ℝ² and write

T: ℝ³ → ℝ² T(p) = q.

This mapping conserves linearity:

$\begin{matrix} [\begin{array}{l} x_{1} \\ y_{1} \\ z_{1} \end{array}] + [\begin{array}{l} x_{2} \\ y_{2} \\ z_{2} \end{array}] = [\begin{array}{l} x_{1} + x_{2} \\ y_{1} + y_{2} \\ z_{1} + z_{2} \end{array}] \to [\begin{array}{l} x_{1} + x_{2} \\ y_{1} + y_{2} \end{array}] = [\begin{array}{l} x_{1} \\ y_{1} \end{array}] + [\begin{array}{l} x_{2} \\ y_{2} \end{array}] \\ α [\begin{array}{l} x_{1} \\ y_{1} \\ z_{1} \end{array}] \end{matrix} = [\begin{array}{l} α x_{2} \\ α y_{2} \\ α z_{2} \end{array}] \to [\begin{array}{l} α x_{1} \\ α y_{1} \end{array}] = α [\begin{array}{l} x_{1} \\ y_{1} \end{array}], α \in R . $$

We see that linear mappings are functions defined on vector spaces that preserve linear combinations. Other names (mostly used synonymously, but strongly depending on the context) are linear transformations, linear operators, or linear maps.

Definition 3.1

Given two vector spaces V and W, we say that T : V → W is a linear mapping if ∀u, v ∈ V and ∀α ∈ ℝ it verifies:

(a) T (u + v) = T (u) + T (v) (additivity).

(b) T (αu) = αT (u) (homogeneity).

These two properties can be combined in a single statement:

T (αu + βv) = αT (u) + βT (v) ∀u, v ∈ V and ∀α, β ∈ R.

Example 3.2

The mapping D : P → P defined by D(p(x)) = p^′(x) is linear.

Proof. We show that the differentiation of polynomials is a linear mapping. Let f and g represent polynomials, written in the canonical basis of polynomials P. Then, we use knowledge of calculus: D(f + g) = D(f) + D(g) and D(αf) = αD(f) and differentiation of polynomials (as actually of any differentiable function) is linear. Note that we need to have a well-defined vector space.

Example 3.3

The mapping T : ℝ² → ℝ² defined by

$T ([\begin{array}{l} x \\ y \end{array}]) = [\begin{matrix} y \\ x \end{matrix}]$

is linear.

Proof. For all vectors $[\begin{array}{l} x \\ y \end{array}] \in R^{2}$ and all α ∈ ℝ, we have:

$T ([\begin{array}{l} x_{1} \\ y_{1} \end{array}] + [\begin{array}{l} x_{2} \\ y_{2} \end{array}]) = T ([\begin{array}{l} x_{1} + x_{2} \\ y_{1} + y_{2} \end{array}]) = [\begin{matrix} y_{1} + y_{2} \\ x_{1} + x_{2} \end{matrix}] = [\begin{matrix} y_{1} \\ x_{1} \end{matrix}] + [\begin{array}{l} y_{2} \\ x_{2} \end{array}] = T ([\begin{array}{l} x_{1} \\ y_{1} \end{array}]) + T ([\begin{array}{l} x_{2} \\ y_{2} \end{array}])$

and

$T (α [\begin{array}{l} x_{1} \\ y_{1} \end{array}]) = T ([\begin{array}{l} α x_{1} \\ α y_{1} \end{array}]) = [\begin{matrix} α y_{1} \\ α x_{1} \end{matrix}] = α [\begin{matrix} y_{1} \\ x_{1} \end{matrix}] = α T ([\begin{array}{l} x_{1} \\ y_{1} \end{array}])$

This simple example allows us to draw a link to the first chapter. The linear mapping of this example is interchanging x and y components of a vector. This can also be achieved by multiplying a matrix to the vector:

$T ([\begin{array}{l} x \\ y \end{array}]) = [\begin{array}{l} y \\ x \end{array}] = [\begin{array}{ll} 0 1 \\ 1 0 \end{array}] [\begin{array}{l} x \\ y \end{array}] .$

Hence, a linear mapping can be represented (for some relevant cases) by

T(u) = Au,

where A is a suitable matrix. We will come back to the matrix associated to a linear mapping below.

Example 3.4

The mapping T : ℝ² → ℝ² defined by

$T ([\begin{array}{l} x \\ y \end{array}]) = [\begin{matrix} y \\ x^{2} \end{matrix}]$

is not linear.

Proof. It is sufficient to show that the additivity property is not fulfilled:

while

$T ([\begin{array}{l} x_{1} \\ y_{1} \end{array}]) + T ([\begin{array}{l} x_{2} \\ y_{2} \end{array}]) = [\begin{array}{l} y_{1} \\ x_{1}^{2} \end{array}] + [\begin{array}{l} y_{2} \\ x_{2}^{2} \end{array}] = [\begin{array}{l} y_{1} + y_{2} \\ x_{1}^{2} + x_{2}^{2} \end{array}]$

Theorem 3.1

Let T : V → W be a linear mapping. Then:

(a) T(0) = 0.

(b) T (-u) = -T (u).

Observe that in part (a), the nullvector on the left hand side is the nullvector from V, on the left hand side there is the nullvector from W. In particular, part (a) means that for a linear mapping, the nullvector maps to the nullvector. In part (c), the number of vectors n is not specified.

Example 3.5

The mapping T : ℝ² → ℝ² defined by

$T ([\begin{array}{l} x \\ y \end{array}]) = [\begin{matrix} x + 1 \\ y \end{matrix}]$

is not linear since

$T ([\begin{array}{l} 0 \\ 0 \end{array}]) = [\begin{array}{l} 1 \\ 0 \end{array}] \neq [\begin{array}{l} 0 \\ 0 \end{array}] .$

3.1.2 Kernel and image of a linear mapping

Definition 3.2

Let T : V → W be a linear mapping. We define the kernel and image of T, respectively, as follows:

KerT = {x ∈ V : T(x) =0},

ImT = {T (x) ∈ W : x∈ V}.

Observe that KerT is a subspace of V and ImT is a subspace of W. Once we establish the associated matrix for a linear mapping, we will see how the kernel is analogous to the nullspace and the image to the column space of the matrix.

Theorem 3.2

Let T : V → W be a linear mapping and let {u₁, u₂, …, u_n} be a system of generators of V. Then, {T (u₁), T (u₂), …, T (u_n)} is a system of generators of ImT.

Example 3.6

Find the kernel and image of the linear mapping T : ℝ³ → ℝ³ defined by

$T ([\begin{array}{l} x \\ y \\ z \end{array}]) = [\begin{matrix} x + z \\ y \\ x + 2 y + z \end{matrix}] .$

For the vector space of this example, we know a very convenient system of generators: the canonical basis of ℝ³:

$\{[\begin{array}{l} 1 \\ 0 \\ 0 \end{array}], [\begin{array}{l} 0 \\ 1 \\ 0 \end{array}], [\begin{array}{l} 0 \\ 0 \\ 1 \end{array}]\} .$

We apply the linear mapping to these vectors and obtain three vectors

$\{T ([\begin{array}{l} 1 \\ 0 \\ 0 \end{array}]), T ([\begin{array}{l} 0 \\ 1 \\ 0 \end{array}]), T ([\begin{array}{l} 0 \\ 0 \\ 1 \end{array}])\} = \{[\begin{array}{l} 1 \\ 0 \\ 1 \end{array}], [\begin{array}{l} 0 \\ 1 \\ 2 \end{array}], [\begin{array}{l} 1 \\ 0 \\ 1 \end{array}]\} .$

Two of these vectors are linearly dependent, and hence we conclude

$Im T = Span \{[\begin{array}{l} 1 \\ 0 \\ 1 \end{array}], [\begin{array}{l} 0 \\ 1 \\ 2 \end{array}]\}$

The kernel is formed by the vectors that map to the nullvector.

$\begin{aligned} x + z = 0, \\ y = 0, \\ + 2 y + z = 0 . \end{aligned}$

From that set of equations we obtain

x = -z,
y=0,

with z being arbitrary, and therefore

$Ker T = Span \{[\begin{matrix} 1 \\ 0 \\ - 1 \end{matrix}]\}$

Before we proceed, we have to review some general classification of functions:

Definition 3.3

Let f : A → B be a function. Then,

(a) f is injective if and only if x ≠ y implies f (x) ≠ f (y) ∀x, y ∈ A.

(b) f is surjective if and only if ∀b ∈ B, ∃a ∈ A such that f (a) = b.

With this classification it is now possible to confirm the following properties for injective and surjective (and hence, bijective) linear mappings:

Theorem 3.3

Let T : V → W be a linear mapping. Then:

(a) T is injective if and only if KerT = {0}.

(b) T is surjective if and only if ImT = W.

There are alternative characterizations for injective, surjective and bijective linear mappings, provided by the following theorem:

Theorem 3.4

Let T : V → W be a linear mapping. Then:

(a) T is injective if and only if for each independent set {u₁, u₂, …, u_n}, the set {T (u₁), T (u₂), …, T (u_n)} is linearly independent.

(b) T is surjective if and only for each system of generators of V, {u₁, u₂, …, u_n}, the set {T (u₁), T (u₂), …, T (u_n)} is a system of generators of W.

(c) T is bijective if and only if for each basis of V, {u₁, u₂, …, u_n}, the set {T (u₁), T (u₂), …, T (u_n)} is a basis for W.

Notice that n for the three cases is not necessarily the same number, as the number of independent vectors can be smal ler than the dimension, and the number of generatoring vectors can be larger than the dimension, respectively.

Let us now move on to generate linear mappings from basic operations.

Definition 3.4

Let f, g : V → W be two linear mappings and λ ∈ ℝ. Then, we can define the following operations:

(a) f + g:V → W, (f + g)(x) = f(x) + g(x)

(b) λf : V → W, (λf)(x) = λf(x)

Theorem 3.5

With the operations of linear mappings, the set of linear mappings between two vector spaces V and W is itself a vector space.

As for general functions, we can also define the composition of linear mappings and find that its result is also a linear mapping, by the next theorem.

Theorem 3.6

Let f : V → W and g : W → U be two linear mappings. Then, their composition g ◦ f : V → U, defined by (g ◦ f)(x) = g (f (x)) is also a linear mapping.

We also would like to define an inverse linear mapping and, for that, we use the general result that if a function is bijective, it has a unique inverse function. In particular, if f : V → W is a bijective linear mapping, then there is a unique function f^-1 : W → V such that:

f^-1 (f (v)) = v for all v ∈ V,
f(f^-1(w)) = w for all w ∈ W.

It turns out that the inverse of a linear mapping is also a linear mapping.

Theorem 3.7

For each bijective linear mapping Let f : V → W, its inverse mapping f^-1 : W → V is also a linear mapping, i.e.,

f^-1 (αx + βy) = αf^-1 (x) + βf^-1 (y) ∀x, y ∈ W and ∀α, β ∈ ℝ.

3.1.3 Associated matrices

Here, we will see that linear mappings between finite-dimensional vector spaces have a matrix representation. In fact, given a linear mapping T : V → W, this matrix representation or associated matrix, denoted by $M_{C}^{B} (T)$ (or just $M_{C}^{B}$ for the sake of simplicity) has the useful property that

$[T (x)]_{C} = M_{C}^{B} [x]_{B}, (3.1)$

where $B$ and $C$ are bases for V and W, respectively. In other words, for each x ∈ V, the matrix $M_{C}^{B}$ should transform the coordinates of x into the coordinates of its image under T. In this way, the effect that $M_{C}^{B}$ has on [x] $B$ is analogous to the effect that T has on x.

An important observation is that $M_{C}^{B}$ depends on the bases that are chosen for the vector spaces and, of course, the mapping itself. We now discuss how to construct the associated matrix of a linear mapping relative to the bases $B$ and $C$ .

Construction of the matrix associated to a linear mapping

Let $B$ = {b₁, b₂, …, b_n} be a basis for V. Then, for each x ∈ V, we can write

x = x₁b₁ +x₂b₂ + … +x_nb_n,

which means that its vector of coordinates relative to $B$ is

$[x]_{B} = [\begin{matrix} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{matrix}] .$

If we apply the linear mapping T to both sides of x = x₁b₁ + x₂b₂ + … + x_nb_n, then we obtain

T(x) = T(x₁b₁ + x₂b₂ + … + x_nb_n),

and, due to the linearity of T, we have

T(x) = x₁ T(b₁) + x₂T(b₂) + … + x_n T (b_n).

Since taking coordinates preserves linear combinations, we deduce that the coordinates of T (x) relative to the basis $C$ are given by

$\begin{aligned} [T (x)]_{C} = {[x_{1} T (b_{1}) + x_{2} T (b_{2}) + \dots + x_{n} T (b_{n})]}_{C} \\ = x_{1} {[T (b_{1})]}_{C} + x_{2} {[T (b_{2})]}_{C} + \dots + x_{2} {[T (b_{n})]}_{C} \end{aligned}$

or, in matrix notation,

$[T (x)]_{C} = [\begin{array}{llll} {[T (b_{1})]}_{C} [\begin{array}{lll} {T (b_{2})]}_{C} & \dots & {[T (b_{n})]}_{C} \end{array}] [x]_{B}, \end{array} (3.2)$

where each [T (bi)]_C is an m × 1 (column) vector.

Comparing (3.1) and (3.2), we see that the m × n matrix

$M_{C}^{B} = [\begin{array}{llll} {[T (b_{1})]}_{C} {[T (b_{2})]}_{C} \dots {[T (b_{n})]}_{C} \end{array}] (3.3)$

is precisely the associated matrix for T that we were looking for.

Example 3.7

Let T : ℝ² → ℝ³ be a linear mapping such that

$T ([\begin{array}{l} 1 \\ 0 \end{array}]) = [\begin{array}{l} 1 \\ 2 \\ 3 \end{array}], T ([\begin{array}{l} 0 \\ 1 \end{array}]) = [\begin{array}{l} 2 \\ 1 \\ 0 \end{array}]$

and let ε₂ and ε₃ be the canonical bases for ℝ² and ℝ³, respectively. Find $M_{E_{3}}^{E_{2}} (T) \equiv M_{E_{3}}^{E_{2}}$ . Then, compute T (x) for

$x = [\begin{array}{l} 1 \\ 1 \end{array}] .$

According (3.3), the columns of $M_{E_{3}}^{E_{2}}$ are

${[T ([\begin{array}{l} 1 \\ 0 \end{array}])]}_{E_{3}} = [\begin{array}{l} 1 \\ 2 \\ 3 \end{array}], and {[T ([\begin{array}{l} 0 \\ 1 \end{array}])]}_{E_{3}} = [\begin{array}{l} 2 \\ 1 \\ 0 \end{array}] .$

Therefore,

$M_{E_{3}}^{E_{2}} = [[T {([\begin{array}{l} 1 \\ 0 \end{array}])}_{E_{3}} {[T ([\begin{array}{l} 0 \\ 1 \end{array}])]}_{E_{3}}] = [\begin{array}{ll} 1 2 \\ 2 1 \\ 3 0 \end{array}]$

In this case, T (x) can be computed directly (even if a definition for T is not explicitly given) using $M_{E_{3}}^{E_{2}}$ , because T (x) = [T (x)]_ε₃ and x = [x]_ε₂. Therefore,

$[T (x)]_{E_{3}} = M_{E_{3}}^{E_{2}} [x]_{E_{2}},$

becomes

$T ([\begin{array}{l} 1 \\ 1 \end{array}]) = [\begin{array}{ll} 1 2 \\ 2 1 \\ 3 0 \end{array}] [\begin{array}{l} 1 \\ 1 \end{array}] = [\begin{array}{l} 3 \\ 3 \\ 3 \end{array}] .$

Example 3.8

Let D : P₃ → P₃ be the linear mapping defined as

D(p(x)) = p^′(x).

Find $M_{S_{3}}^{S_{3}} (D) \equiv M_{S_{3}}^{S_{3}}$ , where S₃ = {1, x, x², x³} is the standard basis for P₃.

Note that, in this example, D is a mapping from P₃ to itself. Moreover, we are asked to use the same basis to represent the elements in P₃ as well as their images under D.

By (3.3), we have

$M_{S_{3}}^{S_{3}} = [\begin{array}{llll} [D (1)]_{S_{3}} [D (x)]_{S_{3}} {[D (x^{2})]}_{S_{3}} {[D (x^{3})]}_{S_{3}}] = [\begin{array}{lll} 00]_{S_{3}} & [1]_{S_{3}} & [2 x]_{S_{3}} \end{array} {[3 x^{2}]}_{S_{3}}] . \end{array}$

The explicit expression of $M_{S_{3}}^{S_{3}}$ is obtained by writing out the derivatives in the last equality as linear combinations of S₃ :

$\begin{aligned} 0 = 0 \cdot 1 + 0 \cdot x + 0 \cdot x^{2} + 0 \cdot x^{3} \\ 1 = 1 \cdot 1 + 0 \cdot x + 0 \cdot x^{2} + 0 \cdot x^{3} \\ 2 x = 0 \cdot 1 + 2 \cdot x + 0 \cdot x^{2} + 0 \cdot x^{3} \\ 3 x^{2} = 0 \cdot 1 + 0 \cdot x + 3 \cdot x^{2} + 0 \cdot x^{3} \end{aligned}$

Therefore,

$M_{S_{3}}^{S_{3}} = [\begin{array}{llll} 0 1 0 0 \\ 0 0 2 0 \\ 0 0 0 3 \\ 0 0 0 0 \end{array}] .$

Let us mention an interesting property of this linear mapping. Observe that the first column of $M_{S_{3}}^{S_{3}}$ consists entirely of zeros and, obviously, it is not a pivot column. Therefore, the equation

$M_{S_{3}}^{S_{3}} x = 0$

has a non trivial solution (the first column corresponds to a free variable). This means that there is at least one non zero polynomial p(x) such that D(p(x)) = 0. In other words, KerD ≠ {0}. It follows from Theorem 3.3 that D is not injective. Indeed, it is well known that for any constant k, D(k) = 0. Consequently, any polynomial of the form f (x) = p(x) + k, k ∈ ℝ, satisfies the equation

D(f (x)) = p^′(x).

Example 3.9

Let T : P₃ → P₄ be the linear mapping defined as

T (p(x)) = x p(x).

Let us compute $M_{S_{4}}^{B} (T) \equiv M_{S_{4}}^{B}$ where $B$ = {1, 1 + x, x + x², x² + x³} and S₄ = {1, x, x², x³, x⁴}.

Since we are using the standard basis for P₄, it is easy to write the explicit expression for $M_{S_{4}}^{B}$ . By (3.3),we have

$∣ M_{S_{4}}^{B} = [[T (1)]_{S_{4}} [T (1 + x)]_{S_{4}} {[T (x + x^{2})]}_{S_{4}} {[T (x^{2} + x^{3})]}_{S_{4}}]$

$\begin{aligned} = [\begin{array}{lll} [x]_{S_{4}} & {[x + x^{2}]}_{S_{4}} & {[x^{2} + x^{3}]}_{S_{4}} \end{array} {[x^{3} + x^{4}]}_{S_{4}}] \\ = [\begin{array}{llll} 0 & 0 & 0 & 0 \\ 1 & 1 & 0 & 0 \\ 0 & 1 & 1 & 0 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 1 \end{array}] . \end{aligned}$

The matrices of injective, surjective, and bijective linear mappings

It is possible to characterize the injective and surjective character of a linear mapping T : V → W in terms of the pivots of its associated matrix independently of the bases we choose for the vector spaces involved.

In the discussion below, $B$ = {b₁, b₂, …, b_n} will denote a basis for V and C will denote a basis for W.

Injectivity. By Theorem 3.4, we know that T is an injective linear mapping if and only if {T (b₁), T (b₂), …, T (b_n)} is a linearly independent set of vectors. Moreover recall that their coordinate vectors relative to any basis constitute a linearly independent set of column vectors. Hence,

{[T (b₁)]_C, [T (b₂)]_C,…, [T (b_n)]_C}

is a linearly independet set of vectors in ℝ^m with m = dim W. These vectors are the columns of the associated matrix $M_{C}^{B}$ . Therefore, the injectivity of T passes down to its associated matrix $M_{C}^{B}$ as follows: T is injective if and only if all the columns of $M_{C}^{B}$ are linearly independent, or, equivalently, all the columns of $M_{C}^{B}$ are pivot columns. Consequently, the equation

$[y]_{C} = M_{C}^{B} [x]_{B}, y \in Im T$

has a unique solution. If we compute the rank of $M_{C}^{B}$ associated with an injective linear mapping T, then we find that

$rank M_{C}^{B} = # pivots of M_{C}^{B} = # columns of M_{C}^{B} = \dim V$

This has the following important consequence:

If there is an injective linear mapping T : V → W, then

• n = dim V ≤ m because any linearly independent set in W has at most m elements.

• #columns of $M_{C}^{B}$ ≤ #rows of $M_{C}^{B}$ .

Surjectivity. Again, by Theorem 3.4, we know that T is surjective if and only if the set {T(b₁), T(b₂), …, T(b_n)} is a generating system of W. Since taking coordinates preserve linear relations, this means that the set

{[T (b₁)]_C, [T (b₂)]_C,…, [T (b_n)]_C}

is a generating system for ℝ^m with m = dim W. These vectors are the columns of the associated matrix $M_{C}^{B}$ , and therefore surjectivity translates into the statement that all rows of $M_{C}^{B}$ have a pivot (in order to avoid degenerate rows), which is the same as confirming that the equation

$[y]_{C} = M_{C}^{B} [x]_{B}, y \in Im T$

has a solution for all b ∈ W. Therefore, computing the rank of $M_{C}^{B}$ , we obtain rank

$rank M_{C}^{B} = # pivots of M_{C}^{B} = # rows of M_{C}^{B} = \dim W .$

This has the following important consequence:

If there is a surjective linear mapping T : V → W, then

• n = dim $V ⩾ m$ because any generatig set for W has at least m elements.

• #columns of $M_{C}^{B}$ #rows of $M_{C}^{B}$ .

Bijectivity. From the previous discussion, we deduce that T is bijective (T is both injective and surjective) if and only if the set of vectors {T (b₁), T (b₂), …, T (b_n)} forms a basis for W. Putting what we know about injectivity and surjectivity together

$rank M_{C}^{B} = # pivots of M_{C}^{B} = # columns of M_{C}^{B} = # rows of M_{C}^{B} .$

This has the following significant implication:

If there is a bijective linear mapping T : V → W, then

• dim V = dim W.

• $M_{C}^{B}$ is an invertible square matrix.

Let us consider a few examples.

Example 3.10

Let T : ℝ² → ℝ³ be a linear mapping such that

$T ([\begin{array}{l} 1 \\ 0 \end{array}]) = [\begin{array}{l} 1 \\ 2 \\ 3 \end{array}], T ([\begin{array}{l} 0 \\ 1 \end{array}]) = [\begin{array}{l} 2 \\ 1 \\ 0 \end{array}] .$

We already found the associated matrix with respect to the canonical bases:

$M_{E_{3}}^{E_{2}} = [\begin{array}{ll} 1 2 \\ 2 1 \\ 3 0 \end{array}] .$

Its row reduced echelon form is given by

$M_{E_{3}}^{E_{2}} \sim [\begin{array}{ll} 1 0 \\ 0 1 \\ 0 0 \end{array}] .$

We see that all columns of $M_{E_{3}}^{E_{2}}$ are pivot columns and therefore T is injective. We can also see that dim ℝ² < dim ℝ³, and hence T cannot be surjective (and, thus, not bijective).

Example 3.11

Let D : P₃ → P₃ a linear mapping defined by D(p(x)) = p^′(x). We already know the associated matrix relative to the standar basis S₃ = {1, x, x², x³}:

$M_{S_{3}}^{S_{3}} = [\begin{array}{llll} 0 1 0 0 \\ 0 0 2 0 \\ 0 0 0 3 \\ 0 0 0 0 \end{array}] .$

This matrix has less pivot columns than columns (hence, D is not injective) and less pivot rows than rows (hence, D not surjective).

Example 3.12

Let T : P₃ → P₄ a linear mapping defined by T (p(x)) = xp(x). We already know the associated matrix relative to the basis $B$ = {1, 1 + x, x + x², x² + x³} for P₃, and the standard basis S₄ = {1, x, x², x³, x⁴}:

$M_{S_{4}}^{B} = [\begin{array}{llll} 0 0 0 0 \\ 1 1 0 0 \\ 0 1 1 0 \\ 0 0 1 1 \\ 0 0 0 1 \end{array}] .$

After computing its row reduced echelon form:

$M_{S_{4}}^{B} \sim [\begin{array}{cccc} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{array}],$

we find that all its columns are pivot columns, which means that T is injective. However, dim P₃ < dim P₄, so T cannot be surjective.

3.1.4 Associated matrices and the change of basis

Whenever we compute a matrix associated with a linear mapping T : V → W, we explicitly use a fixed basis for each vector space V and W. This means that the explicit expression of the associated matrix depends on the specific choice of these bases. Nevertheless, since the underlying mapping T is independent of the choice of bases, we should expect to find a relationship between associated matrices relative to two different choices of bases. In this section, we will investigate this in detail and describe how an associated matrix varies under a change of bases. Let us start with an illustrative example.

Example 3.13

Let T : ℝ³ → ℝ² be a linear mapping defined by

$T ([\begin{array}{l} x \\ y \\ z \end{array}]) = [\begin{array}{l} x + y \\ y - z \end{array}]$

Let $B$ = {b₁,b₂,b₃} and C = {c1, c2} be bases for ℝ³ and ℝ², respectively. We will consider two cases for illustration: when B and C are the canonical bases and when they are not, and give the associated matrices for both cases.

Case 1: $B$ and $C$ are the canonical bases E3 and E2, respectively. The presentation may appear unnecessarily detailed here, but it is illustrative to see the differences with the more complicated case 2 below. We have

$b_{1} = [\begin{array}{l} 1 \\ 0 \\ 0 \end{array}], b_{2} = [\begin{array}{l} 0 \\ 1 \\ 0 \end{array}], b_{3} = [\begin{array}{l} 0 \\ 0 \\ 1 \end{array}] .$

We calculate, by prescription, T (b_i), for i = 1, 2, 3:

$T ([\begin{array}{l} 1 \\ 0 \\ 0 \end{array}]) = [\begin{array}{l} 1 \\ 0 \end{array}], T ([\begin{array}{l} 0 \\ 1 \\ 0 \end{array}]) = [\begin{array}{l} 1 \\ 1 \end{array}], T ([\begin{array}{l} 0 \\ 0 \\ 1 \end{array}]) = [\begin{array}{r} 0 \\ - 1 \end{array}] .$

Actually, the resulting vectors are already written in terms the canonical basis E2, as one can readily confirm:

${[[\begin{array}{l} 1 \\ 0 \end{array}]]}_{E_{2}} = [\begin{array}{l} 1 \\ 0 \end{array}], {[[\begin{array}{l} 1 \\ 1 \end{array}]]}_{E_{2}} = [\begin{array}{l} 1 \\ 1 \end{array}], {[[\begin{array}{r} 0 \\ - 1 \end{array}]]}_{E_{2}} = [\begin{array}{r} 0 \\ - 1 \end{array}] .$

Therefore, we have

$M_{E_{2}}^{E_{3}} = [\begin{array}{rrr} 1 1 0 \\ 0 1 - 1 \end{array}] .$

Case 2: $B$ and $C$ are now given by

$\begin{matrix} b_{1} = [\begin{array}{l} 1 \\ 0 \\ 1 \end{array}], b_{2} = [\begin{array}{l} 0 \\ 1 \\ 1 \end{array}], b_{3} = [\begin{array}{l} 1 \\ 1 \\ 1 \end{array}] . \\ c_{1} = [\begin{array}{l} 1 \\ 2 \end{array}], c_{2} = [\begin{array}{r} - 1 \\ 1 \end{array}] . \end{matrix}$

We calculate, by prescription, T (b_i), for i = 1, 2, 3:

$T ([\begin{array}{l} 1 \\ 0 \\ 1 \end{array}]) = [\begin{array}{r} 1 \\ - 1 \end{array}], T ([\begin{array}{l} 0 \\ 1 \\ 1 \end{array}]) = [\begin{array}{l} 1 \\ 0 \end{array}], T ([\begin{array}{l} 1 \\ 1 \\ 1 \end{array}]) = [\begin{array}{l} 2 \\ 0 \end{array}] .$

Now, we need to compute [T (b₁)]_C, [T(b₂)]_C, and [T (b₃)]_C. For this, we must solve the three systems of equations whose augmented matrices are:

[c₁ c₂T(b₁)], [c₁ c₂T (b₂)], [c₁ c₂T(b₃)].

All three systems can be solved simultaneously by setting up the augmented matrix

c₁ c₂T(b₁) T(b₂) T(b₃),

and reducing it to its row reduced echelon form:

$[\begin{array}{rrrrr} 1 - 1 1 1 2 \\ 2 1 - 1 0 0 \end{array}] \sim [\begin{array}{rrrrr} 1 0 0 \frac{1}{3} \frac{2}{3} \\ 0 1 - 1 - \frac{2}{3} - \frac{4}{3} \end{array}] .$

Therefore,

$M_{C}^{B} = [\begin{array}{rrr} 0 \frac{1}{3} \frac{2}{3} \\ - 1 - \frac{2}{3} - \frac{4}{3} \end{array}] .$

As seen in the previous example, while a linear mapping T : V → W may be independent of the choice of bases for V and W, the explicit expression for its associated matrix will change under different choices of bases.

Our goal here is to start with the associated matrix $M_{C}^{B}$ relative to the old bases $B$ (for V) and $C$ (for W), and use it to produce the associated matrix relative to new bases $\hat{B}$ (for V) and Ĉ (for W). Let us start by recalling the fundamental property of the associated matrix relative to the old bases $B$ and $C$ :

$[T (v)]_{C} = M_{C}^{B} [v]_{B}, for all v \in V .$

Moreover, we can use matrices of change of bases to write the coordinates of the vectors v ∈ V and T (v) ∈ W relative to the new bases: bb

$[v]_{B} = P_{B}^{\hat{B}} [v]_{\hat{B}}, and [T (v)]_{C} = P_{C}^{\hat{C}} [T (v)]_{\hat{C}}$

If we insert these two identities in the previous equation, then we obtain

$P_{C}^{\hat{C}} [T (v)]_{\hat{C}} = M_{C}^{B} P_{B}^{\hat{B}} [v]_{\hat{B}} .$

Multiplying both sides of the equation on the left by ${(P_{C}^{\hat{C}})}^{- 1} = P_{\hat{C}}^{C}$ , we get

$[T (v)]_{\hat{C}} = [P_{\hat{C}}^{C} M_{C}^{B} P_{B}^{\hat{B}}] [v]_{\hat{B}} .$

Comparing this equation with the fundamental property satisfied by M^B_b :

$[T (v)]_{\hat{C}} = M_{\hat{C}}^{\hat{B}} [v]_{\hat{B}}, for all v \in V,$

we can finally conclude that

$M_{\hat{B}}^{\hat{B}} = P_{\hat{C}}^{C} M_{C}^{B} P_{B}^{\hat{B}} .$

Hence, we can obtain the associated matrix for T relative to the new bases $\hat{B}$ and Ĉ through a matrix product of three matrices. Note that, on one hand, $M_{C}^{B}$ and $M_{\hat{C}}^{\hat{B}}$ are both m × n matrices, where n = dim V and m = dim W. On the other hand, the matrix $P_{\hat{C}}^{C}$ is m × m and the matrix $P_{B}^{\hat{B}}$ is n × n.

We state the above important result as a theorem.

Theorem 3.8

Let T : V → W be a linear mapping and let $M_{C}^{B}$ be its associated matrix relative to the basis B for V and the basis $C$ for W. If $\hat{B}$ and Ĉ are new bases for V and W, respectively, then the associate matrix for T relative to the new bases is given by

$M_{\hat{B}}^{\hat{B}} = P_{\hat{C}}^{C} M_{C}^{B} P_{B}^{\hat{B}},$

where $P_{B}^{\hat{B}}$ and $P_{\hat{C}}^{C}$ are matrices of change of basis.

Example 3.14

We review the example from above. Let T : ℝ² → ℝ³ be a linear mapping such that

$T ([\begin{array}{l} 1 \\ 0 \end{array}]) = [\begin{array}{l} 1 \\ 2 \\ 3 \end{array}], T ([\begin{array}{l} 0 \\ 1 \end{array}]) = [\begin{array}{l} 2 \\ 1 \\ 0 \end{array}] .$

Its associated matrix relative to the canonical bases is

$M_{E_{3}}^{E_{2}} = [\begin{array}{ll} 1 2 \\ 2 1 \\ 3 0 \end{array}] .$

We wish to determine $M_{C}^{B}$ relative to the new bases given by

$B = \{[\begin{array}{l} 1 \\ 0 \end{array}], [\begin{array}{l} 1 \\ 1 \end{array}]\} and C = \{[\begin{array}{l} 1 \\ 0 \\ 0 \end{array}], [\begin{array}{l} 1 \\ 1 \\ 0 \end{array}], [\begin{array}{l} 0 \\ 1 \\ 1 \end{array}]\} .$

By Theorem 3.8, we can obtain $M_{C}^{B}$ as follows:

$M_{C}^{B} = P_{C}^{E_{3}} M_{E_{3}}^{E_{2}} P_{E_{2}}^{B} .$

Writing the matrix of change of basis from $B$ and $C$ to the corresponding canonical bases is straightforward since their columns consist of the basis vectors themselves:

$P_{E_{2}}^{B} = [\begin{array}{ll} 1 1 \\ 0 1 \end{array}], P_{E_{3}}^{C} = [\begin{array}{lll} 1 1 0 \\ 0 1 1 \\ 0 0 1 \end{array}] .$

However, we need to find $P_{C}^{E_{3}} = {(P_{E_{3}}^{C})}^{- 1}$ :

$[\begin{array}{rrrrrr} 1 1 0 1 0 0 \\ 0 1 1 0 1 0 \\ 0 0 1 0 0 1 \end{array}] \sim [\begin{array}{rrrrrr} 1 1 0 1 0 0 \\ 0 1 0 0 1 - 1 \\ 0 0 1 0 0 1 \end{array}] \sim [\begin{array}{rrrrrr} 1 0 0 1 - 1 1 \\ 0 1 0 0 1 - 1 \\ 0 0 1 0 0 1 \end{array}] .$

Therefore,

$P_{C}^{E_{3}} = [\begin{array}{rrr} 1 - 1 1 \\ 0 1 - 1 \\ 0 0 1 \end{array}] .$

We can finally compute $M_{C}^{B}$ :

$M_{C}^{B} = P_{C}^{E_{3}} M_{E_{3}}^{E_{2}} P_{E_{2}}^{B} = [\begin{array}{rrr} 1 - 1 1 \\ 0 1 - 1 \\ 0 0 1 \end{array}] [\begin{array}{ll} 1 2 \\ 2 1 \\ 3 0 \end{array}] [\begin{array}{ll} 1 1 \\ 0 1 \end{array}] = [\begin{array}{rr} 2 3 \\ - 1 0 \\ 3 3 \end{array}] .$

Example 3.15

Let T : ℝ³ → ℝ² be the linear mapping defined by

$T ([\begin{array}{l} x \\ y \\ z \end{array}]) = [\begin{array}{l} x + y \\ y - z \end{array}] .$

Its associated matrix relative to the canonical bases is

$M_{E_{2}}^{E_{3}} = [\begin{array}{ccc} 1 & 1 & 0 \\ 0 & 1 & - 1 \end{array}] .$

If we wish to determine its associated matrix relative to the bases $B$ and $C$ given by

$B = \{[\begin{array}{l} 1 \\ 0 \\ 1 \end{array}], [\begin{array}{l} 0 \\ 1 \\ 1 \end{array}], [\begin{array}{l} 1 \\ 1 \\ 1 \end{array}]\} and C = \{[\begin{array}{l} 1 \\ 2 \end{array}], [\begin{matrix} - 1 \\ 1 \end{matrix}]\},$

then we need to compute the matrices of change of bases $P_{E_{3}}^{B}$ and $P_{C}^{E_{2}}$ . We can write down directly the matrices

$P_{E_{2}}^{C} = [\begin{array}{rr} 1 - 1 \\ 2 1 \end{array}], P_{E_{3}}^{B} = [\begin{array}{lll} 1 0 1 \\ 0 1 1 \\ 1 1 1 \end{array}] .$

We only need to compute $P_{C}^{E_{2}} = {(P_{E_{2}}^{C})}^{- 1}$ :

$P_{C}^{E_{2}} = {(P_{E_{2}}^{C})}^{- 1} = [\begin{array}{rr} \frac{1}{3} \frac{1}{3} \\ - \frac{2}{3} \frac{1}{3} \end{array}] .$

By Theorem 3.8, we can obtain $M_{C}^{B}$ as follows:

$M_{C}^{B} = P_{C}^{E_{2}} M_{E_{2}}^{E_{3}} P_{E_{3}}^{B} = [\begin{array}{rr} \frac{1}{3} \frac{1}{3} \\ - \frac{2}{3} \frac{1}{3} \end{array}] [\begin{array}{rrr} 1 1 0 \\ 0 1 - 1 \end{array}] [\begin{array}{rrr} 1 0 1 \\ 0 1 1 \\ 1 1 1 \end{array}] = [\begin{array}{rrr} 0 \frac{1}{3} \frac{2}{3} \\ - 1 - \frac{2}{3} - \frac{4}{3} \end{array}]$

3.2 Eigenvalues, eigenvectors, and diagonalization

In this section, we consider a topic that has a wide range of applications in applied mathematics and by extension in the sciences and engineering: the eigenvectors and eigenvalues of a linear mapping. In the presence of some favourable conditions (namely, when the mapping is diagonalizable), they offer a way to understand the structure of a linear mapping that map a vector space to itself and, in turn, understand the structure of the involved vector space. Moreover, the study of diagonalizable linear mappings opens the avenue to more advanced methods (e.g., orthogonal or Fourier decomposition of vectors) that we discuss in the next chapter.

3.2.1 Introduction

For a finite-dimensional vector space V with dim V = n, let T : V → V be a linear mapping, and let $B$ = {b₁, b₂, …, b_n} be a basis for V. Since maps V into itself then, for each x in V, the vector T (x) can also be written in the same basis $B$ . The matrix associated with T relative to $B$ is given by

$M_{B}^{B} = [\begin{array}{llll} {[T (b_{1})]}_{B} [\begin{array}{ll} {T (b_{2})]}_{B} & \dots \end{array}] {[T (b_{n})]}_{B}] . \end{array}$

For the sake of simplicity, the associated matrix $M_{B}^{B}$ can simply be denoted as M^B. We must mention that any linear mapping that maps a vector space to itself is usually called a linear operator. So, from now on, M^B can be understood as the associated matrix of a linear operator relative to a basis $B$ .

In the previous section, we discussed the behavior of the associated matrices of a linear mapping T : V → W under a change of bases. In the particular case when T is a linear operator (that is, W = V), we can consider the change of basis from $B$ to some other basis $C$ . In this case, by Theorem 3.8, we have

$M^{C} = P_{C}^{B} M^{B} P_{B}^{C},$

where $P_{B}^{C}$ is the matrix of change of basis from $C$ to $B$ and $P_{C}^{B} = {(P_{B}^{C})}^{- 1}$ .

The fundamental goal is to find a basis $C$ (when possible) such that M^C is a diagonal matrix

$M^{C} = [\begin{array}{llll} λ_{1} \\ λ_{2} \\ ⋱ \\ λ_{n} \end{array}],$

where the empty spaces are understood to be filled with 0’s. If such a basis C = {c₁, c₂, …, c_n} exists, then the explicit expression

$M^{C} = [\begin{array}{llll} {T (c_{1})]}_{C} {[T (c_{2})]}_{C} \dots {[T (c_{n})]}_{C} \end{array}] = [\begin{array}{llll} λ_{1} \\ λ_{2} \\ ⋱ \\ λ_{n} \end{array}],$

shows that

$T (c_{i}) = λ_{i} c_{i}, for i = 1, 2, \dots, n .$

This means that if the linear mapping is applied to a vector of this particular basis, then we obtain a multiple of the same basis vector! This is quite extraordinary since, in general, the application of a linear mapping to an input vector does not produce an output vector that is proportional to the input vector. When a non zero vector is mapped by T to a scalar multiple of itself, such a vector is called an eigenvector, which comes from German “eigen,” meaning “self.” The scalar constant that multiplies and eigenvector is called an eigenvalue. Each eigenvector is associated with an eigenvalue. We elaborate on this in the next section.

3.2.2 Eigenvalues and eigenvectors

Definition 3.5

Let T : V → V be a linear mapping. Any non zero vector x ∈ V such that

T (x) = x,

for some scalar λ is called an eigenvector of T. The scalar λ is called the eigenvalue associated with x. Eigenvalues are allowed to be zero.

Observation 3.1

The zero vector is excluded as eigenvector, because it would satisfy the equation for any value of λ. From the above introductory comments, we would expect that there are exactly n = dim V eigenvectors and eigenvalues. While sometimes this can be indeed the case, however, this topic will be treated with care and detail below.

Example 3.16

Let T : ℂ² → ℂ² be the linear mapping defined by

$T ([\begin{array}{l} x \\ y \end{array}]) = [\begin{array}{r} - y \\ x \end{array}] .$

Recall that ℂ² = x y^⊤ : x, y ∈ ℂ. Verify that λ₁ = i and λ₂ = -i are eigenvalues of T. If possible, find a basis for ℂ² such that its elements are eigenvectors of T and write its associated matrix relative to such basis.

We consider first λ₁ = i. If λ_i is an eigenvalue of T, then there is some vector [x y]^⊤ ∈ ℂ² such that

$T ([\begin{array}{l} x \\ y \end{array}]) = i [\begin{array}{l} x \\ y \end{array}] .$

However, by the definition of T, the same vector must satisfy

$T ([\begin{array}{l} x \\ y \end{array}]) = [\begin{array}{r} - y \\ x \end{array}] .$

This implies that

ix = -y,
iy = x.

Solving this system of equations, we obtain

$[\begin{array}{l} x \\ y \end{array}] = α [\begin{array}{l} i \\ 1 \end{array}], α \in R .$

This means that all non zero scalar multiples of $[\begin{array}{l} i \\ 1 \end{array}]$ are eigenvectors of T associated with λ₁.

Now, we consider λ₂ = -i. Then, the following two equations must hold for some [x y]^⊤ ∈ ℂ²:

$T ([\begin{array}{l} x \\ y \end{array}]) = - i [\begin{array}{l} x \\ y \end{array}], and T ([\begin{array}{l} x \\ y \end{array}]) = [\begin{matrix} - y \\ x \end{matrix}] .$

This implies that

-ix = -y,
-iy = x.

The solution of this system of equation is given by

$[\begin{array}{l} x \\ y \end{array}] = β [\begin{array}{r} - i \\ 1 \end{array}], β \in R$

This means that all non zero scalar multiples of $[\begin{matrix} - i \\ 1 \end{matrix}]$ are eigenvectors of T associated with λ₂. We observe that the set

$C = \{[\begin{array}{l} i \\ 1 \end{array}], [\begin{array}{r} - i \\ 1 \end{array}]\},$

formed by one eigenvector associated with λ₁ and one eigenvector associated with λ₂, is a linearly independent set. Moreover, since dim ℂ² = 2, we conclude that ℂ is a basis for ℂ2.

It is not hard to verify that the associated matrix of T relative to $C$ is

$M^{C} = [\begin{array}{rr} i 0 \\ 0 - i \end{array}] .$

Note that the order that you choose for the elements of the basis is important. In fact, if we change the order of the elements of C, then we obtain a new basis

$\hat{C} = \{[\begin{array}{r} - i \\ 1 \end{array}], [\begin{array}{l} i \\ 1 \end{array}]\},$

and the associate matrix of T relative to Ĉ is

$M^{\hat{C}} = [\begin{array}{rr} - i 0 \\ 0 i \end{array}] \neq M^{C} .$

Observation 3.2

The attentive reader would have noticed that in the previous example, the eigenvectors associated with each eigenvalues are not unique: any non zero scalar multiple of the set $C$ is also an eigevector of T. This means that it is possible to normalize eigenvectors depending on the context, e.g., to obtain an eigenvector of unit length. It also implies that it is allowed to multiply an eigenvector by -1, i.e., changing the sign of all coordinate values at once, if convenient. In fact, we have the following general result.

Theorem 3.9

Let T : V → V be a linear operator. If v₁, v₂, …, v_r ∈ V are eigenvectors associated with an eigenvalue λ, then any non zero vector v ∈ Span{v₁, v₂, …, v_r} is also an eigenvector associated with λ.

Proof. Choose any non zero vector v ∈ Span{v₁, v₂, …, v_r}. Then, v can be written as a linear combination of the eigenvectors v₁, v₂, …, v_r. So, for some scalars a₁, a₂, …, a_r, we can write

v = a₁ v₁ + a₂v₂ + … + a_rv_r.

By the linearity of T and the fact that v₁, v₂, …, v_r, are eigenvectors associated with the egienvalue λ, we have that

$\begin{aligned} T (v) = T (a_{1} v_{1} + a_{2} v_{2} + \dots + a_{r} v_{r}) \\ = a_{1} T (v_{1}) + a_{2} T (v_{2}) + \dots + a_{r} T (v_{r}) \\ = a_{1} λ v_{1} + a_{2} λ v_{2} + \dots + a_{r} λ v_{r} \\ = λ (a_{1} v_{1} + a_{2} v_{2} + \dots + a_{r} v_{r}) \\ = λ v \end{aligned}$

Hence, T (v) = λ v, which proves that v is an eigenvector of T associated with λ.

The following important theorem will be needed later.

Theorem 3.10

Let T : V → V be a linear operator and let x₁, x₂, …, x_m be eigenvectors associated with distinct eigenvalues λ₁, λ₂, …, λ_m, i.e., λ_i ≠ λ_j for i ≠ j. Then, {x₁, x₂, …, x_m} is a linearly independent set.

Proof. We prove this theorem by contradiction. Suppose that {x₁, x₂, …, x_m} is a linearly dependent set. Therefore, since x₁≠ 0, we know that one of the vectors in the set is a linear combination of the preceeding vectors (see Theorem 2.6). Let p be the least index such that x_p+1 is a linear combination of the preceeding (linearly independent) vectors. Then there exist scalars c₁, …, c_p such that

c₁ x₁ + … + c_p v₀= v_p+1.

On one hand, applying T to both sides of this equation and using the fact that T (x_k) = λ_k x_k for each k, we obtain

λ₁ c₁ x₁ + … + λ_pc_px_p = λ_p+1 v_p+1.

On the other hand, if we multiply both sides by λ_p+1, we get

λ_p+1 c₁ x₁ + … + λ_p+1 c_p x_p = λ_p+1 v_p+1.

Subtracting the previous two equations, we obtain

(λ₁ − λ_p+1) c₁ x₁ + … + (λ_p − λ_p+1) c_p x_p = 0.

Since {x₁, x₂, …, x_p} is linearly independent, the weights in the equation above are all zero. But none of the factors λ_i - λ_p+1 are zero, because the eigenvalues are distinct. Hence c_i = 0 for i = 1, …, p, which means that v_p+1 = 0, which is impossible. Hence, {x₁, x₂, …, x_m} cannot be linearly dependent and therefore it must be linearly independent.

We note that the reciprocal of the previous theorem is not true in general. That is, it is quite possible for a linear operator to have several linearly independent eigenvectors associated with one eigenvalue. What the previous theorem is saying is that an eigenvector of a linear operator can not be associated with two distinct eigenvalues.

The following definition is motivated by Theorem 3.9.

Definition 3.6

Let T : V → V be a linear operator. For each eigenvalue λ of T, we define the set

E_λ = Span{x : T (x) = λx}.

It can easily be deduced from Theorem 3.9 that the set E_λ for each eigenvalue λ of T is a vector subspace of V. So we will call E_λ the eigenspace associated with λ. Observe that E_λ is composed of all the (non zero) linear combinations of the eigenvectors associated with λ, together with 0 ∈ V (otherwise, E_λ is not a vector subspace), which is not an eigenvector. By Theorem 3.10 we know that any eigenvector of T cannot belong to two different eigenspaces. Therefore we have that E_λi ∩ E_λj = {0} whenever λ_i ≠ λ_j.

We are now paving the way for a central theorem for eigenvectors and eigenvalues. Let T : V → V be a linear operator. Our goal will be to characterize the eigenspaces of T. In order to do this, we search for solutions of the equation

T (x) = λx,

where λ is a scalar and x is a non zero vector in V. This equation can also be written as

(T − λId)(x) = 0,

where Id denotes the identity operator on V defined by Id(v) = v for all v ∈ V. This means that λ is an eigenvalue of T with associated eigenvector x if and only if the above equation has a non trivial solution. Consequently, we have the following characterization of the eigenspaces of T.

Theorem 3.11

Let T : V → V be a linear operator. Then, for each eigenvalue λ of T

E_λ = Ker(T − λId).

Now, let us fix a basis $B$ of V and let M^$B$ be the associated matrix of T relative to $B$ . Then the associated matrix of the operator T - λ Id relative to $B$ is given by M^{$M_{C}^{B}$} - λ I_n, where I_n is the identity matrix and n = dim V. In this way, we can translante the above discussion about eigenspaces to associated matrices and coordinate vectors: the vector x ∈ V satisfies (T - λI d)(x) = 0 if and only if

$(M^{B} - λ I_{n}) [x]_{B} = 0 \in R^{n}$

We immediately have the following result.

Theorem 3.12

Let T : V → V be a linear operator. Let $B$ be a basis for V and let M^{$M_{C}^{B}$} be the associated matrix of T relative to $B$ . Then, for each eigenvalue λ of T :

a. x ∈ E_λ if and only if [x]_$B$ ∈ Nul(M^$B$ - λI_n), with n = dim V.

b. dim E_λ = dim Nul(M^$B$ - λI_n).

So far, we have described the eigenspaces of a linear operator assuming that we already know the eigenvalues. However, we still need to describe how to find the eigenvalues, so we make the following observation. The homogeneous equation M^$B$ − λI_n [x]_$B$ = 0 has a non trivial solution if and only if the value of λ is chosen so that the matrix M^$B$ − λI_n is not invertible. Equivalently, the values of λ must be choses so that

det(M^$B$ − λI_n) = 0.

It turns out that det(M^$B$ - λI_n) is a polynomial of degree at most n and, therefore, if λ is an eigenvalue of T then it is a root of this polynomial. This means that a linear operator has at most n distinct eigenvalues.

This polynomial is important enough to deserve a special name.

Definition 3.7

Let T : V → V be a linear operator and let M^$B$ be its associated matrix relative to a basis $B$ .

(a) The polynomial p(λ) = det(M^$B$ - λI_n) of degree n = dim V is called the characteristic polynomial of T.

(b) p(λ) = det(M^$B$ - λI_n) = 0 is called the characteristic equation of T.

We must remark that important fact that the characteristic polynomial of a linear operator is independent of the basis that we choose to compute the associated matrix. To show this, suppose that P is a matrix of change of basis from $B$ to some new basis, and let M = P^-1M^$B$P is the associated matrix relative to the new basis. Then,

$\det (M - λ I_{n}) = \det [P^{- 1} M^{B} P - λ P^{- 1} I_{n} P] = \det [P^{- 1} (M^{B} - λ I_{n}) P] .$

By the multiplicative property of determinants (det AB = (det A)(det B)), we have

$\det (M - λ I_{n}) = (\det P^{- 1}) [\det (M^{B} - λ I_{n})] (\det P)) = \det (M^{B} - λ I_{n}),$

where we have used the fact that (det P^-1)(det P) = 1. This means that the characteristic polynomial (and its roots) does not change regardless of its matrix representation.

Theorem 3.13

Let T : V → V be a linear operator and n = dim V. Then T has at most n linear independent eigenvectors.

Proof. Since the characteristic polynomial has degree n, then T has at most n distinct eigenvalues. It follows from Theorem 3.10 that T has at most n linearly independent eigenvectors.

Example 3.17

Find the eigenvalues and eigenvectors of T : ℝ² → ℝ² defined by

$T (x) = A x with A = [\begin{array}{rr} 2 3 \\ 3 - 6 \end{array}] .$

(Although it is not explicitly mentioned, the expressions for x ε ℝ² and T are given in terms of the canonical basis.)

We may proceed as follows.

(1) First, we find the eigenvalues. Recall that the eigenvalues of T are the roots of its characteristic polynomial given by

$p (λ) = \det (A - λ I_{2}) = \det [\begin{array}{rr} 2 - λ 3 \\ 3 - 6 - λ \end{array}] = λ^{2} + 4 λ - 21 = (λ + 7) (λ - 3) .$

Therefore, the eigenvalues of T are

λ₁ = -7, λ₂ = 3.

(2) Now, we obtain the eigenvectors associated with each eigenvalue.

(a) Let us start with λ₁ = -7. The eigenvectors associated with this eigenvalue are vectors in the eigenspace E_λ₁ = Ker(T + 7 Id). Since we have (implicitly) fixed the canonical basis, we have that x = [x]ε₂ and M^ε2 = A. Moreover, by Theorem 3.12 we have that x ∈ E_λ₁ if and oly if x ∈ Nul(A + 7 I₂); that is,

(A+7I₂)x= 0.

Noticing that

$A + 7 I_{2} = [\begin{array}{ll} 9 3 \\ 3 1 \end{array}] \sim [\begin{array}{ll} 1 \frac{1}{3} \\ 0 0 \end{array}],$

we deduce that

$Nul (A + 7 I_{2}) = Span \{[\begin{array}{r} - \frac{1}{3} \\ 1 \end{array}]\} .$

Therefore, dim E_λ₁ = 1 and every eigenvector associated with λ₁ is of the form

$x = μ [\begin{array}{r} - \frac{1}{3} \\ 1 \end{array}] μ \neq 0$

(eigenvectors must be non zero vectors).

(b) Similarly, for λ₁ = 3, we have

$Nul (A - 3 I_{2}) = Span \{[\begin{array}{l} 3 \\ 1 \end{array}]\} .$

Therefore, dim E_λ₂ = 1 and every eigenvector associated with λ₂ is of the form

$x = η [\begin{array}{l} 3 \\ 1 \end{array}] η \neq 0$

We would like to remark that the two eigenvalues are distinct (λ₁ ≠ λ₂). Then, by Theorem 3.10, the set $B$ = {v₁, v₂} with

$v_{1} = [\begin{array}{r} - \frac{1}{3} \\ 1 \end{array}], v_{2} = [\begin{array}{l} 3 \\ 1 \end{array}],$

is linearly independent since v₁ and v₂ are eigenvectors associated with λ₁ and λ₂, respectively. Furthermore, dim ℝ² = 2, so these two linearly independent eigenvectors form a basis of ℝ² ! It turns out that if we write the matrix representation of T relative to B, we obtain

$M^{B} = [\begin{array}{cc} - 7 & 0 \\ 0 & 3 \end{array}],$

which, conveniently, is a diagonal matrix whose non zero element are precisely the eigenvalues of T.

Example 3.18

Here, we revisit Example 3.16 with the emphasis on showing the systematic procedure of calculating eigenvalues and eigenvectors starting from a linear operator.

Let T : ℂ² → ℂ² be a linear operator defined by

$T (x) = A x, with A = [\begin{array}{cc} 0 & 1 \\ - 1 & 0 \end{array}] .$

We have implicitly fixed the canonical basis for ℂ² to represent the operator as a matrix. Find the eigenvalues and eigenvectors of T.

(1) First, let us find the eigenvalues of T. The characteristic polynomial is given by

$p (λ) = \det (A - λ I_{2}) = \det [\begin{array}{cc} - λ & 1 \\ - 1 & - λ \end{array}] = λ^{2} + 1 .$

The eigenvalues of T are the roots of the characteristic polynomial. In this case, we have two distinct eigenvalues given by λ₁ = i and λ₂ = −i. Notice that these eigenvalues are complex numbers.

(2) Next, we describe the eigenvectors associated with each of the eigenvalues.

(a) For λ₁ = i, E_λ₁ = Ker(T − i Id). As in Example 3.17, we have fixed the canonical basis and, therefore, E_λ₁ coincides with Nul(A − i I2). Hence, we compute

$A - i I_{2} = [\begin{array}{cc} - i & 1 \\ - 1 & - i \end{array}] \sim [\begin{array}{ll} 1 i \\ 0 0 \end{array}],$

and therefore $E_{λ_{1}} = Span \{[\begin{array}{r} - i \\ 1 \end{array}]\}$ .

(b) Similarly, for λ₂ = −i, we have E_λ₂ = Nul(A + i I₂). Since,

$A + i I_{2} = [\begin{array}{cc} i & 1 \\ - 1 & i \end{array}] \sim [\begin{array}{cc} 1 & - i \\ 0 & 0 \end{array}],$

we obtain $E_{λ_{2}} = Span \{[\begin{array}{l} i \\ 1 \end{array}]\}$ .

Now, consider the set

$B = \{[\begin{array}{r} - i \\ 1 \end{array}], [\begin{array}{l} i \\ 1 \end{array}]\},$

composed of one eigenvector associated with each eigenvalue. Since the eigenvalues are distinct, $B$ is a linearly independent set in ℂ² with dim ℂ² = 2. Therefore, $B$ is a basis for ℂ². It is straightforward to verify that the matrix representation of T in this new basis is

$M^{B} = [\begin{array}{rr} i 0 \\ 0 - i \end{array}] .$

3.2.3 Multiplicity of eigenvalues

So far, we have encountered examples of linear operators whose eigenvalues are all distinct and, consequently, it was possible to construct a basis for the involved vector space consisting entirely of eigenvectors of the linear operators. However, if we take into account the multiplicity of the linear factors of the characteristic polynomial, then it is not the case that the eigenvalues of every linear operator are all distinct as we show in the following example.

Example 3.19

Let T : ℝ³ → ℝ³ be a linear operator defined by

$T (x) = A x with A = [\begin{array}{ccc} 4 & - 1 & 6 \\ 2 & 1 & 6 \\ 2 & - 1 & 8 \end{array}] .$

As usual, we have fixed the canonical basis of ℝ³ and, thus, A is the matrix associated with T relative to the canonical basis.

Let us compute the characteristic polynomial of T :

$p (λ) = \det (A - λ I_{3}) = \det [\begin{array}{rrr} 4 - λ - 1 6 \\ 2 1 - λ 6 \\ 2 - 1 8 - λ \end{array}] = - (λ - 2)^{2} (λ - 9) .$

Notice that p(λ) has two distinct real roots λ₁ = 2 and λ₂ = 9. However, if we take into account the multiplicity of each linear factor, then p(λ) has three roots: one double root λ₁ = 2 and a single root λ₂ = 9. Hence, T does not have three distinct eigenvalues and, unfortunately, this means that the existence of a basis for ℝ³ consisting entirely of eigenvectors of T is not guaranteed. At this point, we know that we can choose two linearly independent eigenvectors: one from E_λ₁ and another one from E_λ₂ because λ₁ ≠ λ₂. But it is not clear if a third appropriate eigenvector exists or, even worse, where to find it.

However, if we are lucky enough, one of the eigenspaces could have dimension 2 and, therefore, we could obtain the missing third eigenvector from such eigenspace. In fact, this is precisely what happens in this example. You can verify that

$E_{λ_{1}} = Span \{[\begin{matrix} 1 / 2 \\ 1 \\ 0 \end{matrix}], [\begin{array}{r} - 3 \\ 0 \\ 1 \end{array}]\} and E_{λ_{2}} = Span \{[\begin{array}{l} 1 \\ 1 \\ 1 \end{array}]\} .$

Fortunately, dim E_λ₁ = 2 and, therefore, we can choose two linearly independent eigenvectors associated with λ₁; for instance

$B = \{[\begin{matrix} 1 / 2 \\ 1 \\ 0 \end{matrix}], [\begin{array}{r} - 3 \\ 0 \\ 1 \end{array}]\} .$

Moreover, the set obtained from B by adding any eigenvector associated with λ₂ will be linearly independent (because λ₁ ≠ λ₂). In total, we have three linearly independent eigenvectors that can be used to construct a basis for ℝ³. For example, This means that the set

$B = \{[\begin{matrix} 1 / 2 \\ 1 \\ 0 \end{matrix}], [\begin{array}{r} - 3 \\ 0 \\ 1 \end{array}], [\begin{array}{l} 1 \\ 1 \\ 1 \end{array}]\},$

is a basis for ℝ³ consisting entirely of eigenvectors of T.

The following example shows that not every linear operator T : V → V admits enough linearly independent eigenvectors to constitute a basis for V.

Example 3.20

Fix the canonical basis of ℝ³, and let T : ℝ³ → ℝ³ be a linear operator defined by

$T (x) = A x with A = [\begin{array}{lll} 2 2 3 \\ 0 2 2 \\ 0 0 2 \end{array}] .$

Let us examine the eigenvalues and eigenvectors of T.

(1). First, we find the eigenvalues. The characteristic polynomial of T is

$p (λ) = \det (A - λ I_{3}) = \det [\begin{array}{ccc} 2 - λ & 2 & 3 \\ 0 & 2 - λ & 2 \\ 0 & 0 & 2 - λ \end{array}] = (2 - λ)^{3}$

Therefore, λ = 2 is a triple root of p(λ) which, in turn, is the only eigenvalue of T.

(2) The corresponding eigenspace for λ = 2 is E₂ = Nul(A - 2 I₃). We have

$E_{2} = Nul (A - 2 I_{3}) = Nul [\begin{array}{lll} 0 2 3 \\ 0 0 2 \\ 0 0 0 \end{array}] = Nul [\begin{array}{lll} 0 1 0 \\ 0 0 1 \\ 0 0 0 \end{array}] = Span \{[\begin{array}{l} 1 \\ 0 \\ 0 \end{array}]\} .$

In this case, T only admits one linearly independent eigenvector since dim E₂ = 1. Therefore, it is not possible to produce a basis for ℝ³ consisting entirely of eigevenctors of T.

There are two quantities that play an important role in the general analysis of the eigenspaces of a linear operator: the multiplicity of each eigenvalue as a root of the characteristic polynomial and the dimension of the corresponding eigenspaces. These quantities deserve their own terminology.

Definition 3.8

Let T : V → V be a linear operator and let p(x) be its characteristic polynomial. For each distinct eigenvalue λ of T, we have

p(x) = (x - λ)^m^λ q(λ),

where q(x) is a non zero polynomial such that q(λ) ≠ 0, and m_λ is a positive integer.

(a) We say that m_λ is the algebraic multiplicity of λ.

(b) We say that dim E_λ is the geometric multiplicity of λ.

The following theorem states the relationship between the algebraic and geometric multiplicities of the eigenvalues of a linear operator.

Theorem 3.14

Let T : V → V be a linear operator. Then, for each distinct eigenvalue λ,

1 ≤ dim E_λ ≤ mλ.

3.2.4 Diagonalization of linear operators

Now, we return to the original motivation of this chapter: given a linear operator T : V → V, find a basis of V such that the matrix associated with T is a diagonal matrix. Let us introduce the following definition.

Definition 3.9

A linear operator T : V → V is said to be diagonalizable if there is a basis $C$ = {c₁, c₂, …, c_n} of V made up entirely of eigenvectors of T.

If a linear operator is diagonalizable, then its associated matrix relative to the basis of eigenvector $C$ is of the form

$M^{C} = [\begin{array}{llll} λ_{1} \\ λ_{2} \\ ⋱ \\ λ_{n} \end{array}]$

where T(ci) = λ_i ci, for i = 1, 2, …, n (see the introductory comments of this section). As we saw before, not every linear operator is diagonalizable.

An immediate consequence of the above definition is the following theorem.

Theorem 3.15

Let T : V → V be a linear operator and n = dim V. Then T is diagonalizable if and only if it admits n linear independent eigenvectors.

The previous theorem, together with Theorem 3.10, yields the following result.

Corollary 3.1

Let T : V → V be a linear operator and n = dim V. If T has n distinct eigenvalues, then T is diagonalizable.

Observe that from Definition 3.8, we have that T has n distinct eigenvalues if and only if m_λ = 1; that is, each eigenvalue of T has algebraic multiplicity equal to 1. Furthermore, by Theorem 3.14, this means that dim E_λ = 1 for each eigenvalue λ.

In general, however, a linear operator may admit eigenvalues with algebraic multiplicity greater than 1. In this general setting, a linear operator is diagonalizable if and only if, for each eigenvalue λ with algebraic multiplicity m_λ, there are m_λ linear independent eigenvectors associated with λ. We formalize this fact in the following theorem.

Theorem 3.16

For a linear operator T : V → V, let λ₁, λ₂, …, λ_k, be all the distinct eigenvalues of T. Then T is diagonalizable if and only if

dim E_λ₁ + dim E_λ₂ + … + dim E_λ_k = dim V.

Equivalently, T is diagonalizable if and only if

dim E_λ_j = m_λ_j, for j = 1,2, …, k.

Now, we turn our attention the the associated matrix of a diagonalizable linear operator.

Theorem 3.17

Let T : V → V be a diagonalizable linear operator, n = dim V. Then there is a basis for V, $C$ = {c₁, …, c_n}, formed by eigenvectors of T; that is, T (c_i) = λ_i c_i for i = 1, 2, …, n. Moreover, the matrix associated with T relative to C is a diagonal matrix given by

$M^{C} = [\begin{array}{llll} λ_{1} \\ λ_{2} \\ ⋱ \\ λ_{n} \end{array}]$

Some remarks are in order. In Theorem 3.17, the associated matrix M^C is a diagonal matrix with the eigenvalues of T in its main diagonal. If $B$ is a different basis for V, then

$M^{B} = P_{B}^{C} M^{C} {(P_{B}^{C})}^{- 1}$

where the change of basis matrix is given by

$P_{B}^{C} = [\begin{array}{lll} {[c_{1}]}_{B} \dots {[c_{n}]}_{B} \end{array}] .$

Notice that the columns of this change of basis matrix are the coordinates of the eigenvectors of T relative to $B$ . Moreover, the eigenvalues of T appear in the main diagonal of M^$C$ in the same order as the corresponding eigenvectors in the basis $C$ .

Most textbooks present the topic of diagonalization of linear operators as a method for factorizing matrices (referring to the associated matrix M^{$M_{C}^{B}$}) into a product P D P^-1, where P is an invertible matrix and D is a diagonal matrix. However, these notes are intended to approach diagonalization from an operator point of view.

Finally, observe that we have total freedom in choosing the order of the eigenvectors in the basis $C$ . This means that the diagonal associated matrix for T is only uniquely determined up to the order in which the eigenvectors of T appear in $C$ .

Example 3.21

Fix the standard basis of ℝ³. Let T : ℝ³ → ℝ³ be a linear operator defined by T (x) = Ax, with

$A = [\begin{array}{rrr} 1 3 3 \\ - 3 - 5 - 3 \\ 3 3 1 \end{array}]$

If possible, diagonalize the linear operator T.

(1) First, we find the eigenvalues. The characteristic equation is

$p (λ) = \det (A - λ I_{3}) = \det [\begin{array}{ccc} 1 - λ & 3 & 3 \\ - 3 & - 5 - λ & - 3 \\ 3 & 3 & 1 - λ \end{array}] = - (λ - 1) (λ + 2)^{2} .$

Therefore, the eigenvalues of T are λ₁ = 1 and λ₂ = −2 with algebraic multiplicities m_λ₁ = 1 and m_λ₂ = 2, respectively.

(2) Now we find the eigenvectors.

(a) For λ₁ = 1, we have m_λ₁ = 1, so we expect know that dim E_λ₁ = 1. Indeed,

$E_{λ_{1}} = Nul (A - I_{3}) = Nul [\begin{array}{rrr} 0 3 3 \\ - 3 - 6 - 3 \\ 3 3 0 \end{array}] = Span \{[\begin{array}{r} 1 \\ - 1 \\ 1 \end{array}]\} .$

(b) For λ₂ = -2 with m_λ₂ = 2, we have 1 ≤ dim E_λ₂ ≤ 2. Let us compute

$E_{λ_{2}} = Nul (A + 2 I_{3}) = Nul [\begin{array}{rrr} 3 3 3 \\ - 3 - 3 - 3 \\ 3 3 3 \end{array}] = Span \{[\begin{array}{r} - 1 \\ 1 \\ 0 \end{array}], [\begin{array}{r} - 1 \\ 0 \\ 1 \end{array}]\} .$

Therefore, dim E_λ₂ = 2.

Since dim E_λ₁ + dim E_λ₂ = dimℝ³, by Theorem 3.16, T is diagonalizable.

(3) Let us compute a diagonal associated matrix for T. For this, we must construct a basis for ℝ³ consisting entirely of eigenvectors. We are free to choose the order of the eigenvectors, but we need to be consistent. Let

$C = \{[\begin{array}{r} 1 \\ - 1 \\ 1 \end{array}], [\begin{array}{r} - 1 \\ 1 \\ 0 \end{array}], [\begin{array}{r} - 1 \\ 0 \\ 1 \end{array}]\}$

be the basis of eigenvectors. Then, A = P D P^-1 with

$D = [\begin{array}{rrr} 1 0 0 \\ 0 - 2 0 \\ 0 0 - 2 \end{array}], P = [\begin{array}{rrr} 1 - 1 - 1 \\ - 1 1 0 \\ 1 0 1 \end{array}] .$

Here, D ≡ M^C and $P \equiv P_{E_{3}}^{C}$ (ε₃ denotes the canonical basis).

Example 3.22

Fix the canonical basis of ℝ³. Let T : ℝ³ → ℝ³ defined by T (x) = Ax, with

$A = [\begin{array}{rrr} 2 4 3 \\ - 4 - 6 - 3 \\ 3 3 1 \end{array}] .$

If possible, diagonalize the linear operator T.

(1) First, we find the eigenvalues. The characteristic equation is

$p (λ) = \det (A - λ I_{3}) = \det [\begin{array}{ccc} 2 - λ & 4 & 3 \\ - 4 & - 6 - λ & - 3 \\ 3 & 3 & 1 - λ \end{array}] = - (λ - 1) (λ + 2)^{2} .$

Therefore, the eigenvalues of T are λ₁ = 1 and λ₂ = -2 with algebraic multiplicities m_λ₁ = 1 and m_λ₂ = 2, respectively. Note that the characteristic polynomial is identical to the one of the previous example, and therefore the eigenvalues are the same as well.

(2) Now we find the eigenvectors. Let us start examining the eigenvectors associated with λ₂ = −2 with m_λ₂ = 2, since there resides the only possiblility for the matrix not being diagonalizable if dim E_λ₂ ≠ m_λ₂. We compute

$E_{λ_{2}} = Nul (A + 2 I_{3}) = Nul [\begin{array}{rrr} 4 4 3 \\ - 4 - 4 - 3 \\ 3 3 3 \end{array}] = Nul [\begin{array}{lll} 1 1 0 \\ 0 0 1 \\ 0 0 0 \end{array}] = Span \{[\begin{array}{r} - 1 \\ 1 \\ 0 \end{array}]\} .$

Since dim E_λ₂ ≠ m_λ₂, it is not possible to find a basis for ℝ³ consisting entirely of eigenvectors of T and, therefore, T is not diagonalizable.