2. VECTOR SPACES

This chapter introduces the underlying algebraic structure of linear algebra – that of a finite dimensional vector space. The definition of a vector space involves an arbitrary field whose elements are called scalars. The following notation will be used (unless otherwise stated or implied):

𝕂	the field of scalars
a, b, c, …	the elements of 𝕂
V	the given vector space
u, v, w	the elements of V

Nothing essential is lost if the reader assumes that 𝕂 is the real field ℝ or the complex field ℂ.

2.1 Vector spaces and subspaces

The following defines the notion of vector space or linear space.

Definition 2.1

Let 𝕂 be a given field (ℝ or ℂ) and let V be a non empty set with rules of addition, and scalar multiplication which assigns to any u, v ∈ V a sum u + v ∈ V and to any u ∈ V and scalar k ∈ 𝕂 a product ku ∈ V. Then V is called a vector space over 𝕂 (and the elements of V are called vectors) if the following axioms hold:

(A1) For any vectors u, v, w ∈ V, (u + v) + w = u + (v + w).

(A2) There is a vector in V, denoted by 0 and called the zero vector, for which u+0 = u for any vector u ∈ V.

(A3) For each vector u ∈ V there is a vector in V, denoted by -u, for which u+(-u) = 0.

(A4) For any vectors u, v ∈ V, u + v = v + u.

(M1) For any scalar k ∈ 𝕂 and any vectors u, v ∈ V, k (u + v) = k u + k v.

(M2) For any scalars a, b ∈ 𝕂 and any vector u ∈ V, (a + b) u = a u + b u.

(M3) For any scalars a, b ∈ 𝕂 and any vector u ∈ V, (ab) u = a (b u).

(M4) For the unit scalar 1 ∈ 𝕂, 1 u = u for any vector u ∈ V.

The above axioms naturally split into two sets. The first four are only concerned with the additive structure of V. From these four axioms it follows that any sum of vectors of the form

v₁ + v₂ + … + v_m

requires no parentheses and does not depend upon the order of the summands.

Theorem 2.1

For any vector space V,

a. The zero vector 0 ∈ V is unique.

b. For any vector u ∈ V, the additive inverse -u ∈ V is unique.

c. For any vectors u, v, w ∈ V, if u + w = v + w then u = v.

Proof. a. Suppose that $\hat{0} \in V$ is a vector for which $u + \hat{0} = u$ for any vector u ∈ V. Then,

$\begin{aligned} \hat{0} & = \hat{0} + 0 \\ = 0 \end{aligned}$

(by A2)

= 0 (by the definition of $\hat{0}$ ).

Since $\hat{0} = 0$ , the zero vector is unique.

b. Given a vector u ∈ V, let w ∈ V be a vector for which u + w = 0. Then,

$\begin{aligned} w & = w + 0 \\ = w + [u + (- u)] \\ = [w + u] + (- u) \\ = 0 + (- u) \\ = - u \end{aligned}$

(by A2)

(by A3)

(by A1)

(by the definition of w)

(by A2).

Since w = -u, the negative of u is unique.

c. For any vectors u, v, w ∈ V, such that u + w = v + w then

$\begin{aligned} u & = u + 0 \\ = u + [w + (- w)] \\ = [u + w] + (- w) \\ = [v + w] + (- w) \\ = v + [w + (- w)] \\ = v + 0 \\ = v \end{aligned}$

(by A2)

(by A3)

(by A1)

(by u + w = v + w)

(by A1)

(by A3)

(by A2).

Therefore, u + w = v + w implies u = v.

Also, subtraction is defined by

u − v = u + (−v).

On the other hand, the remaining four axioms are concerned with the “action” of the field 𝕂 on V. Observe that the labelling of the axioms reflects this splitting. Using these additional axioms we prove the following simple properties of a vector space.

Theorem 2.2

Let V be a vector space over a field K.

a. For any scalar k ∈ K and 0 ∈ V, k 0 = 0.

b. For 0 ∈ K and any vector u ∈ V, then 0 u = 0.

c. For any k ∈ K and any u ∈ V, (-k) u = k (-u) = -(k u).

Proof. a. We compute:

k0 + 0 = k0 = k (0 + 0) = k0 + k0.

But k 0 + 0 = k 0 + k0 implies k 0 = 0 (Theorem 2.1 part c).

b. Similarly, for any vector u ∈ V,

0u + 0 = 0u = (0 + 0) u = 0u + 0u,

which implies 0 u = 0.

c. On one hand, for any k ∈ K and any u ∈ V,

(-k) u = (-k) u + ku - (ku) = (-k + k) u - (ku) = 0u - (ku) = -(k u).

Therefore, (-k) u = -(k u). On the other hand,

k (-u) = k (-u) + ku - (ku) = k (u - u) - (ku) = k0 - (ku) = -(k u).

Therefore, k (-u) = -(k u).

Observe that when k = 1 in part c of the previous theorem, we have -u = (-1) u for each u ∈ V. That is, multiplying u by -1 produces its additive inverse.

Now we list a number of important examples of vector spaces which will be used throughout this course.

Example 2.1

Let 𝕂 be an arbitrary field. The notation 𝕂ⁿ is frequently used to denote the set of all column matrices with n rows and entries which are elements in 𝕂. Here 𝕂ⁿ is viewed as a vector space over 𝕂 where vector addition and scalar multiplication defined by

$[\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n} \end{matrix}] + [\begin{matrix} b_{1} \\ b_{2} \\ ⋮ \\ b_{n} \end{matrix}] = [\begin{matrix} a_{1} + b_{1} \\ a_{2} + b_{2} \\ ⋮ \\ a_{n} + b_{n} \end{matrix}]$

and

$k [\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n} \end{matrix}] = [\begin{matrix} k \\ k \end{matrix} a_{1}] [\begin{matrix} a_{2} \\ ⋮ \\ k \end{matrix} a_{n}] .$

The zero vector in 𝕂ⁿ is

$0 = [\begin{matrix} 0 \\ 0 \\ ⋮ \\ 0 \end{matrix}]$

and the inverse of a vector is defined by

$- [\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n} \end{matrix}] = [\begin{matrix} - a_{1} \\ - a_{2} \\ ⋮ \\ - a_{n} \end{matrix}] .$

The proof that 𝕂ⁿ is a vector space is left to the reader, so we now regard as stating that ℝⁿ with the usual operations is a vector space over ℝ.

Example 2.2

The notation M_m,n, or simply M, will be used to denote the set of all m × n matrices over an arbitrary field 𝕂. Then M_m,n is a vector space over 𝕂 with respect to the usual operations of matrix addition and scalar multiplication.

Example 2.3

Let P denote the set of all polynomials

a₀ + a₁ t + a₂ t + … + a_n tⁿ, n = 0, 1, 2, 3, …,

with coefficients ai is some field 𝕂. Then P is a vector space over 𝕂 with respect to the usual operations of addition of polynomials and multiplication of polynomials by constants.

Example 2.4

Let X be any non empty set and let 𝕂 be an arbitrary field 𝕂. Consider the set F(X) of all functions from X into 𝕂. The sum of two functions f, g ∈ F(X) is the function f + g ∈ F(X) defined by

(f + g)(x) = f (x) + g(x), ∀x ∈ X,

and the product of a scalar k ∈ 𝕂 and a function f ∈ F(X) is the function k f ∈ F(X) defined by

(kf)(x) = k f (x), ∀x ∈ X.

Then F(X) with the above operations is a vector space over 𝕂.

The zero vector in F(X) is the zero function 0 which maps each x ∈ X into 0 ∈ 𝕂, that is,

0(x) = 0, ∀x ∈ X.

Also, for any function f ∈ F(X), the function -f defined by

(-f)(x) = -f (x), ∀x ∈ X,

is the additive inverse of the function f.

Let W be a subset of a vector space V over a field 𝕂. Then W is called a subspace of V if W is itself a vector space over 𝕂 with respect to the operations of vector addition and scalar multiplication on V. Simple criteria for identifying subspaces follow.

Theorem 2.3

Suppose that W is a subset of a vector space V. Then W is a subspace of V if and only if the following hold:

a. 0 ∈ W

b. W is closed under vector addition, that is:

For every u, v ∈ W, the sum u + v ∈ W.

c. W is closed under scalar multiplication, that is:

For every u ∈ W, k ∈ 𝕂, the product k u ∈ W.

Conditions b and c may be combined into one condition.

Corollary 2.1

W is a subspace if and only if

a. 0 ∈ W

b. W is closed under linear combinations, that is:

a u + bv ∈ W for every u, v ∈ W and a, b ∈ K.

Example 2.5

Let V be any vector space. Then the set {0} consisting of the zero vector alone, and also the entire space V are subspaces of V.

Example 2.6

Let W be the set of vectors in ℝ³ consisting of those vectors whose third component is 0; or, in other words

$W = {[\begin{array}{l} a \\ b \\ 0 \end{array}] : a, b, \in R} .$

Notice that 0 ∈ W. Further, for any vectors

$[\begin{array}{l} a \\ b \\ 0 \end{array}], [\begin{array}{l} c \\ d \\ 0 \end{array}] \in W,$

and scalars r, s ∈ ℝ, we have

$r [\begin{array}{l} a \\ b \\ 0 \end{array}] + s [\begin{array}{l} c \\ d \\ 0 \end{array}] = [\begin{matrix} r a + s c \\ r b + s d \\ 0 \end{matrix}] \in W .$

Thus W is a subspace of ℝ³.

Example 2.7

Let V = M_n,n, the space of n × n matrices. Then the subset W₁ of (upper) triangular matrices and the subset W₂ of matrices satisfying A^⊤ = A, called the set of symmetric matrices, are subspaces of V since they are non empty and closed under linear combinations.

Example 2.8

Recall that P denotes the vector space of polynomials. Let P_n denote the subset of P that consists of all polynomials of degree ≤ n, for a fixed n. Then P_n is a subspace of P.

Recall that any solution of a homogeneous system Ax = 0 is an element of ℝⁿ. Thus, the solution set of Ax = 0 is a subset of ℝⁿ.

Theorem 2.4

The solution set W of a homogeneous system Ax = 0 in n unknowns is a subset of Rⁿ.

Proof. First, 0 ∈ W because 0 is always a solution of a homogeneous system: A 0 = 0.

Next we show that W is closed under linear combinations. So let u, v ∈ W and a, b ∈ ℝ. Since u and v are solutions of the homogeneous system, we have A u = 0 and A v = 0. Moreover,

A(au + bv) = aAu + bAv = a0 + b0 = 0.

Consequently, a u + b v is a solution of the homogeneous solution and, thus, a u + b v ∈ W.

2.2 Linear combinations and spans

Let V be a vector space over a field 𝕂 and let v1, …, v_m ∈ V. Any vector in V of the form

a₁ v₁ + a₂ v₂ + … + a_m v_m

where ai ∈ 𝕂, is called a linear combination of v1, …, v_m. The set of all such linear combinations, denoted by

Span{v1, …, v_m},

is called the span of v₁, …, v_m.

Generally, for any subset S of V, Span S = {0} when S is empty and Span S consists of all the linear combinations of vectors in S.

Theorem 2.5

If v₁, …, v_m ∈ V then Span{v₁, …, v_m} is a subspace of V.

Proof. Since 0 ∈ V can be written as a linear combination of v₁, …, v_m:

0 = 0v₁ + … + 0vn,

then 0 ∈ Span{v₁, …, v_m}. To show that Span{v₁, …, v_m} is closed under linear combinations, choose any two vectors u, v ∈ Span{v₁, …, v_m}. Therefore,

u = a₁v₁ + a₂v₂ + … + amv_m,
v = b₁v₁ + b₂v₂ + … + bmv_m,

for some scalars a_i, b_i ∈ 𝕂. Then, for any r, s ∈ 𝕂,

$\begin{aligned} r u + s v & = r (a_{1} v_{1} + a_{2} v_{2} + \dots + a_{m} v_{m}) + s (= b_{1} v_{1} + b_{2} v_{2} + \dots + b_{m} v_{m}) \\ = (r a_{1} + s b_{1}) v_{1} + (r a_{2} + s b_{2}) v_{2} + \dots + (r a_{m} + s b_{m}) v_{m} \end{aligned}$

Then r u + s v is a linear combination of the vectors v₁, …, v_m, and, consequently, r u + s v ∈ Span{v₁, …, v_m}.

On the other hand, given a vector space V, the vectors u1, u2, …, ur are said to span or generate or to form a generating set of V if

V = Span{u₁, u₂, …, u_r}.

In other words, u₁, u₂, …, u_r span V if, for every v ∈ V, there are scalars a₁, a₂, …, a_r such that

v = a₁u₁ + a₂u₂ + … + a_rur,

that is, if v is a linear combination of u₁, u₂, …, u_r.

Example 2.9

Let

$H = {[\begin{matrix} a - 3 b \\ b - a \\ a \\ b \end{matrix}] : a, b \in R} .$

Show that H is a subspace of ℝ⁴.

In light of Theorem 2.5, if we can write H as the span of a set of vectors, then we can conclude that H is a subspace of ℝ⁴. Hence, write a generic element of H as a linear combination of vectors:

$[\begin{matrix} a - 3 b \\ b - a \\ a \\ b \end{matrix}] = a [\begin{array}{r} 1 \\ - 1 \\ 1 \\ 0 \end{array}] + b [\begin{array}{r} - 3 \\ 1 \\ 0 \\ 1 \end{array}], a, b \in R .$

This means that a generic element of H can always be expressed as a linear combination of the two vectors on the right. Therefore,

$H = Span {[\begin{array}{r} 1 \\ - 1 \\ 1 \\ 0 \end{array}], [\begin{array}{r} - 3 \\ 1 \\ 0 \\ 1 \end{array}]} .$

So H is a subspace by Theorem 2.5.

Example 2.10

For what value(s) of h will the polynomial q(t) be in the subspace of P₂ spanned by p₁ (t), p₂ (t), p₃(t), if

$p_{1} (t) = 1 - t - 2 t^{2}, p_{2} (t) = 5 - 4 t - 7 t^{2}, p_{3} (t) = - 3 + t, q (t) = - 4 + 3 t + h t^{2}$

The polynomial q(t) is in the subspace spanned by p₁ (t), p₂(t), p₃ (t) if and only if there are scalars a₁, a₂, a₃ ∈ ℝ such that

q(t) = a₁ p₁ (t) + a₂ p₂ (t) + a₂p₃(t).

The above equality holds if and only if the coefficients multiplying 1, t, and t², respectively, on the left side of the equation are equal to the corresponding coefficients on the right side. Therefore, comparing the corresponding coefficients we get,

$\begin{aligned} 1 : & - 4 & = a_{1} + 5 a_{2} - 3 a_{3} \\ t : & 3 & = - a_{1} - 4 a_{2} + a_{3} \\ t^{2} : & h & = - 2 a_{1} - 7 a_{2} + 0 a_{3} \end{aligned}$

or, in vector form,

$[\begin{array}{rrr} 1 & 5 & - 3 \\ - 1 & - 4 & 1 \\ - 2 & - 7 & 0 \end{array}] [\begin{array}{l} a_{1} \\ a_{2} \\ a_{3} \end{array}] = [\begin{array}{r} - 4 \\ 3 \\ h \end{array}] .$

We study the solution set of this equation by reducing the augmented matrix to its row reduced echelon form:

$[\begin{array}{rrrr} 1 & 5 & - 3 & - 4 \\ - 1 & - 4 & 1 & 3 \\ - 2 & - 7 & 0 & h \end{array}] \sim [\begin{array}{rrrr} 1 & 5 & - 3 & - 4 \\ 0 & 1 & - 2 & - 1 \\ 0 & 3 & - 6 & h - 8 \end{array}] \sim [\begin{array}{rrrr} 1 & 0 & 7 & 1 \\ 0 & 1 & - 2 & - 1 \\ 0 & 0 & 0 & h - 5 \end{array}] .$

Observe that the third row in the last matrix corresponds to the degenerate equation 0 = h - 5, which has a solution if and only if h = 5. Therefore, q(t) ∈ Span{p₁ (t), p₂(t), p₃(t)} if and only if h = 5.

2.3 Null space and column space of a matrix

There are two important subspaces associated with any m × n matrix A. In this section, we introduce both of these subspaces and study some of their properties.

Definition 2.2

The null space of an m × n matrix A, written as NulA, is the set of all solutions of the homogeneous equation A x = 0. In set notation,

NulA = {x ∈ ℝⁿ : Ax= 0}.

A more dynamic description of NulA is the set of all x ∈ ℝⁿ that are annihilated by A in the sense that each x ∈ NulA is “transformed” into the zero vector.

The term space in null space is appropriate because the null space of an m × n matrix is a vector subspace of ℝⁿ. This is a direct consequence of Theorem 2.4.

Example 2.11

Let H be the set of all vectors in R⁴ whose coordinates a, b, c, d satisfy the equations

a - 2b + 5c - d = 0,
-a - b + c = 0.

In other words, each element x of H satisfies the homogeneous equation

$[\begin{array}{rrrr} 1 & - 2 & 5 & - 1 \\ - 1 & - b & 1 & 0 \end{array}] x = 0$

So by the definition of the null space of a matrix, we have the following expression of H :

$H = Nul [\begin{array}{rrrr} 1 & - 2 & 5 & - 1 \\ - 1 & - b & 1 & 0 \end{array}] .$

Example 2.12

Find a generating set for the null space of the matrix

$A = [\begin{array}{rrrrr} - 3 & 6 & - 1 & 1 & - 7 \\ 1 & - 2 & 2 & 3 & - 1 \\ 2 & - 4 & 5 & 8 & - 4 \end{array}] .$

The first step is to find the general solution of the homogeneous equation A x = 0. So we reduce the augmented matrix to its row reduced echelon form:

$[\begin{array}{ll} A & 0 \end{array}] \sim [\begin{array}{rrrrrr} 1 & - 2 & 0 & - 1 & 3 & 0 \\ 0 & 0 & 1 & 2 & - 2 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \end{array}] .$

Then, the solution set of A x = 0 is the same as the solution set of the simpler system

$\begin{aligned} x_{1} - 2 x_{2} - x_{4} + 3 x_{5} & = 0, \\ x_{3} + 2 x_{4} - 2 x_{5} & = 0, \\ 0 & = 0, \end{aligned}$

with free variables x₂, x₄, x₅. If we write the leading variables in terms of the free variables, then we can express the general solution of A x = 0 as:

$[\begin{array}{l} x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \\ x_{5} \end{array}] = [\begin{matrix} 2 a + b - 3 c \\ a \\ - 2 b + 2 c \\ b \\ c \end{matrix}] = a [\begin{array}{l} 2 \\ 1 \\ 0 \\ 0 \\ 0 \end{array}] + b [\begin{array}{r} 1 \\ 0 \\ - 2 \\ 1 \\ 0 \end{array}] + c [\begin{array}{r} - 3 \\ 0 \\ 2 \\ 0 \\ 1 \end{array}] .$

This means that every solution of the homogeneous equation can be expressed as a linear combination of the three vectors on the right. Then, the generating set of NulA is

${[\begin{array}{l} 2 \\ 1 \\ 0 \\ 0 \\ 0 \end{array}], [\begin{array}{r} 1 \\ 0 \\ - 2 \\ 1 \\ 0 \end{array}], [\begin{array}{r} - 3 \\ 0 \\ 2 \\ 0 \\ 1 \end{array}]} .$

Using the notation of this and the previous section, we may write

$Nul A = Span {[\begin{array}{l} 2 \\ 1 \\ 0 \\ 0 \\ 0 \end{array}], [\begin{array}{r} 1 \\ 0 \\ - 2 \\ 1 \\ 0 \end{array}], [\begin{array}{r} - 3 \\ 0 \\ 2 \\ 0 \\ 1 \end{array}]} .$

Observation 2.1

The thoughtful reader would observe that the number of non zero vectors in the generating set of NulA equals the number of free variables in the system A x = 0.

Another important subspace associated with a matrix is its column space. Unlike the null space, the column space is defined explicitly via linear combinations.

Definition 2.3

The column space of an m × n matrix A, written as ColA, is the set of all linear combinations of the columns of A. If A = [a₁ … a_n], then

ColA = Span{a₁, …, a_n}.

The fact that ColA is a subspace of ℝ^m follows from Theorem 2.5. Note that by the definition of matrix multiplication, a typical vector in ColA can be written as A x for some x ∈ ℝⁿ. That is,

ColA = {b ∈ ℝ^m : b = A x for some x ∈ ℝⁿ}.

Example 2.13

Find a matrix A such that W = ColA, where

$W = {[\begin{matrix} 6 a - b \\ a + b \\ - 7 a \end{matrix}] : a, b \in R} .$

We first find a generating set of W and use the vectors in the generating set as columns of the matrix A.

$W = {a [\begin{array}{r} 6 \\ 1 \\ - 7 \end{array}] + b [\begin{array}{r} - 1 \\ 1 \\ 0 \end{array}] : a, b \in R} = Span {[\begin{array}{r} 6 \\ 1 \\ - 7 \end{array}], [\begin{array}{r} - 1 \\ 1 \\ 0 \end{array}]} .$

The matrix A whose columns are the vectors in the generating set of W is

$A = [\begin{array}{rr} 6 & - 1 \\ 1 & 1 \\ 0 & - 7 \end{array}] .$

Then W = ColA.

The following example is meant to emphasize the differences between the null space and the column space of a matrix.

Example 2.14

$Let A = [\begin{array}{rrrr} 2 & 4 & - 2 & 1 \\ - 2 & - 5 & 7 & 3 \\ 3 & 7 & - 8 & 6 \end{array}] .$

a. If the column space of A is a subspace of R^k, what is k?

b. If the null space of A is a subspace of R^k, what is k?

c. Find a non zero vector of ColA and a non zero vector of NulA?

d. Is there a vector in ColA that also belongs to NulA?

The answers to these questions are:

a. Since the columns of A belong to ℝ³, then they must generate a subspace of ℝ³. So k = 3.

b. The matrix A has four columns, which means that the product A x is defined if and only if x ∈ ℝ⁴. Therefore, the general solution of the homogeneous equation A x = 0 belongs to ℝ⁴. Hence, k = 4.

c. Finding a non zero element of ColA is easy. Clearly, any column of A belongs to ColA. Then it is sufficient to choose any column, say, the first one: $[\begin{array}{r} 2 \\ - 2 \\ 3 \end{array}]$ .

Finding a non zero element of NulA is harder. For this, we need to solve the homogeneous equation A x = 0. Reduce the augmented matrix to row reduced echelon form:

$[\begin{array}{rrrrr} 2 & 4 & - 2 & 1 & 0 \\ - 2 & - 5 & 7 & 3 & 0 \\ 3 & 7 & - 8 & 6 & 0 \end{array}] \sim [\begin{array}{rrrrr} 1 & 0 & 9 & 0 & 0 \\ 0 & 1 & - 5 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \end{array}]$

which corresponds to the system of equations

$\begin{aligned} x_{1} + 9 x_{3} & = 0, \\ x_{2} - 5 x_{3} & = 0, \\ x_{4} & = 0, \end{aligned}$

with x₃ free. The general solution of the homogeneous equation is

$[\begin{array}{l} x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \end{array}] = [\begin{array}{r} - 9 λ \\ 5 λ \\ λ \\ 0 \end{array}] = λ [\begin{array}{r} - 9 \\ 5 \\ 1 \\ 0 \end{array}] .$

Then, a no zero element of the null space of A is $[\begin{array}{r} - 9 \\ 5 \\ 1 \\ 0 \end{array}]$ .

d. Each element of ColA is a vector in ℝ³ and each element of NulA is a vector in ℝ⁴. Thus, with only three entries, the elements of ColA could not possible be in NulA.

NulA	ColA
1. NulA is a subspace of Rⁿ.	1. ColA is a subspace of R^m.
2. NulA is defined implicitly via the condition (A x = 0) that its elements must satisfy.	2. ColA is defined explicitly via how to construct its elements.
3. A typical vector v is NulA has that property that A v = 0.	3. A typical vector v is ColA has the property that Ax = v is consistent.
4. Given a specific vector v, it is to tell if v is in NulA. Just compute A v.	4. Given a specific vector v, it takes time to tell if v is in ColA. The solutions of A x = v must be studied.
5. NulA = {0} if and only if the equation A x = 0 has a unique solution.	5. ColA = ℝ^m if and only if the equation A x = b has a solution for every b ∈ ℝ^m.

Table 1: Contrast between NulA an ColA of an m × n matrix A.

2.4 Linear independence

We now define the notion of linear independence. This concept plays an essential role in the theory of linear algebra and in mathematics in general.

Definition 2.4

Let V be a vector space over a field 𝕂. The vectors u₁, …, u_m ∈ V are said to be linearly independent if the vector equation

$c_{1} u_{1} + c_{2} u_{2} + \dots + c_{m} u_{m} = 0 (2.1)$

has a unique solution: c₁ = c₂ = … = c_m = 0. The vectors u₁, …, um ∈ V are said to be linearly dependent if (2.1) has a non trivial solution.

If 0 is one of the vectors in (2.1), say u₁, then the vectors must be linearly dependent; for

1 u₁ + 0 u₂ + … + 0 um = 1 0 + 0 + … + 0 = 0,

which means that(2.1) has the non trivial solution c₁ = 1, c₂ = … = c_m = 0.

Any non zero vector u is, by itself, linearly independent; for the equation c u = 0 has the unique solution c = 0.

If two of the vectors in (2.1) are equal or one is a scalar multiple of the other, say u₁= k u₂, then the vectors are linearly dependent. For, in this case,

u₁ - k u₁ + 0 u₃ + … + 0 um = 0,

which means that (2.1) has the non trivial solution c₁ = 1, c₂ = −k, c₃ = … = c_m = 0. In particular, two vectors are linearly dependent if and only if one is a multiple of the other.

Clearly, if the vectors u₁, …, u_m ∈ V are linearly independent, then any rearrangement of these vectors is also linearly independent.

If the vectors u₁, …, u_m ∈ V are linearly independent, then any subset of these vectors, say u₁, …, ur (r < m), is linearly independent. Indeed, consider the equation

d₁ u₁ + d₂ u₂ + … + dr ur = 0.

But we can also write

d₁ u₁ + d₂ u₂ + … + d_r u_r + 0 u_r₊₁ + … + 0 u_m = 0.

Since u₁, …, um ∈ V are linearly independent, this equation has only the trivial solution, which means that d₁ = … = d_r = 0 and, thus, u₁, …, u_r are linearly independent. By contrapositive (see the Introduction), if some subset of u₁, …, u_m ∈ V, say u₁, …, u_r (r < m), is linearly dependent, then u₁, …, u_m is linearly dependent.

The following theorem will often be useful. It states that a linearly dependent list of vectors, with the first vector not 0, one of the vectors is in the span of the previous ones.

Theorem 2.6

An indexed set {u₁, …, u_m} of two or more vectors, with u₁ ≠ 0, is linearly dependent if and only if some uj (j > 1) is a linear combination of the preceding vectors u₁, …, uj-₁.

Proof. Suppose that, for some j > 1, uj is a linear combination of u₁, …, u_j-1. Then there are scalars d₁, …, d_j-1, such that

uj = d₁ u₁ + d₂ u₂ + … + d_j-1 u_j-1,

or, equivalently,

-d₁ u₁- … - d_j-1 u_j-1 + uj + 0 u_j+1 + … + 0 u_m = 0.

Therefore, (2.1) has the non trivial solution c₁ = −d₁, …, c_j-1 = −d_j-1, cj = 1, c_j+1 = … = c_m = 0, which means that u₁, …, u_m are linearly dependent.

On the other hand, suppose that u₁, …, u_m are linearly dependent. Then there are scalars a₁, a₂, …, a_m, not all equal to 0, such that

a₁ u₁ + a₂ u₂ + … + am um = 0.

Not all a₂, a₃, …, a_m can be equal to 0. Otherwise, we would have a₁ u₁= 0; but u₁≠ 0 which means that a₁ = 0. Then, in this case, a₁ = … = a_m = 0 making the vectors linearly independent which contradicts our assumption. Let j be the largest element of {2, …, m} such that a_j ≠ 0. Then,

$u_{j} = - \frac{a_{1}}{a_{j}} u_{1} - ? - \frac{a_{j - 1}}{a_{j}} u_{j - 1}$

That is, there is some uj that is a linear combination of the preceding vectors u₁, …, u_j-1.

Now we come to a key result. It says that, if in a set of vectors one can be written as a linear combination of the remaining ones, then we can throw out that vector without changing the span of the original list.

Theorem 2.7

Let S = {u₁, …, u_m} be a set in a vector space V, and let H = Span S. If one of the vectors – say u_k – is a linear combination of the remaining vectors in S, then the set formed from S by removing u_k still spans H.

Proof. By rearranging the list of vectors in S, if necessary, we may suppose that um is a linear combination of u₁, …, u_m_-1^– say

um = a₁ u₁ + a₂ u₂ + … + a_m-1 u_m-1.

Given any w ∈ H, we may write

w = c₁ u₁ + c₂u₂ + … + c_mum

for suitable scalars c₁, c₂, …, c_m. Substituting the expression for um we obtain

w = (c₁ + c_m a₁) u₁ + (c₂ + c_m a₂) u₂ + … + (c_m_-1 + c_m a_m_-1) u_m-1.

Thus, w is a linear combination of u₁, …, u_m_-1. This means that {u₁, …, u_m_-1} spans H because w was an arbitrary element of H.

Consider a set of vectors S = {u₁, …, u_m} in a vector space V. We now know that for any vector w ∈ V, the equation

$w = x_{1} u_{1} + x_{2} u_{2} + \dots + x_{m} u_{m} (2.2)$

has a solution if and only if x ∈ Span S. So the notion of generating set deals with the existence of solutions of vector equations such as (2.2). So, what about uniqueness? It turns out that vector equations such as (2.2) with unique solutions can be characterized in terms of the notion of linear independence.

Theorem 2.8

Let S = {u₁, …, u_m} be a set in a vector space V. Then each w ∈ Span S has a unique expression

w = c₁ u₁ + c₂ u₂ + … + c_mum,

for suitable scalars c₁, …, c_m if and only if S is a linearly independent set.

Proof. Suppose that each w ∈ Span S has a unique expression as a linear combination of the vectors in S. In particular, 0 ∈ Span S so

c₁ u₁ + c₂ u₂ + … + c_m u_m = 0,

which clearly is satisfied with c₁ = c₂ = … = c_m = 0. But this expression is unique which means that S is a linearly independent set of vectors.

Now suppose that S is a linearly independent set. If a vector w ∈ Span S can be expressed as

w = c₁ u₁ + c₂ u₂ + … + c_mu_m,

and

w = d₁ u₁ + d₂ u₂ + … + dm u_m,

then subtracting the expressions we get

0 = (c₁ - d₁) u₁ + (c₂ - d₂) u₂ + … + (c_m - d_m) u_m.

But S is a linearly independent set, so the last homogeneous equations has the unique solution c₁ - d₁ = c₂ - d₂ = … = c_m - d_m = 0. Clearly, this means that c₁ = d₁, c₂ = d₂, …, c_m = d_m.

Example 2.15

Consider the three vectors in ℝ³:

$e_{1} = [\begin{array}{l} 1 \\ 0 \\ 0 \end{array}], e_{2} = [\begin{array}{l} 0 \\ 1 \\ 0 \end{array}], e_{3} = [\begin{array}{l} 0 \\ 0 \\ 1 \end{array}] .$

Clearly, the homogeneous equation

$[\begin{array}{l} 0 \\ 0 \\ 0 \end{array}] = x_{1} e_{1} + x_{2} e_{2} + x_{3} e_{3} = [\begin{array}{lll} 1 0 0 \\ 0 1 0 \\ 0 0 1 \end{array}] [\begin{array}{l} x_{1} \\ x_{2} \\ x_{3} \end{array}] = [\begin{array}{l} x_{1} \\ x_{2} \\ x_{3} \end{array}],$

has only the trivial solution x₁ = x₂ = x₃ = 0. Therefore, e₁, e₂, e₃ are independent vectors. In fact, since it is obvious that ℝ³ = Span{e₁, e₂, e₃}, we conclude that each vector in ℝ³ has a unique expression as linear combination of e₁, e₂, e₃.

Example 2.16

Suppose that the scalars c₀, c₁, …, c_n satisfy

$c_{0} + c_{1} t + c_{2} t^{2} + \dots + c_{n} t^{n} = 0$

This equality means that the polynomial on the left has the same values as the zero polynomial on the right. It is well known that a polynomial of degree n with more than n roots is the zero polynomial. Therefore, c₀ = c₁ = … = cn = 0. This proves that the set {1, t, t², …, tⁿ} is a linearly independent set in P_n. It is obvious that P_n = Span{1, t, t², …, tⁿ}. Then each polynomial of degree at most n has a unique expression as a linear combination of 1, t, t², …, tⁿ.

Recall that any linear dependence relationship among the columns of a matrix A can be expressed in the form A x = 0, where x is a column of weights. When A is row reduced to a matrix B, the columns of B are often totally different from the columns of A. However, the equations A x = 0 and B x = 0 have exactly the same set of solutions. That is, the columns of A have exactly the same linear dependence relationships as the columns of B. So we have the following important observation:

Observation 2.2

Elementary row operations on a matrix preserve the linear dependence relations among the columns of the matrix.

The previous observation provides a useful shortcut for reducing a matrix to its row reduced echelon form when the linear dependence relations among the columns of a matrix are easy to detect.

Example 2.17

Consider the matrix

$A = [\begin{array}{lllll} a_{1} a_{2} a_{3} a_{4} a_{5} \end{array}] = [\begin{array}{rrrrr} 1 4 0 2 - 1 \\ 3 12 1 5 5 \\ 2 8 1 3 2 \\ 5 20 2 8 8 \end{array}] .$

a. Find the smallest generating set for ColA.

b. Use the linear dependence relations among the columns of A for compute its row reduced echelon form.

These are the solutions:

a. Recall that ColA = Span{a₁, a₂, a₃, a₄, a₅}. By Theorem 2.7, the set of columns formed by removing those columns that are linear combinations of the remaining ones still spans ColA. Then, since

a₂ = 4 a₁ and a₄ = 2 a₁ - a₃,

we can remove a₂ and a₄ from the spanning set. Therefore, ColA = Span{a₁, a₃, a₅}. Since a₁, a₃, a₅ are linearly independent (you should show this), no more columns should be removed. This means that {a₁, a₃, a₅} is the smallest generating set of ColA.

b. The columns of the row reduced echelon form B = [b₁b₂b₃b₄b₅] should have the same linear dependence relations as the columns of A. Therefore,

b₂ = 4b₁, b₄ = 2b₁ - b₃,

and b₁, b₃, b₅ are linearly independent. The pivot columns of a matrix in row reduced echelon form are linearly independent columns since they are the columns of the identity matrix. Moreover, all the rows of zeros should be at the bottom of the row reduced echelon matrix. From these two facts we deduce that b₁, b₃, b₅ are pivots and correspond to the first three columns of the identity matrix, respectively. Putting everything together, we obtain

$A \sim [\begin{array}{rrrrr} 1 4 0 2 0 \\ 0 0 1 - 1 0 \\ 0 0 0 0 1 \\ 0 0 0 0 0 \end{array}] .$

2.5 Bases and dimension

In a vector space V, sets that guarantee that each vector in V con be expressed uniquely in terms of these sets are one of the most important objects in linear algebra.

Definition 2.5

An ordered set $B = \{u_{1}, \dots, u_{n}\}$ of vectors is a basis of V if the following two conditions hold:

1. $B$ is a linearly independent set.

2. V = Span $B$ .

The following characterization of a basis is a direct consequence of Theorem 2.8.

Theorem 2.9

An ordered set $B$ = {u₁, …, u_n} of vectors is a basis of V if and only if every vector v ∈ V can be written uniquely as a linear combination of the basis vectors.

We have already seen an example of a basis of ℝ³ in Example 2.15. Such basis consisting of the columns of the identity matrix is called the canonical basis of ℝ³.

Example 2.16 presents a basis of P_n. Such basis consisting of consecutive powers of the variable t is called the standard basis of P_n.

Let us consider less obvious examples of bases.

Example 2.18

Let

$v_{1} = [\begin{array}{r} 3 \\ 0 \\ - 6 \end{array}], v_{2} = [\begin{array}{r} - 4 \\ 1 \\ 7 \end{array}], v_{3} = [\begin{array}{r} - 2 \\ 1 \\ 5 \end{array}] .$

Determine if {v₁, v₂, v₃} is a basis of ℝ³.

We must show that these vector are linearly independent and generate ℝ³. Both of these conditions hold if and only if the equation [v₁v₂v₃] x = b has a unique solution for each b ∈ ℝ³. This equation has a solution for each b if each row of the matrix A = [v₁v₂v₃] has a pivot and the solution is unique if each column of A is a pivot column. In other words, the equation A x = b has a unique solution for each b if A ∼ I. So we row reduce A to its row reduced echelon form:

$A \sim [\begin{array}{rrr} 3 - 4 - 2 \\ 0 1 1 \\ 0 - 1 1 \end{array}] \sim [\begin{array}{lll} 3 0 2 \\ 0 1 1 \\ 0 0 2 \end{array}] \sim [\begin{array}{lll} 3 0 0 \\ 0 1 0 \\ 0 0 2 \end{array}] \sim [\begin{array}{lll} 1 0 0 \\ 0 1 0 \\ 0 0 1 \end{array}] .$

Therefore {v₁, v₂, v₃} is a basis of ℝ³.

Example 2.19

Let

p₀(t) = (1 - t)³, p1 (t) = 3t (1 - t)², p2(t) = 3t² (1 - t), p₃(t) = t³.

Determine if {p₀(t), p₁ (t), p₂(t), p₃(t)} is a basis for P₃.

We need to determine if each polynomial in P₃ can be expressed uniquely as a linear combination of p₀(t), p₁ (t), p₂(t), p₃(t). Let q(t) = a₀ + a₁ t + a₂ t² + a₃ t³ and suppose that the scalars c₀, c₁, c₂, c₃ satisfy

q(t) = c₀ p₀ (t) + c₁ p₁ (t) + c₂ p₂ (t) + c₃p₃(t).

Comparing the coefficients multiplying each power of t on both sides of the equality, we obtain

$\begin{aligned} c_{0} = a_{0}, \\ - 3 c_{0} + 3 c_{1} = a_{1}, \\ 3 c_{0} - 6 c_{1} + 3 c_{2} = a_{2}, \\ - c_{0} + 3 c_{1} - 3 c_{2} + c_{3} = a_{3}, \end{aligned}$

or in matrix form,

$A c = [\begin{array}{rrrr} 1 0 0 0 \\ - 3 3 0 0 \\ 3 - 6 3 0 \\ - 1 3 - 3 1 \end{array}] [\begin{array}{l} c_{0} \\ c_{1} \\ c_{2} \\ c_{3} \end{array}] = [\begin{array}{l} a_{0} \\ a_{1} \\ a_{2} \\ a_{3} \end{array}] .$

It is not difficult to see that the matrix A has a pivot in every row and every column, which means that the equation above has a unique solution for each choice of the polynomial q(t). Therefore, {p₀(t), p₁ (t), p₂(t), p₃(t)} is a basis for P₃.

The following theorem characterizes an important basis of the column space of a matrix.

Theorem 2.10

The pivot columns of a matrix A form a basis for ColA.

Proof. Let B be the reduced echelon form of A. The set of pivot columns of B is linearly independent, for no vector in the set is a linear combination of the vectors that precede it. Since A is row equivalent to B, the pivot columns of A are linearly independent as well, because any linear dependence relation among the columns of A corresponds to a linear dependence relation among the columns of B. For this same reason, every non pivot column of A is a linear combination of the pivot columns of A. Thus the non pivot columns of A may be discarded from the spanning set for ColA. This leaves the pivot columns of A as a basis for ColA.

Warning: Be careful to use pivot columns of A itself for the basis of ColA. Row operations can change the column space of a matrix. We illustrate this in the following example.

Example 2.20

Consider again the matrix given in Example 2.17:

$A = [\begin{array}{lllll} a_{1} a_{2} a_{3} a_{4} a_{5} \end{array}] = [\begin{array}{rrrrr} 1 4 0 2 - 1 \\ 3 12 1 5 5 \\ 2 8 1 3 2 \\ 5 20 2 8 8 \end{array}] .$

We showed that

$A \sim [\begin{array}{rrrrr} 1 4 0 2 0 \\ 0 0 1 - 1 0 \\ 0 0 0 0 1 \\ 0 0 0 0 0 \end{array}] .$

It can be seen from the row reduced echelon form that the first, third, and last columns are pivot columns, which means that {a₁, a₃, a₅} is a basis for ColA. However, the columns of the row reduced echelon form all have zeros in their last entries, so they cannot span the column space of A.

Every generating set in a vector space can be reduced to a basis of the vector space. Indeed, removing vectors that are linear combinations of the remaining vectors will not change the span of the original set. The deletion of vectors from the generating set must stop when the set becomes linearly independent. If additional vectors are deleted, it will not be a linear combination of the remaining vectors, and hence the smaller set will no longer span V. Thus a basis is a generating set that is as small as possible.

Also, every linearly independent list of vectors can be extended to a basis of the vector space. This is done by adjoining a vector that is not in the span of the original list. The extended list of vectors is still linearly independent because the new vector cannot be written as a linear combination of the preceding ones. This step is repeated until a generating set of V is obtained. The addition of vectors must stop as soon as a generating set for V is obtained. For if one more vector is added, then the new set cannot be linearly independent because the old set spans V, and the new vector is therefore a linear combination of the vectors preceding it. Thus, a basis is a linearly independent set that is as large as possible.

Remark 2.1

Let B = {u₁, …, u_n} be a basis of a vector space V. Then every generating set of V has at least n elements, and every linearly independent set in V has at most n elements.

We say that a vector space V is finite-dimensional if it has a basis with a finite number of elements.

The notion of basis would be useless if different basis of the same vector space V had different numbers of elements. Fortunately, that turns out not to be the case.

Theorem 2.11

Any two bases of a finite-dimensional vector space have the same number of elements.

Proof. Suppose that V is a finite-dimensional vector space. Let $B_{1}$ and $B_{2}$ be two bases of V. Then $B_{1}$ and $B_{2}$ are linearly independent in V, so the number of vectors in $B_{1}$ is at most the number of elements in $B_{2}$ . Moreover, $B_{1}$ and $B_{2}$ span V, then the number of vectors in $B_{1}$ is at least the number of elements in $B_{2}$ . Thus the number of vectors in $B_{1}$ and $B_{2}$ must be equal.

Now we know that the number of vectors in a basis of a finite-dimensional vector space is an inherent property of V (that is, it does not depend on the choice of basis). Therefore we give the following definition.

Definition 2.6

The dimension of a finite-dimensional vector space, denoted by dim V, is the number of vectors in any basis of V. In other words, if $B = \{u_{1}, \dots, u_{n}\}$ is a basis of V, then dim V = n.

If V is a finite-dimensional vector space, then every generating set of vectors of V with n = dim V elements is a basis of V. For suppose that v₁, …, v_n span V. However, every basis of V has n elements, so in this case the reduction must be the trivial one, meaning that no elements are deleted from v₁, …, v_n. In other words, v₁, …, v_n is a basis of V, as desired.

Moreover, every linearly independent set of vectors in V with n elements is a basis of V. For suppose that {v₁, …, v_n} is a linearly independent set. Then this set can be extended to a basis of V. However, every basis of V has n elements, so in this case the extension must be the trivial one, meaning that no vectors are adjoined to {v₁, …, v_n}. In other words, {v₁, …, v_n} is a basis of V.

Example 2.21

The canonical basis for ℝⁿ contains n vectors, so dim ℝⁿ = n. The standard polynomial basis {1, t, t², …, tⁿ} shows that dim P_n = n + 1.

Example 2.22

Find the dimension of the subspace

$H = \{[\begin{matrix} a - 3 b + 6 c \\ 5 a + 4 d \\ b - 2 c - d \\ 5 d \end{matrix}] : a, b, c, d \in R\} .$

The strategy is to produce a basis for H by finding a generating set and then reduce it to a linearly independent generating set. The number of elements in the final set is equal to dim H.

First, we find a generating set for H. A generic vector in H can be written as

$[\begin{matrix} a - 3 b + 6 c \\ 5 a + 4 d \\ b - 2 c - d \\ 5 d \end{matrix}] = a [\begin{array}{l} 1 \\ 5 \\ 0 \\ 0 \end{array}] + b [\begin{array}{r} - 3 \\ 0 \\ 1 \\ 0 \end{array}] + c [\begin{array}{r} 6 \\ 0 \\ - 2 \\ 0 \end{array}] + d [\begin{array}{r} 0 \\ 4 \\ - 1 \\ 5 \end{array}], a, b, c, d \in R .$

Then

$H = Span \{[\begin{array}{l} 1 \\ 5 \\ 0 \\ 0 \end{array}], [\begin{array}{r} - 3 \\ 0 \\ 1 \\ 0 \end{array}], [\begin{array}{r} 6 \\ 0 \\ - 2 \\ 0 \end{array}], [\begin{array}{r} 0 \\ 4 \\ - 1 \\ 5 \end{array}]\} .$

This is not a basis, however. Notice that the third vector is a multiple of the second vector, so it can be removed from the set. The remaining vectors are linearly independent. Thus, we have found a basis for H ; namely,

$\{[\begin{array}{l} 1 \\ 5 \\ 0 \\ 0 \end{array}], [\begin{array}{r} - 3 \\ 0 \\ 1 \\ 0 \end{array}], [\begin{array}{r} 0 \\ 4 \\ - 1 \\ 5 \end{array}]\} .$

Since it has three vectors, we conclude that dim H = 3.

Example 2.23

Find the dimensions of the null space and the column space of the matrix

$A = [\begin{array}{rrrrr} - 3 6 - 1 1 - 7 \\ 1 - 2 2 3 - 1 \\ 2 - 4 5 8 - 4 \end{array}] .$

The pivot columns of A form a basis of ColA, so the dimension of the column space is the number of pivot columns. Row reduce the matrix to row reduced echelon form:

$A \sim B = [\begin{array}{rrrrr} 1 - 2 0 - 1 3 \\ 0 0 1 2 - 2 \\ 0 0 0 0 0 \end{array}] .$

Therefore, dim ColA = 2.

Now we find a basis for NulA. If we row reduce the augmented matrix [A 0] we obtain [B 0]. The corresponding system of linear equations is:

$\begin{aligned} x_{1} - 2 x_{2} - x_{4} + 3 x_{5} = 0, \\ x_{3} + 2 x_{4} - 2 x_{5} = 0, \\ 0 = 0, \end{aligned}$

with x₂, x₄, x₅ free. Its general solution is

We already know that the number of generating vectors produced with this method is equal to the number of free variables. Furthermore, this method produces automatically a set of linearly independent vectors because the free variables are the weights on the spanning vectors. For instance, look at the second, fourth, and fifth entries of the three vectors above, and note that 0 is a linear combination of these vectors if and only if a = b = c = 0. Therefore, these vectors form a basis for NulA and dim NulA = 3.

2.6 Coordinates

The purpose of bases in vector spaces is to provide a method of computation, and we are going to learn to use them in this section. We will consider two topics: how to express a vector in terms of a given basis, and how to relate two different bases of the same vector space.

Let V be a vector space with dim V = n over a field 𝕂, and suppose that

$B = \{u_{1}, u_{2}, \dots, u_{n}\}$

is a basis of V. Then any vector v ∈ V can be expressed uniquely as a linear combination of the basis vectors in $B$ , say

v = a₁ u₁ +a₂u₂ + … +a_nu_n.

The n scalars a₁, a₂, …, a_n are called the coordinates of v relative to $B$ : and they form the vector

$[v]_{B} = [\begin{matrix} a_{1} \\ a_{2} \\ ⋮ \\ a_{n} \end{matrix}]$

called the coordinate vector of v relative to $B$ .

Example 2.24

Let ε₃ = {e₁, e₂, e₃} be the canonical basis for ℝ³. That is,

$e_{1} = [\begin{array}{l} 1 \\ 0 \\ 0 \end{array}], e_{2} = [\begin{array}{l} 0 \\ 1 \\ 0 \end{array}], e_{3} = [\begin{array}{l} 0 \\ 0 \\ 1 \end{array}] .$

For any vector x ∈ ℝ³, it is obvious that [x]ε₃ = x.

Example 2.25

Let S_n = {1, t, t², …, tⁿ} be the standard basis of P_n. For any polynomial q(t) = a₀ + a₁ t + a₂ t² + … + a_n tⁿ, we have

$[q (t)]_{S_{n}} = [\begin{matrix} a_{0} \\ a_{1} \\ ⋮ \\ a_{n} \end{matrix}] .$

Example 2.26

Let

$v = [\begin{array}{l} 1 \\ 4 \end{array}], u_{1} = [\begin{array}{l} 1 \\ 2 \end{array}], u_{2} = [\begin{array}{l} 3 \\ 5 \end{array}] .$

Find the coordinates of v relative to the basis $B$ = {u₁, u₂}.

We must find scalars a₁, a₂ such that v = a₁ u₁ + a₂ u₂, or, in matrix form

$[\begin{array}{ll} 1 3 \\ 2 5 \end{array}] [\begin{array}{l} a_{1} \\ a_{2} \end{array}] = [\begin{array}{l} 1 \\ 4 \end{array}] .$

Row reduce the augmented matrix:

$[\begin{array}{ccc} 1 & 3 & 1 \\ 2 & 5 & 4 \end{array}] \sim [\begin{array}{ccc} 1 & 0 & 7 \\ 0 & 1 & - 2 \end{array}] .$

Therefore,

$[v]_{B} = [\begin{array}{r} 7 \\ - 2 \end{array}] .$

We want to emphasize the $v \neq [v]_{B}$ . The correct relation between v and its coordinates $[v]_{B}$ is v = 7 u₁ - 2 u₂.

The following example is to remind you that the material in this course is applicable to every vector space, not just to the well-known ones like ℝⁿ and P_n.

Example 2.27

Let us consider the vector space V generated by the set of linearly independent functions $B = \{e^{- i π t}, e^{- i \frac{n}{2} t}, 1, e^{i \frac{n}{2} t}, e^{i π t}\}$ . Observe that $B$ is a basis for V and dim V = 5. Some examples of vector in V are

$\cos (\frac{π n}{2} t), \sin (\frac{π n}{2} t), n = - 2, - 1, 0, 1, 2$

We will use Euler’s formula

$e^{i θ} = \cos θ + i \sin θ, θ \in R, i = \sqrt{- 1}$

to find the coordinates of some of these vectors relative to $B$ .

By Euler’s identity

$\cos (π t) = \frac{e^{i π t} + e^{- i π t}}{2} and \sin (\frac{π}{2} t) = \frac{e^{i \frac{π}{2} t} - e^{- i \frac{π}{2} t}}{2 i} .$

Therefore,

$[\cos (π t)]_{B} = [\begin{matrix} 1 / 2 \\ 0 \\ 0 \\ 0 \\ 1 / 2 \end{matrix}] and {[\sin (\frac{π}{2} t)]}_{B} = [\begin{array}{r} 0 \\ - \frac{1}{2 i} \\ 0 \\ \frac{1}{2 i} \\ 0 \end{array}] .$

Another example is

${[1 + 2 \cos (π t) - 2 i \sin (\frac{π}{2} t)]}_{B} = [\begin{array}{r} 1 \\ 1 \\ 1 \\ - 1 \\ 1 \end{array}] .$

The coordinates relative to any basis have the property that they preserve the main structure of vector spaces: linear combinations.

Theorem 2.12

Let $B$ = {u₁, …, u_n} be a basis for a vector space V. For any v, w ∈ V and scalar r:

1. $[v + w]_{B} = [v]_{B} + [w]_{B}$

2. $[r v]_{B} = r [v]_{B}$

Proof. Take two typical vectors in V, say

v = c₁ u₁ + c₂ u₂ + … + c_nu_n,
w = d₁ u₁ + d₂ u₂ + … + d_n u_n.

Then, using vector operations,

v + w = (c₁ + d₁) u₁ + (c₂ + d₂) u₂ + … + (c_n + d_n) u_n.

It follows that

$[v + w]_{B} = [\begin{matrix} c_{1} + d_{1} \\ c_{2} + d_{2} \\ ⋮ \\ c_{n} + d_{n} \end{matrix}] = [\begin{matrix} c_{1} \\ c_{2} \\ ⋮ \\ c_{n} \end{matrix}] + [\begin{matrix} d_{1} \\ d_{2} \\ ⋮ \\ d_{n} \end{matrix}] = [v]_{B} + [w]_{B} .$

If r is any scalar, then

rv = (rc₁)u₁ + (rc₂)u₂ + … + (rc_n)u_n.

$[r v]_{B} = [\begin{array}{cc} r & c_{1} \\ r & c_{2} \\ ⋮ \\ r & c_{n} \end{array}] = r [\begin{matrix} c_{1} \\ c_{2} \\ ⋮ \\ c_{n} \end{matrix}] = r [v]_{B} .$

We now come to a very important computational method: change of basis. Identifying vectors in V with column vectors in 𝕂ⁿ is useful when a natural basis is presented to us, but not when the given basis is poorly suited to the problem at hand. In that case, we will want to change basis. So let us suppose that we are given two bases for the same vector space V, say $B$ = {b₁, …,b_n} and $C$ = {c₁, …, c_n}. We will think of $B$ as the old basis, and $C$ as a new basis. There are two computations which we wish to clarify. We ask first: How are the two bases related? Secondly, a vector v ∈ V will have coordinates relative to each of these bases, but of course they will be different. So we ask: How are the two coordinate vectors related? These are the computations called change of basis.

We begin by noting that since the new basis spans V, every vector in the old basis $B$ is a linear combination of the new basis $C$ . So we can write the coordinates of each vector in $B$ relative to $C$ :

[b₁]_C, [b₂]_C, … [b_n]_C.

Now let

$[v]_{B} = [\begin{matrix} v_{1} \\ v_{2} \\ ⋮ \\ v_{n} \end{matrix}]$

be the coordinate vector of v relative to the old basis $B$ . That is,

v = v₁ b₁ + v₂b₂ + … + v_nb_n.

Using Theorem 2.12, we compute the coordinate vector of v relative to the new basis $C$ :

$[v]_{C} = {[v_{1} b_{1} + v_{2} b_{2} + \dots + v_{n} b_{n}]}_{C} = v_{1} {[b_{1}]}_{C} + v_{2} {[b_{2}]}_{C} + \dots + v_{n} {[b_{n}]}_{C} .$

If we define the matrix

$P_{C}^{B} = [\begin{array}{llll} {[b_{1}]}_{C} {[b_{2}]}_{C} \dots {[b_{n}]}_{C} \end{array}],$

then we can write

$[v]_{C} = P_{C}^{B} [v]_{B}$

Following the same argument as above but interchanging the roles of the bases B and C, we obtain

$[v]_{B} = P_{B}^{C} [v]_{C}$

where

$P_{B}^{C} = [\begin{array}{llll} {[c_{1}]}_{B} {[c_{2}]}_{B} \dots {[c_{n}]}_{B}] . \end{array}$

Then we make the following observation: $P_{C}^{B}$ is an invertible matrix and ${(P_{C}^{B})}^{- 1} = P_{B}^{C}$ .

Recapitulating, we have an invertible matrix $P_{C}^{B}$ , called the matrix of change of basis from $B$ to $C$ , whose columns are the coordinate vectors of each element of the old basis $B$ relative to the new basis $C$ , which transforms [v]_$B$ into [v]_$C$. Moreover, the columns of ${(P_{C}^{B})}^{- 1}$ are the coordinate vectors of each element of the new basis $C$ relative to the old basis $B$ .

An important consequence of the computations described above is that, given any basis $B$ of V, it is possible to obtain a new basis $C$ = {c₁, …, c_n} by choosing any invertible n × n matrix P = [a₁ a₂… a_n] and letting its columns be the coordinate vectors of the vectors in the new basis relative to the old one. That is,

${[c_{1}]}_{B} = a_{1}, {[c_{2}]}_{B} = a_{2}, \dots, {[c_{n}]}_{B} = a_{n}$

In this case, $P = P_{B}^{C}$ .

Example 2.28

Let ε₃ = {e₁, e₂, e₃} be the canonical basis for ℝ³. That is,

$e_{1} = [\begin{array}{l} 1 \\ 0 \\ 0 \end{array}], e_{2} = [\begin{array}{l} 0 \\ 1 \\ 0 \end{array}], e_{3} = [\begin{array}{l} 0 \\ 0 \\ 1 \end{array}] .$

Recall that for any vector x ∈ ℝ³, it is obvious that [x]_ε₃ = x. Furthermore, consider another basis for $B$ = {b₁, b₂, b₃}, where

$b_{1} = [\begin{array}{l} 1 \\ 2 \\ 0 \end{array}], b_{2} = [\begin{array}{r} 5 \\ 4 \\ - 2 \end{array}], b_{3} = [\begin{array}{r} 0 \\ - 1 \\ 0 \end{array}] .$

The change of basis matrix from $B$ to ε₃ is

$P_{E_{3}}^{B} = [{[b_{1}]}_{E_{3}} {[\begin{array}{l} b_{2} \end{array}]}_{E_{3}} {[b_{3}]}_{E_{3}}] = [\begin{array}{rrr} 1 5 0 \\ 2 4 - 1 \\ 0 - 2 0 \end{array}] .$

We can also compute $P_{B}^{E_{3}} = {(P_{E_{3}}^{B})}^{- 1}$ :

$P_{B}^{E_{3}} = [\begin{array}{lll} {[e_{1}]}_{B} {[\begin{array}{ll} e_{2} \end{array}]}_{B} {[e_{3}]}_{B} \end{array}] = [\begin{array}{rrr} 1 0 \frac{5}{2} \\ 0 0 - \frac{1}{2} \\ 2 - 1 3 \end{array}] .$

It is interesting to verify that the columns of $P_{B}^{E_{3}}$ are, indeed, the coordinate vectors of the elements of the canonical basis relative to $B$ :

$e_{1} = b_{1} + 2 b_{3}, e_{2} = - b_{3}, e_{3} = \frac{5}{2} b_{1} - \frac{1}{2} b_{2} + 3 b_{3} .$

Example 2.29

Let

$b_{1} = [\begin{array}{r} - 9 \\ 1 \end{array}], b_{2} = [\begin{array}{l} - 5 \\ - 1 \end{array}], c_{1} = [\begin{array}{r} 1 \\ - 4 \end{array}], c_{2} = [\begin{array}{r} 3 \\ - 5 \end{array}],$

and consider the bases for ℝ² given by $B$ = {b₁, b₂} and $C$ = {c₁, c₂}. Find the change of basis matrices $P_{C}^{B}$ and $P_{B}^{C}$ .

First, the matrix $P_{C}^{B}$ involves the coordinate vectors of b₁ and b₂ relative to $C$ . Let ${[b_{1}]}_{C} = [\begin{array}{l} x_{1} \\ x_{2} \end{array}]$ and ${[b_{2}]}_{C} = [\begin{array}{l} y_{1} \\ y_{2} \end{array}]$ . Then, by defintion

$[\begin{array}{ll} c_{1} c_{2} \end{array}] [\begin{array}{l} x_{1} \\ x_{2} \end{array}] = b_{1} and [\begin{array}{ll} c_{1} c_{2} \end{array}] [\begin{array}{l} y_{1} \\ y_{2} \end{array}] = b_{2} .$

To solve both systems simultaneously, augment the coefficient matrix with b₁and b₂, and row reduce:

$[\begin{array}{llll} c_{1} c_{2} b_{1} b_{2} \end{array}] = [\begin{array}{rrrr} 1 3 - 9 - 5 \\ - 4 - 5 1 - 1 \end{array}] \sim [\begin{array}{rrrr} 1 0 6 4 \\ 0 1 - 5 - 3 \end{array}] .$

Thus,

${[b_{1}]}_{C} = [\begin{array}{r} 6 \\ - 5 \end{array}] and {[b_{2}]}_{C} = [\begin{array}{r} 4 \\ - 3 \end{array}]$

The matrix $P_{C}^{B}$ is therefore

$P_{C}^{B} = [\begin{array}{ll} {[b_{1}]}_{C} {[b_{2}]}_{C}] = [\begin{array}{rr} 6 & 4 \\ - 5 & - 3 \end{array}] . \end{array}$

We can also compute $P_{B}^{C} = {(P_{C}^{B})}^{- 1}$ :

$P_{B}^{C} = [\begin{array}{ll} {[c_{1}]}_{B} {[c_{2}]}_{B} \end{array}] = [\begin{array}{rr} - \frac{3}{2} - 2 \\ \frac{5}{2} 3 \end{array}] .$

Example 2.30

Let

p₀(t) = (1 - t)³, p₁ (t) = 3t (1 - t)², p₂(t) = 3t² (1 - t), p₃(t) = t³.

and consider the bases for P₃ given by $B$ = {p₀(t), p₁ (t), p₂(t), p₃(t)} and S₃ = {1, t, t², t³}. Use a change of basis matrix to write the polynomial q(t) = t² + 1 as a linear combination of the basis $B$ .

Observe that the polynomial q(t) is written as a linear combination of the basis S₃. So we need to compute the change of basis matrix $P_{B}^{S_{3}}$ . We will use this matrix to compute $q (t)]_{B} = P_{B}^{S_{3}} [q (t)]_{S_{3}}$ since we know that [q(t)]_S₃ = [1 0 1 0]^⊤.

We will compute $P_{B}^{S_{3}}$ in two steps. We first compute $P_{S_{3}}^{B}$ :

Now, we compute $P_{B}^{S_{3}} = {(P_{S_{3}}^{B})}^{- 1}$ :

$P_{B}^{S_{3}} = [\begin{array}{rrrr} 1 0 0 0 \\ 1 1 / 3 0 0 \\ 1 2 / 3 1 / 3 0 \\ 1 1 1 1 \end{array}] .$

Since $[q (t)]_{B} = P_{B}^{S_{3}} [q (t)]_{S_{3}} = {[\begin{array}{llll} 1 1 \frac{4}{3} 2 \end{array}]}^{⊤}$ we have $q (t) = p_{0} (t) + p_{1} (t) + \frac{4}{3} p_{2} (t) + 2 p_{3} (t)$ .