Return to the course home page here.

Why the hell is matrix multiplication so weird? And as we’ve seen already—transformations of the plane, row operation through elementary matrices, representing linear systems—why is it so got-darn effective?

One might look at the list of matrix multiplication’s successes, throw up their hands, say multiplication is defined *ingeniously*, and make no attempt to question the situation further. In a good math class, though (and I’m not claiming to be teaching one of those), investigation is the name of the game, so further down the rabbit hole we go.

*Note well. In the interest of time, these notes form the bare bones of my lecture content. You are expected to come to class to see examples and light proofs.*

### An introductory example: Three-dimensional space

You are already familiar with one example of a vector space, (three-dimensional) Euclidean space . Recall that it is the set of all triples whose components , , and are real numbers. Remember,

Don’t worry so much about the transpose: so that matrix multiplication works out correctly, it is standard to write vectors in and later, as columns. However, it is much easier to type row vectors than column vectors, so now that you know about transpose, I am transposing all of them.

Remember that you learned

and if λ is a real number, then

These are the rules for addition and scalar multiplication from calculus, but they line up exactly with the rules for matrix addition and scalar multiplication. You also learned that, letting

(called **i**, **j**, and **k** in calculus), through adding and scaling any vector may be written as a **linear combination **of these. For example,

.

*In general, a linear combination *

*vectors is a sum of scalar multiples of those vectors.*

*A set of vectors is linearly independent if none can be written as a linear combination of the others.*

In , for and to be linearly independent means that there are no and that make *any *of the equations

true. Or, combining all the equations together, we say that there are no *nonzero* real numbers and such that

When , we say the linear combination is **trivial**. Another way to express linear independence is that no nontrivial combination of the vectors makes zero.

How can we show this? Well, it’s a linear system! What else have we been doing all month? The equation

reduces to

which is true if and only if . The facts that , and generate every vector in through linear combination and are themselves linearly independent, in other words they each represent a different piece of information, mean that they form a **basis **for .

Let’s slow up there, because this is important. *Every *vector in is made up of some of , some of , and some of . Particularly, there is no redundancy: you could not get the part out of the other two, for example. The basis of a vector space, in this case but we’ll be talking about all vector spaces later, tells you all and only what you need to know about the space.

How? Well, let’s consider what the matrix

If we let be any vector in , then

In this way, left-multiplication by gives a function

like the ones we recall from vector calculus. There’s something special about this function, though. Since matrix multiplication distributes over addition and scalars pass through matrices, if **x **and **y **are 3-vectors and and are scalars then

The function is a **linear function**, meaning it passes through addition of vectors and through scalar multiplication.

Finally, we get to see why bases are so handy. For *any *3-vector ,

In other words, the fact that is linear means that we can know everything there is to know about from what it does the basis. If only there were an easy way to know what does to the basis…

So, is just the first column of ! Likewise,

With all this machinery built, it becomes very easy to compute :

Linear algebra is all about linear transformations between vector spaces. As this example illustrates, linear transformations can be known entirely by what they do to the vector space’s basis. Going forward, we’ll look at these ideas in their abstract form, and then by the end of the class, bring everything back to the study of matrices.

### Vector spaces

The remainder of the section will *formalize *a lot of the ideas we saw in the example. Let *V *be a set that is **closed under **addition where and scalar multiplication where . We call *V *a **vector space **if addition:

- Associates:
- Commutes:
- Has an identity, a vector
**0**with the property - Has inverses, a vector for every such that

and if scalar multiplication:

- Associates:
- Identity:
- Distributes over vectors:
- Distributes over scalars:

In other words, if addition and scalar multiplication follow the nice properties we expect from real numbers and the example 3-vectors.

It’s important not to confuse *consequences *of the vector space definition with the definition itself. For example, it is not in the definition that or that , but both of these facts have “stupid” proofs that follow from the rules. Mathematics is a game in which you must make do with exactly what you have and nothing more, but often you find that you can build what you need.

** e.g. **Examples of vector spaces included , the set

of all real -column vectors, the set of all polynomials with real coefficients and degree at most , the set of all continuous real-valued functions , and the set of all matrices with real entries. Perhaps more usefully, non-examples include the set of all invertible matrices (the zero matrix is missing) and the set of all polynomials of even degree (it is possible to subtract them to get a polynomial of odd degree, so the set is not closed).

As with , a **linear combination **of a set of vectors is a sum of scalar multiples of those vectors,

The same set of vectors is **linearly independent **if it is impossible to write one as a linear combination of the others, or equivalently if the linear system

only holds when all , and **linearly dependent **otherwise.

If every vector in can be written as a linear combination of the vectors in , then we say **spans **or is a **spanning set **for . If additionally is linearly independent, or morally speaking has no redundant elements, we say forms a **basis **for . A basis is the alphabet of a vector space: it contains all and only that information you need to build the whole space. Since there is no redundancy, each vector in a vector space can be written uniquely in the language of its basis. This turns out to be hugely important.

** e.g. **We saw earlier that the set is a basis for .

All bases for —there are many, and which one you choose to use depends on which one makes easiest the problem you want to solve—have the same number of elements. This number is called the **dimension** of , denoted .

* e.g. *Since a basis for contains three elements, unsurprisingly is three-dimensional.

We say that is a **subspace **of if itself is a vector space. It turns out that one only needs to verify that , and that is closed under addition and scalar multiplication. Though it is not always the case that a basis for can be cut down into a basis for , it *is *true that a basis for can always be expanded into a basis for . This tells us that if is a subspace of such that , then .

* e.g. *Pick a nonzero vector . Then any line

is a subspace of . Likewise, for any two nonzero independent vectors the plane

is a subspace of .

These are the basic notions of vector spaces, but to really get a feel for them one must do examples (the textbook and your class notes contain many). This includes painstakingly verifying that so-and-so* *is a vector space and such-and-such is true given the rules of addition and scalar multiplication—this is how mathematics is done.

Next, we’ll relate vector spaces to one another using linear transformations.