Why the hell is matrix multiplication so weird? And as we’ve seen already—transformations of the plane, row operation through elementary matrices, representing linear systems—why is it so got-darn effective?
One might look at the list of matrix multiplication’s successes, throw up their hands, say multiplication is defined ingeniously, and make no attempt to question the situation further. In a good math class, though (and I’m not claiming to be teaching one of those), investigation is the name of the game, so further down the rabbit hole we go.
Note well. In the interest of time, these notes form the bare bones of my lecture content. You are expected to come to class to see examples and light proofs.
An introductory example: Three-dimensional space
You are already familiar with one example of a vector space, (three-dimensional) Euclidean space . Recall that it is the set of all triples whose components , , and are real numbers. Remember,
Don’t worry so much about the transpose: so that matrix multiplication works out correctly, it is standard to write vectors in and later, as columns. However, it is much easier to type row vectors than column vectors, so now that you know about transpose, I am transposing all of them.
Remember that you learned
and if λ is a real number, then
These are the rules for addition and scalar multiplication from calculus, but they line up exactly with the rules for matrix addition and scalar multiplication. You also learned that, letting
(called i, j, and k in calculus), through adding and scaling any vector may be written as a linear combination of these. For example,
In general, a linear combination vectors is a sum of scalar multiples of those vectors.
A set of vectors is linearly independent if none can be written as a linear combination of the others.
In , for and to be linearly independent means that there are no and that make any of the equations
true. Or, combining all the equations together, we say that there are no nonzero real numbers and such that
When , we say the linear combination is trivial. Another way to express linear independence is that no nontrivial combination of the vectors makes zero.
How can we show this? Well, it’s a linear system! What else have we been doing all month? The equation
which is true if and only if . The facts that , and generate every vector in through linear combination and are themselves linearly independent, in other words they each represent a different piece of information, mean that they form a basis for .
Let’s slow up there, because this is important. Every vector in is made up of some of , some of , and some of . Particularly, there is no redundancy: you could not get the part out of the other two, for example. The basis of a vector space, in this case but we’ll be talking about all vector spaces later, tells you all and only what you need to know about the space.
How? Well, let’s consider what the matrix
If we let be any vector in , then
In this way, left-multiplication by gives a function
like the ones we recall from vector calculus. There’s something special about this function, though. Since matrix multiplication distributes over addition and scalars pass through matrices, if x and y are 3-vectors and and are scalars then
The function is a linear function, meaning it passes through addition of vectors and through scalar multiplication.
Finally, we get to see why bases are so handy. For any 3-vector ,
In other words, the fact that is linear means that we can know everything there is to know about from what it does the basis. If only there were an easy way to know what does to the basis…
So, is just the first column of ! Likewise,
With all this machinery built, it becomes very easy to compute :
Linear algebra is all about linear transformations between vector spaces. As this example illustrates, linear transformations can be known entirely by what they do to the vector space’s basis. Going forward, we’ll look at these ideas in their abstract form, and then by the end of the class, bring everything back to the study of matrices.
The remainder of the section will formalize a lot of the ideas we saw in the example. Let V be a set that is closed under addition where and scalar multiplication where . We call V a vector space if addition:
- Has an identity, a vector 0 with the property
- Has inverses, a vector for every such that
and if scalar multiplication:
- Distributes over vectors:
- Distributes over scalars:
In other words, if addition and scalar multiplication follow the nice properties we expect from real numbers and the example 3-vectors.
It’s important not to confuse consequences of the vector space definition with the definition itself. For example, it is not in the definition that or that , but both of these facts have “stupid” proofs that follow from the rules. Mathematics is a game in which you must make do with exactly what you have and nothing more, but often you find that you can build what you need.
e.g. Examples of vector spaces included , the set
of all real -column vectors, the set of all polynomials with real coefficients and degree at most , the set of all continuous real-valued functions , and the set of all matrices with real entries. Perhaps more usefully, non-examples include the set of all invertible matrices (the zero matrix is missing) and the set of all polynomials of even degree (it is possible to subtract them to get a polynomial of odd degree, so the set is not closed).
As with , a linear combination of a set of vectors is a sum of scalar multiples of those vectors,
The same set of vectors is linearly independent if it is impossible to write one as a linear combination of the others, or equivalently if the linear system
only holds when all , and linearly dependent otherwise.
If every vector in can be written as a linear combination of the vectors in , then we say spans or is a spanning set for . If additionally is linearly independent, or morally speaking has no redundant elements, we say forms a basis for . A basis is the alphabet of a vector space: it contains all and only that information you need to build the whole space. Since there is no redundancy, each vector in a vector space can be written uniquely in the language of its basis. This turns out to be hugely important.
e.g. We saw earlier that the set is a basis for .
All bases for —there are many, and which one you choose to use depends on which one makes easiest the problem you want to solve—have the same number of elements. This number is called the dimension of , denoted .
e.g. Since a basis for contains three elements, unsurprisingly is three-dimensional.
We say that is a subspace of if itself is a vector space. It turns out that one only needs to verify that , and that is closed under addition and scalar multiplication. Though it is not always the case that a basis for can be cut down into a basis for , it is true that a basis for can always be expanded into a basis for . This tells us that if is a subspace of such that , then .
e.g. Pick a nonzero vector . Then any line
is a subspace of . Likewise, for any two nonzero independent vectors the plane
is a subspace of .
These are the basic notions of vector spaces, but to really get a feel for them one must do examples (the textbook and your class notes contain many). This includes painstakingly verifying that so-and-so is a vector space and such-and-such is true given the rules of addition and scalar multiplication—this is how mathematics is done.
Next, we’ll relate vector spaces to one another using linear transformations.