Linear Algebra (Langara MATH 2362) in University Mathematics

There is a moment in first-year linear algebra where the course quietly changes the question, and students who miss the change spend the rest of the term confused.

Early on, a system of equations has an answer, and you find it. Then one day the answer is a plane, or a line, or a description rather than a point — and the exam is no longer asking what x is. It is asking what the set of all solutions looks like. That is a different kind of question, and no amount of extra row-reducing prepares you for it.

This guide is about that half of the course: rank, solution sets, and the projections that turn out to be what regression really is. If it is the geometry of matrices that is not landing — determinants, eigenvectors — we cover that in our guide to linear algebra fundamentals, and it is worth reading first.

Rank counts how much information a matrix really carries

The rank of a matrix is the number of genuinely independent rows it has — how many of them are telling you something new.

Two rows. But the second is just twice the first — it contains no information the first did not already give you. Written as equations, "x + 2y + 3z = 0" and "2x + 4y + 6z = 0" are the same constraint said twice, at different volumes. So the rank is 1, not 2.

This matters because rank is what actually determines whether a system is solvable and how many solutions it has. Counting equations tells you nothing: you can write down a hundred equations that are all the same equation, and you have still only constrained one thing.

Rank-nullity: the course's tidiest idea

The null space is everything the matrix sends to zero — the directions it destroys. Its dimension is called the nullity, and it is tied to the rank by an equation that is exactly as neat as it looks:

where n is the number of columns — the dimension you started in. And it says something almost obvious once you hear it plainly: every dimension you feed in either survives or dies. The rank counts the survivors, the nullity counts the casualties, and together they must account for all of them. Nothing goes missing.

For A above: three columns, rank 1, so the nullity is 2. And indeed the null space is the set of $(x,y,z)$ with $x+2y+3z=0$ — a plane through the origin, which is 2-dimensional. Two whole dimensions get crushed to a point, and one survives.

Once you have that, a family of exam questions collapses into arithmetic. "Can this transformation be one-to-one?" Only if nothing dies — nullity 0, so rank must equal n. "Can it be onto?" Only if the survivors fill the target space. Students memorise those as separate conditions; they are one sentence with the numbers moved around.

What a solution set actually looks like

Here is where the change of question happens, and it has a shape worth carrying.

Solve Ax = b and you do not get a point — you get a point plus everything the matrix ignores:

One particular solution ${\mathbf{x}}_p$ that works, plus anything in the null space ${\mathbf{x}}_h$, because adding something the matrix sends to zero cannot change the output. Add nothing to b, and you land in the same place.

So the geometry of the answer is decided entirely by the null space. Nullity 0 means exactly one solution. Nullity 1 means a line of them. Nullity 2 means a plane. There is never "a few" — it is one, none, or infinitely many, and the null space tells you which before you have finished solving.

This is also why "free variables" in a row-reduced matrix are not an accident of the method. Each free variable is a dimension the matrix could not see, and it is a basis vector for the null space. The algorithm was showing you the null space all along.

Projection: finding the closest thing to an answer

Now the part that makes the course worth taking, and it starts with a problem that sounds like failure.

What do you do when Ax = b has no solution at all? In school you would write "no solution" and stop. In practice — and this is nearly always the practical case, because real data does not sit exactly on a line — you find the closest thing to a solution instead.

First, what the dot product is really for

The projection formula looks arbitrary until you know what the dot product measures, and most students only ever learn it as "multiply the matching components and add".

It measures alignment. That is the whole meaning:

Pointing the same way gives the biggest value. Perpendicular gives exactly zero — which is why "dot product is zero" and "at right angles" are the same statement, and why you can test perpendicularity without ever finding an angle. Pointing opposite ways gives the most negative value.

Check it on our vectors: $\mathbf{a}\cdot \mathbf{b}=15$, and both have length 5, so $\cos \theta =15/25=0.6$, giving $\theta =53.13^{\circ }$. And independently, (3,4) sits at $\arctan (4/3)=53.13^{\circ }$ from the x-axis, where b lies. The same angle, two ways.

So when the projection formula divides by a·a, it is dividing by a's length squared — normalising, so the answer does not change if you write a as (6, 8) instead. It is not an arbitrary denominator; it is the formula refusing to care how long you drew the arrow.

Now project b onto a:

With $\mathbf{a}=(3,4)$ and $\mathbf{b}=(5,0)$: $\mathbf{a}\cdot \mathbf{b}=15$, $\mathbf{a}\cdot \mathbf{a}=25$, so the projection is $0.6(3,4)=(1.8,2.4)$.

The residual — what is left over — is $(5,0)-(1.8,2.4)=(3.2,-2.4)$. And now check the one thing that matters:

Perpendicular. That right angle is not a decorative feature of the diagram — it is the entire reason the projection is the closest point. If the error were not perpendicular, it would have some component along the line, and you could slide along the line and do better. Being unable to improve is exactly what "perpendicular error" means, and it is why the ugly-looking formula is the right one.

Least squares is a projection, and so is every regression you will ever run

Now scale that idea up, and a whole applied field falls out of it.

Fit a straight line through (1, 1), (2, 2) and (3, 2). No line passes through all three — the points are not collinear, and the line through the first and last predicts 1.5 at x = 2 where the actual value is 2. As a system of equations, this has no solution.

So do not solve it. Project instead — find the line whose errors are perpendicular to everything you could adjust:

The best line is $y=0.5x+0.667$, with residuals $-0.167,+0.333,-0.167$. Two checks confirm it. First, those residuals sum to exactly zero — the signature of a least-squares fit with an intercept, and a free error check on any regression you ever do. Second, nudge the slope in either direction and the sum of squared errors rises from 0.167. It is a genuine minimum, not merely a formula that produced a number.

And here is the thing worth taking away from the whole course: that is linear regression. The formula in your statistics textbook, the trendline in a spreadsheet, the fit in every lab report, the machine-learning model — all of it is this projection, done in more dimensions. Linear algebra did not borrow the idea from statistics. Statistics is standing on it.

Where the marks actually go

• Counting equations instead of computing rank. Ten copies of one equation constrain one thing.
• Giving a single solution when the answer is a set. If there are free variables, the answer has a parameter in it, and a bare point is not the answer.
• Treating the null space as an abstraction. It is the reason the solution set has the shape it has, and it is where the free variables live.
• Memorising the projection formula without the right angle. The formula is forgettable; "the error must be perpendicular or you could do better" is not, and it reconstructs the formula.
• Answering "how many solutions?" by solving. Rank and nullity answer it faster and more reliably than the arithmetic does.

How to study this half

• For every computation, say what shape the answer is. Point, line, plane. If you cannot, you have not answered the question that will be asked.
• Do everything in 2D or 3D first, where you can draw it. Every idea here is visible with two or three arrows, and the theorem you are examined on is the same one.
• Connect it to something real. Least squares is the bridge — once you see that regression is a projection, the abstraction stops feeling arbitrary and starts feeling like the point.
• Check the residual sum on any fit. Zero, or you have made an arithmetic error.

Getting help with linear algebra

If you can row-reduce faultlessly and still lose marks on "describe the solution set", that is a specific problem with a specific fix, and it is the most common one in this course. Our one-on-one linear algebra help works from your actual assignments and past midterms.

Sessions run in person in Burnaby or online across Metro Vancouver, and online suits this material particularly well — the working stays on screen to re-read. Book a free 30-minute consultation and bring a recent problem set.