A calm, sequential guide for anyone entering the field, The article will also be divided into smaller chapters to make it easier to understand.
Part 1.2: The Mathematics of Learning
Part 1.1 ended at a precise standing point: we had built a model, ŷ = wx + b, and we had a single number, the mean squared error, that told us how wrong that model was on average. What we did not have was a way to act on that number. A loss of 4.67 tells you the model is wrong; it does not tell you whether to push w up or down, and it does not tell you by how much. Part 1.2 is where that missing piece arrives.
We will introduce the derivative, derive the gradient of the loss with respect to w and b by hand, and then turn those gradients into an actual update rule through gradient descent. By the end of this part, the same three data points from Chapter 2 will be used in a full training example, and you will watch the loss fall step by step until the model converges to the true underlying relationship. The slope idea from Chapter 1 does not change. It simply gets sharpened into the tool that makes learning possible.
Chapter 4: The Derivative — The Slope of the Error
Why a Number Alone Is Not Enough
At the end of Chapter 3 we had a single number, L, that summarized how wrong the model was. With w = 1 and b = 0, the loss was 4.67. We know the goal is to make this number as small as possible. What we do not yet know is how.
A reader might suggest trying different values of w and b until the loss becomes small. With two parameters and a simple dataset, this is not impossible, but consider what it involves. Should w be increased or decreased? By how much? Should b be changed at the same time or held steady? A real neural network has millions of parameters, not two, and trying random combinations would take longer than the age of the universe.
The deeper issue is that the loss value, taken on its own, does not contain a direction. The number 4.67 tells you that you are wrong, but it does not tell you which way is less wrong. If I handed you only the number 4.67 and asked you whether to increase or decrease w, you could not answer. You would have to guess. The loss is a position, not a direction. These are two fundamentally different kinds of information, and you cannot extract one from the other by rearranging the same data. You have to ask a new question.
The Equations Have Been Building Toward This
To see why the derivative is the only thing that fits here, it helps to trace the logic of every equation we have written so far. Each one was forced into existence by a problem the previous one could not solve.
We began with the slope formula:
This worked perfectly when the data lay on a clean line, because the slope could be read off directly from any two points. But real data scatters, and the slope formula stopped applying. Different pairs of points gave different slopes, and none of them represented the whole dataset.
So we wrote a candidate line with unknowns to be learned:
This solved the previous problem by giving us a form for the line, but it created a new one. The values of w and b were unknown, and we had no way to evaluate whether a particular guess was good. The equation describes the model but contains no built-in measure of its quality.
So we invented one. We compared the prediction ŷ to the truth y across all data points and summarized the gap into a single number:
This solved the evaluation problem by giving us one number that measures total error. But it created the deepest problem of all, the one we are now standing at. The loss tells us that we are wrong but not which way to change w or b to be less wrong. The information we need is not contained in L. It is contained in how L changes when w changes, which is a different quantity entirely.
To extract that quantity, we need a new equation, one that asks the loss a question the loss formula on its own never asks: if w were slightly different, by how much and in what direction would L change ? The mathematical object that answers this question is the derivative.
From Slope to Derivative
We already have the tool we need, in a simpler form. In Chapter 1, slope told us how y changes when x changes. The derivative is the same idea applied to a different setting. Instead of asking how y changes with x in the data, we now ask how the loss L changes with the parameters w and b in the model. The variables have changed, but the underlying question — how does one thing change as another changes? — is identical.
There is one refinement. The slope formula in Chapter 1 measured the rate of change between two distinct points. The derivative measures the instantaneous rate of change at a single point, which is what you get when those two points move infinitely close together:
This is not a different concept from slope. It is slope made precise. As the gap between the two points shrinks to zero, the slope of the line connecting them becomes the slope of the curve at that exact point.
Why the Derivative and Nothing Else
It is worth addressing directly why the derivative is the right tool here, rather than some other technique. The honest answer is that the derivative is not the only conceivable approach, but it is the only approach ....... .
To complete the article, you can visit the blogs and view the entire series on : https://husseinmahdi.xyz/writing/series/math-behind-neural-networks/
