How Models Learn

You are standing on a hillside in thick fog, and you want to reach the bottom of the valley. You cannot see the valley. You can only feel the ground under your feet. So you do the sensible thing: feel which way the ground slopes down, take a step that way, and repeat. When the ground feels flat, you are at the bottom.

That is how most modern models learn, neural networks included. The "height" is how wrong the model is. The "position" is the values of its parameters. The step-downhill rule is called gradient descent.

#A model with one dial

In What Is Machine Learning the model had one parameter, and we found the best value by trying every candidate. That stops working as soon as a parameter can be any number, or there are thousands of them. You need a method that improves the parameters a little at a time.

Here is the running example. A courier app wants to estimate trip time from distance with the model:

minutes = w × km

The parameter w is "minutes per km". The app has two past trips: 1 km took 3 minutes, and 3 km took 9 minutes. You can see the answer is w = 3. The point is to watch a method find it without being told, because the same method works when nobody can see the answer.

#Step 1: measure how wrong the model is

For each example, the error is the prediction minus the real answer. With w = 0, the model predicts 0 minutes for both trips, so the errors are 0 − 3 = −3 and 0 − 9 = −9.

To boil all the errors down to one number, square each one and take the average. That number is the loss, and this version is called the mean squared error:

loss = ((−3)² + (−9)²) / 2 = (9 + 81) / 2 = 45

Squaring does two jobs. It makes every error positive, so a too-high guess and a too-low guess cannot cancel out. And it punishes big errors much more than small ones: an error of 9 costs 81, nine times as much as an error of 3. Lower loss means a better model. Training means finding the w with the lowest loss.

#Step 2: find which way is downhill

If you plot the loss for every possible w, you get a valley. The bottom is at w = 3, where the loss is 0.

The gradient is the slope of that curve at your current w: how much the loss changes when you nudge w up a tiny bit. If the slope is negative, increasing w lowers the loss. If it is positive, decreasing w does.

You can work the slope out with school algebra. Take one example with input x, answer y and error e = w·x − y. Nudge w up by a tiny amount h. The prediction grows by h·x, so the new error is e + h·x, and its square is:

(e + h·x)² = e² + 2·e·x·h + (h·x)²

When h is tiny, the last term is so small it does not matter. So the squared error grew by about 2·e·x for every unit of h. That is the slope for one example. For the whole loss, average it over all examples:

gradient = average of 2 × error × input

#Step 3: take a step, and repeat

Move w a little in the downhill direction, which is the opposite sign of the gradient:

w = w − learning rate × gradient

The learning rate sets the step size. Here it is 0.05. Step through the first few rounds.

w = 0, rate = 0.05repeat:  errors = w * km - minutes  loss = average of errors squared  gradient = average of 2 * error * km  w = w - rate * gradient

Notice that the steps shrink by themselves. Near the bottom the valley is flatter, the gradient is smaller, and so is the step. You never have to tell the method to slow down.

#The code

// Average squared error of the model minutes = w * km.
function loss(w, km, minutes) {
  let total = 0;
  for (let i = 0; i < km.length; i++) {
    const error = w * km[i] - minutes[i];
    total += error * error;
  }
  return total / km.length;
}
 
// Slope of the loss at w: average of 2 * error * input.
function gradient(w, km, minutes) {
  let total = 0;
  for (let i = 0; i < km.length; i++) {
    const error = w * km[i] - minutes[i];
    total += 2 * error * km[i];
  }
  return total / km.length;
}
 
let w = 0;
for (let step = 1; step <= 5; step++) {
  w = w - 0.05 * gradient(w, [1, 3], [3, 9]);
}
// w is 2.90625, loss(w, [1, 3], [3, 9]) is about 0.044

def loss(w, km, minutes):
    """Average squared error of the model minutes = w * km."""
    errors = [w * x - y for x, y in zip(km, minutes)]
    return sum(e * e for e in errors) / len(km)
 
def gradient(w, km, minutes):
    """Slope of the loss at w: average of 2 * error * input."""
    total = 0
    for x, y in zip(km, minutes):
        error = w * x - y
        total += 2 * error * x
    return total / len(km)
 
w = 0
for step in range(5):
    w = w - 0.05 * gradient(w, [1, 3], [3, 9])
# w is 2.90625, loss(w, [1, 3], [3, 9]) is about 0.044

#From one dial to millions

A real model has many parameters, not one. The idea does not change. For each parameter, you ask "if I nudge just this one, how does the loss change?" That list of slopes, one per parameter, is the gradient. Then every parameter takes a small step downhill at once.

Two practical changes appear at scale. First, computing the loss over millions of examples for every step is too slow, so each step uses a small random batch of examples instead. The steps are noisier but far cheaper. Second, the loss surface has many dimensions and many dips, so gradient descent finds a good low point, not a guaranteed best one. That is usually good enough.

#Common mistakes

#Try it yourself

#Check yourself

#What to practise next

Our model predicted a number. Regression and Classification adds a second parameter for a better line, then turns the same idea into a model that picks a category, like spam or not spam.