What Is Machine Learning

You run a small delivery service, and customers keep asking whether their parcel will be late. You could sit down and write rules. Late if the distance is over some number of kilometres. Unless it is a weekday morning. Unless the driver is new. Every week you find a case your rules got wrong, and every fix makes the rules harder to read.

You already have something better than your guesses: a spreadsheet of past deliveries, each with its distance and whether it actually arrived late. Machine learning means letting the program read that spreadsheet and work out the rule itself.

#Rules versus examples

In ordinary programming, you write the rule and the computer applies it. In machine learning, you give the computer examples together with the right answers, and it finds a rule that fits them. You then use that rule on new cases it has never seen.

The rule it finds is called a model. A model is just a function: it takes the facts about one case and returns a guess. What makes it "learned" is that some numbers inside it were chosen by looking at the examples, not typed in by you.

#The words you will keep seeing

Every lesson in this module uses the same handful of terms. Here they are on the delivery example.

#Watch a rule being learned

Start with the simplest possible model: "a delivery is late if its distance is at least t km". The model has one parameter, t. Training means picking the best value for it.

Here are seven past deliveries, sorted by distance. Notice the 10 km one: it arrived on time even though longer trips are usually late. Real data has cases like that. Maybe the roads were empty that day.

The plan is blunt: try each distance as the cut-off, count how many past deliveries that rule gets wrong, and keep the cut-off with the fewest mistakes.

for each candidate t:  mistakes = 0  for each delivery:    guess = late if distance >= t    if guess != actual: mistakes += 1  keep t if it has the fewest mistakes so far

A new order comes in for 7 km. The model says late, because 7 is at least 6. That is a prediction: an answer for a case that was not in the training data.

The model still gets the 10 km delivery wrong, and that is fine. No simple rule explains every quirk in real data. A model that matches every single example has often memorized noise, which is the trap covered in Overfitting and Evaluation.

#The code

function learnThreshold(distances, late) {
  let bestT = null;
  let fewest = Infinity;
  for (const t of distances) {         // try each distance as the cut-off
    let mistakes = 0;
    for (let i = 0; i < distances.length; i++) {
      const guess = distances[i] >= t; // the rule: late if at least t km
      if (guess !== late[i]) mistakes++;
    }
    if (mistakes < fewest) {
      fewest = mistakes;
      bestT = t;
    }
  }
  return { threshold: bestT, mistakes: fewest };
}
 
const distances = [2, 3, 5, 6, 8, 10, 12];
const late = [false, false, false, true, true, false, true];
learnThreshold(distances, late); // { threshold: 6, mistakes: 1 }

def learn_threshold(distances, late):
    best_t = None
    fewest = float("inf")
    for t in distances:                # try each distance as the cut-off
        mistakes = 0
        for d, was_late in zip(distances, late):
            guess = d >= t             # the rule: late if at least t km
            if guess != was_late:
                mistakes += 1
        if mistakes < fewest:
            fewest = mistakes
            best_t = t
    return best_t, fewest
 
distances = [2, 3, 5, 6, 8, 10, 12]
late = [False, False, False, True, True, False, True]
learn_threshold(distances, late)  # (6, 1)

Trying every option works here because there is one parameter and seven examples. Real models have thousands to billions of parameters, and you cannot try every combination. They need a smarter way to search, which is the subject of How Models Learn.

#Three kinds of learning

• Supervised learning. Every example has a label, like our late or on-time column. Spam filters, price estimates and photo tagging work this way. Most of this module is about it.
• Unsupervised learning. There are no labels. The program looks for structure on its own, such as groups of customers who shop alike.
• Reinforcement learning. The program acts, gets a reward or penalty, and adjusts. Game-playing programs and robot control are typical uses.

#When machine learning is the wrong tool

Machine learning gives you a rule that is usually right. That is a great trade when the rule is hard to write by hand and an occasional mistake is acceptable. It is a bad trade in other cases.

• The rule is already known. Sales tax, sorting a list, checking a password. Write the code.
• You have few examples, or examples without answers. The model can only learn what the data shows.
• Every answer must be exact and explainable. "Usually right" is not good enough for a bank balance.

#Try it yourself

#Check yourself

#What to practise next

Trying every value stops working once a model has more than a few parameters. How Models Learn shows the method that replaces it: measure the error with a loss, then walk downhill with gradient descent.