Answer key · Unit 9

Machine learning.

Each lesson’s practice questions, their correct answers and the mistakes they spot. Answers read off the Scene use the same setting the learner sees.

Misconception catalogue

Answer key 4

Lesson 1 of 4 · About 25 minutes

Learning a line

Learning from data in its simplest form: a model with two parameters, the best-fit line as what the computer learns, a prediction for a new point, the error, one outlier, and data with nothing to learn.

Open the lesson

The 5 steps and their answers

Step 1 of 5
Open this step
A model
Practice · Predict, then look
Question
The twenty measurements were made by a rule in which $y$ ⁠ rises by $0.6$ ⁠ for each $1$ ⁠ of $x$ ⁠, plus noise. A computer will find their best-fit line, the line that fits them best. What will its slope be?

Correct answer
- Correct Close to $0.6$ ⁠, but not exactly
Setup, choices and feedback
The Scene at this step
Slope
0.6fixed
Noise
0.8fixed
Outlier
offfixed
Measurements / Best-fit line
Measurementsfixed
Choices
- Correct Close to $0.6$ ⁠, but not exactly
- Wrong Exactly $0.6$ ⁠
  Misconception: Thinks a model learned from noisy data recovers the true rule exactly.
  Feedback from the catalogue
  A model learns only from the examples it has, and the examples are noisy. Twenty measurements made by the rule $y = 2 + 0.6x$ ⁠ plus noise give a best-fit line that is close, $\hat y \approx 1.90 + 0.62x$ ⁠, but not the same. The more data and the less noise, the closer the model comes to the rule; it hits it exactly only with no noise at all.
- Wrong Far from $0.6$ ⁠, because there is noise
  Feedback
  The noise largely cancels between the points, so the line comes out close to the rule. It would be far only with very few points or a great deal of noise.
Once the prediction is locked in, the Scene changes
Measurements / Best-fit line
Best-fit linefixed
The Scene then shows: 0.6205 · B, the least-squares slope Sxy / Sxx/nodes/scatter/outputs/slope
A prediction is never marked as a failure: the learner sees the outcome and the explanation either way.
Feedback for a right answer
$\hat y \approx 1.90 + 0.62x$ ⁠: the computer has learned from the data a rule very close to the true one, $y = 2 + 0.6x$ ⁠, without anyone telling it.

Worked solution
1. The best-fit line is the line with the smallest sum of squared vertical distances to the points.
2. The noise moves each point a little above or below the rule, and over twenty points the moves mostly cancel.
3. So the slope is close to $0.6$ ⁠, but this sample’s noise tilts it slightly: $0.62$ ⁠.
Step 2 of 5
Open this step
Predicting
Practice · Calculate
Question
The board shows the learned line, $\hat y = 1.90 + 0.62x$ ⁠. What does it predict for $y$ ⁠ in a new measurement at $x = 10$ ⁠?

Correct answer
8.1 ± 0.05
Setup, choices and feedback
The Scene at this step
Slope
0.6fixed
Noise
0.8fixed
Outlier
offfixed
Measurements / Best-fit line
Best-fit line
Accepted within
± 0.05
A fixed value in the question
Wrong answers it recognises
- 6.2 ± 0.05
  Misconception: Predicts with the slope alone, leaving out the intercept.
  Feedback from the catalogue
  A line is $\hat y = a + bx$ ⁠: the slope times $x$ ⁠, plus the intercept. $0.62 \times 10 = 6.2$ ⁠ is only the change from $x = 0$ ⁠; the line starts at a height of $1.90$ ⁠, so it predicts $1.90 + 6.2 = 8.1$ ⁠.
Feedback for a right answer
$1.90 + 0.62 \times 10 = 8.1$ ⁠. That is what learning buys: an answer to a question that was not in the data, since there is no measurement at $x = 10$ ⁠.

Hint
Put $x = 10$ ⁠ into the line’s formula.

Worked solution
1. The line: $\hat y = 1.90 + 0.62x$ ⁠.
2. $0.62 \times 10 = 6.2$ ⁠.
3. $1.90 + 6.2 = 8.1$ ⁠.
If you got $6.2$ ⁠, you forgot the intercept.
Step 3 of 5
Open this step
The error
Practice · Reach the target
Question
The model’s error is the spread of the points about the line. Change Noise until the error is below $0.5$ ⁠, and press Check.

Goal
- 0.25 ± 0.25
  At the start of the step: 0.7984
Setup, choices and feedback
The Scene at this step
Slope
0.6fixed
Noise
0.8
Outlier
offfixed
Measurements / Best-fit line
Best-fit line
Read from the Scene at
The residuals’ spread about the line, √(SSres / n)/nodes/scatter/outputs/lineSpread
One setting that reaches it
Noise
0
Feedback for a right answer
The line’s error nearly equals the noise in the data: the line learns the rule, and what is left is the noise, which cannot be predicted. Cleaner data gives sharper predictions.

Feedback for a wrong answer
Not yet. The more noise, the further the points lie from the line.

Worked solution
1. The spread about the line is $\sqrt{\dfrac{\sum (y - \hat y)^2}{n}}$ ⁠.
2. With the line close to the rule, what is left is the noise, so the spread nearly equals the Noise.
3. Set Noise to $0.4$ ⁠ or less.
Step 4 of 5
Open this step
Bad data
Practice · Predict, then look
Question
We will add one bad measurement to the data, far below the rule, near the right-hand end. What will happen to the slope of the learned line?

Correct answer
- Correct It will drop clearly: the line tilts towards the point
Setup, choices and feedback
The Scene at this step
Slope
0.6fixed
Noise
0.8fixed
Outlier
offfixed
Measurements / Best-fit line
Best-fit line
Choices
- Correct It will drop clearly: the line tilts towards the point
- Wrong Hardly anything: one point out of twenty-one
  Misconception: Thinks a single outlier among many points cannot shift a fitted model.
  Feedback from the catalogue
  Least squares penalises an error squared: a point five times as far from the line adds twenty-five times as much to the sum. So a single outlier pulls the line towards it hard, here dropping the slope from $0.62$ ⁠ to $0.47$ ⁠. Models that learn from data are sensitive to bad data, which is why data is checked and cleaned before training.
Once the prediction is locked in, the Scene changes
Outlier
onfixed
The Scene then shows: 0.4675 · B, the least-squares slope Sxy / Sxx/nodes/scatter/outputs/slope
A prediction is never marked as a failure: the learner sees the outcome and the explanation either way.
Feedback for a right answer
The slope drops from $0.62$ ⁠ to $0.47$ ⁠, and the line’s error rises from $0.80$ ⁠ to $1.28$ ⁠. The square of its large distance dominates the sum.

Worked solution
1. The line minimises the sum of the squares of the distances.
2. The outlier is $5$ ⁠ units off the rule, so it adds about $25$ ⁠, more than all the other points together.
3. To shrink it the line moves towards it, and since it sits at the end, the line tilts: the slope drops.
Step 5 of 5
Open this step
Nothing to learn
Practice · Predict, then look
Question
Now $x$ ⁠ and $y$ ⁠ are unrelated: the rule’s slope is $0$ ⁠, and the points are scattered about the mean with a spread of $0.8$ ⁠. Will the line the computer learns predict $y$ ⁠ much better than guessing the mean?

Correct answer
- Correct No: its error will be nearly $0.8$ ⁠
Setup, choices and feedback
The Scene at this step
Slope
0fixed
Noise
0.8fixed
Outlier
offfixed
Measurements / Best-fit line
Mean and spreadfixed
Choices
- Correct No: its error will be nearly $0.8$ ⁠
- Wrong Yes: the computer always finds a line that fits
  Misconception: Thinks fitting a model always gives good predictions, even when the data has no pattern.
  Feedback from the catalogue
  A model can learn only a pattern that is in the data. When $x$ ⁠ and $y$ ⁠ are unrelated, the best-fit line comes out nearly flat, at the height of the mean, and its error is nearly the spread about the mean: it predicts no better than guessing the mean. An algorithm always returns an answer, but that does not make it useful.
Once the prediction is locked in, the Scene changes
Measurements / Best-fit line
Best-fit linefixed
The Scene then shows: 0.7984 · The residuals’ spread about the line, √(SSres / n)/nodes/scatter/outputs/lineSpread
A prediction is never marked as a failure: the learner sees the outcome and the explanation either way.
Feedback for a right answer
The line is nearly flat, $\hat y \approx 4.90 + 0.02x$ ⁠, and its error is $0.798$ ⁠, almost exactly the spread about the mean. $r \approx 0.06$ ⁠: there is nothing to learn.

Worked solution
1. Guessing the mean is off, on average, by the spread, $0.8$ ⁠.
2. When $x$ ⁠ and $y$ ⁠ are unrelated, the best line is nearly the mean itself.
3. So its error is nearly $0.8$ ⁠: it predicts no better.

Lesson 2 of 4 · About 20 minutes

Neurons and weights

What an artificial neuron computes: a weighted sum plus a bias, a sigmoid that turns it into a value between 0 and 1, a network of layers, and how many weights and parameters it holds.

Open the lesson

The 5 steps and their answers

Step 1 of 5
Open this step
A neuron
Practice · Calculate
Question
A neuron has two inputs, $x_1 = 0.5$ ⁠ and $x_2 = -1$ ⁠, with weights $w_1 = 2$ ⁠ and $w_2 = 0.5$ ⁠, and its bias is $b = 0.3$ ⁠. What is the weighted sum $z = w_1 x_1 + w_2 x_2 + b$ ⁠?

Correct answer
0.8 ± 0.01
Setup, choices and feedback
The Scene at this step
Training steps
0fixed
Data
Corners (XOR)fixed
Network / Decision boundary
Networkfixed
Accepted within
± 0.01
A fixed value in the question
Wrong answers it recognises
- 0.5 ± 0.01
  Misconception: Leaves the bias out of a neuron's weighted sum, or out of a count of its parameters.
  Feedback from the catalogue
  A neuron computes $z = w_1 x_1 + w_2 x_2 + b$ ⁠: the inputs times the weights, plus the bias $b$ ⁠. The bias is like a line’s intercept: it moves the threshold at which the neuron “fires”. It is learned in training too, so it counts among the network’s parameters.
- 1.8 ± 0.01
  Misconception: Treats a negative input or weight as positive in a weighted sum.
  Feedback from the catalogue
  In an input times a weight the signs matter: a negative input times a positive weight makes a negative contribution, which lowers the sum. $0.5 \times (-1) = -0.5$ ⁠, not $+0.5$ ⁠. A negative weight means “this input argues against”: the bigger it is, the less active the neuron.
Feedback for a right answer
$2 \times 0.5 + 0.5 \times (-1) + 0.3 = 1 - 0.5 + 0.3 = 0.8$ ⁠.

Hint
Multiply each input by its weight, add, and add the bias.

Worked solution
1. $w_1 x_1 = 2 \times 0.5 = 1$ ⁠.
2. $w_2 x_2 = 0.5 \times (-1) = -0.5$ ⁠.
3. $z = 1 - 0.5 + 0.3 = 0.8$ ⁠.
Without the bias you get $0.5$ ⁠, and with $+0.5$ ⁠ instead of $-0.5$ ⁠, $1.8$ ⁠.
Step 2 of 5
Open this step
Activation
Practice · Choose
Question
The neuron passes $z = 0.8$ ⁠ through the sigmoid, $\sigma(z) = \dfrac{1}{1 + e^{-z}}$ ⁠, which turns any number into a value between $0$ ⁠ and $1$ ⁠. What comes out?

Correct answer
- Correct About $0.69$ ⁠: above a half, since $z$ ⁠ is positive
Setup, choices and feedback
The Scene at this step
Training steps
0fixed
Data
Corners (XOR)fixed
Network / Decision boundary
Networkfixed
Choices
Shown in a random order.
- Correct About $0.69$ ⁠: above a half, since $z$ ⁠ is positive
- Wrong $0.8$ ⁠: the sigmoid leaves numbers between $0$ ⁠ and $1$ ⁠ alone
  Feedback
  The sigmoid bends every number, those between $0$ ⁠ and $1$ ⁠ too: $\sigma(0) = 0.5$ ⁠, and $\sigma(0.8) \approx 0.69$ ⁠.
- Wrong About $0.31$ ⁠
  Feedback
  $0.31$ ⁠ is $\sigma(-0.8)$ ⁠. For a positive $z$ ⁠ the sigmoid gives more than a half.
Feedback for a right answer
$\sigma(0.8) = \dfrac{1}{1 + e^{-0.8}} \approx \dfrac{1}{1.45} \approx 0.69$ ⁠. That is how the output neuron in the Scene’s network works: above $0.5$ ⁠ it says “blue”, below, “orange”.

Worked solution
1. $e^{-0.8} \approx 0.45$ ⁠.
2. $1 + 0.45 = 1.45$ ⁠.
3. $\dfrac{1}{1.45} \approx 0.69$ ⁠.
Step 3 of 5
Open this step
Layers
Practice · Calculate
Question
The Scene’s network has $2$ ⁠ input neurons, two hidden layers of $5$ ⁠, and one output neuron, and each neuron is joined to every neuron in the next layer. Each link has one weight. How many weights does the network have?

Correct answer
40 ± 0.01
Setup, choices and feedback
The Scene at this step
Training steps
400
Data
Corners (XOR)
Network / Decision boundary
Networkfixed
Accepted within
± 0.01
Computed from the Scene at this step’s setting
Neural: links, each with one weight (2×5 + 5×5 + 5×1)/nodes/net/outputs/weights
Wrong answers it recognises
- 13 ± 0.01
  Misconception: Counts the neurons when asked for the weights, which belong to the links between them.
  Feedback from the catalogue
  A weight belongs to a link between two neurons, not to a neuron. Each neuron in a layer is joined to every neuron in the next, so the weights between two layers are the product of their sizes. In a 2–5–5–1 network: $2 \times 5 + 5 \times 5 + 5 \times 1 = 40$ ⁠ weights, across only $13$ ⁠ neurons.
- 51 ± 0.01
  Misconception: Leaves the bias out of a neuron's weighted sum, or out of a count of its parameters.
  Feedback from the catalogue
  A neuron computes $z = w_1 x_1 + w_2 x_2 + b$ ⁠: the inputs times the weights, plus the bias $b$ ⁠. The bias is like a line’s intercept: it moves the threshold at which the neuron “fires”. It is learned in training too, so it counts among the network’s parameters.
Feedback for a right answer
$2 \times 5 + 5 \times 5 + 5 \times 1 = 40$ ⁠ weights. Language-model networks have billions.

Hint
Between two layers the number of links is the product of their sizes.

Worked solution
1. Inputs to the first hidden layer: $2 \times 5 = 10$ ⁠.
2. Between the two hidden layers: $5 \times 5 = 25$ ⁠.
3. To the output: $5 \times 1 = 5$ ⁠.
4. $10 + 25 + 5 = 40$ ⁠.
$13$ ⁠ is the number of neurons, and $51$ ⁠ counts the $11$ ⁠ biases too.
Step 4 of 5
Open this step
Parameters
Practice · Calculate
Question
Every neuron except the inputs also has a bias. How many parameters, weights and biases together, does training have to tune?

Correct answer
51 ± 0.01
Setup, choices and feedback
The Scene at this step
Training steps
400
Data
Corners (XOR)
Network / Decision boundary
Networkfixed
Accepted within
± 0.01
A fixed value in the question
Wrong answers it recognises
- 40 ± 0.01
  Misconception: Leaves the bias out of a neuron's weighted sum, or out of a count of its parameters.
  Feedback from the catalogue
  A neuron computes $z = w_1 x_1 + w_2 x_2 + b$ ⁠: the inputs times the weights, plus the bias $b$ ⁠. The bias is like a line’s intercept: it moves the threshold at which the neuron “fires”. It is learned in training too, so it counts among the network’s parameters.
- 53 ± 0.01
  Misconception: Counts the neurons when asked for the weights, which belong to the links between them.
  Feedback from the catalogue
  A weight belongs to a link between two neurons, not to a neuron. Each neuron in a layer is joined to every neuron in the next, so the weights between two layers are the product of their sizes. In a 2–5–5–1 network: $2 \times 5 + 5 \times 5 + 5 \times 1 = 40$ ⁠ weights, across only $13$ ⁠ neurons.
Feedback for a right answer
$40$ ⁠ weights plus $5 + 5 + 1 = 11$ ⁠ biases: $51$ ⁠ parameters. “A seventy-billion-parameter model” means a count like this, only far larger.

Worked solution
1. Weights: $40$ ⁠.
2. Biases: one per neuron in the hidden layers and the output, $5 + 5 + 1 = 11$ ⁠.
3. $40 + 11 = 51$ ⁠.
$53$ ⁠ gives a bias to the two input neurons too, which only pass the data on.
Step 5 of 5
Open this step
Reading the links
Practice · Choose
Question
In the drawing, each link’s thickness is the size of its weight, and its colour the sign. What does a thick orange link into a neuron say?

Correct answer
- Correct A large negative weight: this input strongly holds the neuron back
Setup, choices and feedback
The Scene at this step
Training steps
400
Data
Corners (XOR)
Network / Decision boundary
Network
Choices
Shown in a random order.
- Correct A large negative weight: this input strongly holds the neuron back
- Wrong A large positive weight: this input strongly drives the neuron
  Feedback
  Positive links are blue. Orange is negative.
- Wrong A lot of information is flowing through it right now
  Feedback
  The thickness is the weight, which stays fixed until the next training step. The flowing beads are the signal of the point now passing through.
Feedback for a right answer
A positive input times a large negative weight takes a lot off the sum $z$ ⁠, so it holds the neuron back. Negative weights are knowledge too: “when this happens, it is not that”.

Worked solution
1. Thick: the weight’s size is large.
2. Orange: the weight is negative.
3. A contribution $w \cdot x$ ⁠ with a large negative $w$ ⁠ lowers $z$ ⁠ a lot.

Lesson 3 of 4 · About 25 minutes

Training a network

How a network learns: a loss that training lowers, gradient descent step by step, XOR that one neuron cannot learn, hard tasks that take longer, and whether the network will be right on new data too.

Open the lesson

The 6 steps and their answers

Step 1 of 6
Open this step
Random weights
Practice · Predict, then look
Question
The network is untrained, with random weights: it is right on about half of the XOR corners. How many of the points will it classify correctly after 400 training steps?

Correct answer
- Correct Nearly all of them
Setup, choices and feedback
The Scene at this step
Training steps
0fixed
Data
Corners (XOR)fixed
Network / Decision boundary
Networkfixed
Choices
- Correct Nearly all of them
- Wrong Still about half: a network cannot learn XOR
  Misconception: Thinks no neural network can learn XOR, since a single neuron cannot.
  Feedback from the catalogue
  A single neuron separates two groups with a straight line, and the XOR corners cannot be split by one straight line: that was the famous criticism of the perceptron in 1969. But a network with hidden layers combines several lines into a bending boundary, and learns XOR easily. The hidden layers are what give a network its power.
Once the prediction is locked in, the Scene changes
Training steps
400fixed
Network / Decision boundary
Decision boundaryfixed
The Scene then shows: 100 · Neural: the share of the 80 data points the network classifies right, percent (as the board prints it, rounded)/nodes/net/outputs/accuracy
A prediction is never marked as a failure: the learner sees the outcome and the explanation either way.
Feedback for a right answer
100%. Look inside shows the decision boundary: two bending lines splitting the corners. One neuron alone could not do it; the hidden layers can.

Worked solution
1. A single neuron splits with a straight line, and the XOR corners cannot be split by one.
2. In a hidden layer each neuron learns a line of its own, and the next layer combines them.
3. So the network learns a bending boundary and gets every point right.
Step 2 of 6
Open this step
Training steps
Practice · Reach the target
Question
Train the network on Two clusters: raise Training steps until it is right on all 80 points, and press Check.

Goal
- 100 ± 0.01
  At the start of the step: 5
Setup, choices and feedback
The Scene at this step
Training steps
0
Data
Two clustersfixed
Network / Decision boundary
Decision boundary
Read from the Scene at
Neural: the share of the 80 data points the network classifies right, percent (as the board prints it, rounded)/nodes/net/outputs/accuracy
One setting that reaches it
Training steps
17
Feedback for a right answer
Two clusters can be split by a straight line, so the network learns them fast, in under 20 steps. The loss keeps falling even after that: the network grows more confident in answers that are already right.

Feedback for a wrong answer
Not yet. Keep training.

Worked solution
1. Each training step moves every weight a little in the direction that lowers the loss.
2. The clusters can be split by a straight line, an easy task.
3. About 20 steps are enough for 100%; any larger number works too.
Step 3 of 6
Open this step
What changes
Practice · Choose
Question
Between one training step and the next, what changes in the network?

Correct answer
- Correct The weights and biases
Setup, choices and feedback
The Scene at this step
Training steps
60
Data
Two clusters
Network / Decision boundary
Decision boundary
Choices
Shown in a random order.
- Correct The weights and biases
- Wrong The points it learns from
  Misconception: Thinks training changes the data or the shape of the network, rather than the weights.
  Feedback from the catalogue
  The data and the network’s structure stay fixed throughout training: the same 80 points, the same 13 neurons. What changes at each step are the weights and biases: each moves a little in the direction that reduces the error. In the Scene you see it in the links, which change thickness and colour as training goes on.
- Wrong The number of neurons and links
  Misconception: Thinks training changes the data or the shape of the network, rather than the weights.
  Feedback from the catalogue
  The data and the network’s structure stay fixed throughout training: the same 80 points, the same 13 neurons. What changes at each step are the weights and biases: each moves a little in the direction that reduces the error. In the Scene you see it in the links, which change thickness and colour as training goes on.
Feedback for a right answer
Gradient descent: for each weight, work out how the loss would change if it moved a little, and move it a small step in the direction that lowers the loss. All 51 parameters move together, at every step.

Worked solution
1. The data is fixed: the same 80 points.
2. The structure is fixed: 13 neurons and 40 links.
3. What training tunes are the parameters: the weights and biases.
Step 4 of 6
Open this step
The loss
Practice · Predict, then look
Question
The network starts learning XOR, and at step 0 it is right on 55% of the points. What will happen to that share after 20 training steps?

Correct answer
- Correct It can drop for a while, even as the network learns
Setup, choices and feedback
The Scene at this step
Training steps
0fixed
Data
Corners (XOR)fixed
Network / Decision boundary
Decision boundaryfixed
Choices
- Correct It can drop for a while, even as the network learns
- Wrong It will rise: every step improves
  Misconception: Expects every training step to improve the share of answers the network gets right.
  Feedback from the catalogue
  Training does not try directly to raise the share of right answers; it lowers the loss, a smooth measure of how confident the network is in wrong answers. The loss falls at almost every step, but the share right can drop for a while, as the network rearranges its weights. On XOR it drops from 55% to 41% before climbing to 100%.
Once the prediction is locked in, the Scene changes
Training steps
20fixed
The Scene then shows: 41 · Neural: the share of the 80 data points the network classifies right, percent (as the board prints it, rounded)/nodes/net/outputs/accuracy
A prediction is never marked as a failure: the learner sees the outcome and the explanation either way.
Feedback for a right answer
It drops to 41%, while the loss falls from $0.717$ ⁠ to $0.689$ ⁠. Training improves the loss, and the share right only updates when points cross the $0.5$ ⁠ threshold. By 150 steps the network is at 95%.

Worked solution
1. Training lowers the loss, not the number of mistakes directly.
2. At first the weights move to a new region, and some points cross to the wrong side of the threshold.
3. The loss went down, and the share right dipped for a while.
Step 5 of 6
Open this step
A harder shape
Practice · Reach the target
Question
Now the data is a Ring: blue points round a disc of orange ones. Raise Training steps until the network is right on at least 90% of the points, and press Check.

Goal
- 95 ± 5
  At the start of the step: 30
Setup, choices and feedback
The Scene at this step
Training steps
0
Data
Ringfixed
Network / Decision boundary
Decision boundary
Read from the Scene at
Neural: the share of the 80 data points the network classifies right, percent (as the board prints it, rounded)/nodes/net/outputs/accuracy
One setting that reaches it
Training steps
197
Feedback for a right answer
The ring is harder than the clusters: the boundary must close round the centre, so the network needs about 200 steps. A harder task takes more training.

Feedback for a wrong answer
Not yet. Keep training; it takes longer here.

Worked solution
1. It needs a closed boundary round the centre, which several straight lines combined can make.
2. It takes the network a while to tune the five neurons in each layer for that.
3. About 200 steps give 91%, and 400 give 100%.
Step 6 of 6
Open this step
New data
Practice · Choose
Question
The trained network is right on all 80 training points. Will it also be right on new points of the same kind?

Correct answer
- Correct Not necessarily: it must be tested on points it has not seen
Setup, choices and feedback
The Scene at this step
Training steps
400
Data
Ring
Network / Decision boundary
Decision boundary
Choices
Shown in a random order.
- Correct Not necessarily: it must be tested on points it has not seen
- Wrong Yes: it is already 100% right
  Misconception: Thinks a network that is right on all its training data will be right on new data too.
  Feedback from the catalogue
  The network has been tested only on the points it trained on, which it could simply memorise. The question that matters is whether it generalises: whether it is right on new points as well. A network that is too big or trained too long can learn the noise in its data and fail on new data: that is overfitting. So some data is always kept aside, unseen, to test on.
Feedback for a right answer
Here the boundary is smooth and the data simple, so it will probably generalise well, but that is checked, not assumed. Real systems keep part of the data as test data, and measure on that alone.

Worked solution
1. On the training points the network might simply memorise.
2. The score that matters is on new data: how well it generalises.
3. So it is tested on data that took no part in training.

Lesson 4 of 4 · About 25 minutes

Attention in a language model

How a language model reads and writes: tokens as numbers, an attention head choosing what to look at, copying a made-up word, and a wheel of next-token probabilities that the temperature sharpens or flattens.

Open the lesson

The 6 steps and their answers

Step 1 of 6
Open this step
Tokens
Practice · Choose
Question
What does a language model receive when it “reads” the sentence on the rack?

Correct answer
- Correct A sequence of tokens, each a number in the vocabulary, turned into a vector
Setup, choices and feedback
The Scene at this step
Sentence
The keys…
Attending token
8
Temperature
1
Tokens / Next token
Tokensfixed
Choices
Shown in a random order.
- Correct A sequence of tokens, each a number in the vocabulary, turned into a vector
- Wrong The letters, one after another
  Misconception: Thinks a language model reads letters or whole words, rather than tokens turned into numbers.
  Feedback from the catalogue
  A language model does not see letters. Text is cut into tokens, whole words or parts of words, and each token has a number in the vocabulary, as each character has a number in ASCII. Each number becomes a vector, a list of numbers, and the model computes on those. A new word, like zorblat, is cut into familiar pieces, zor and blat.
- Wrong The meaning of each word, from a dictionary
  Misconception: Thinks a language model reads letters or whole words, rather than tokens turned into numbers.
  Feedback from the catalogue
  A language model does not see letters. Text is cut into tokens, whole words or parts of words, and each token has a number in the vocabulary, as each character has a number in ASCII. Each number becomes a vector, a list of numbers, and the model computes on those. A new word, like zorblat, is cut into familiar pieces, zor and blat.
Feedback for a right answer
Each tile is a token, with its number in the vocabulary, and behind it a column of lit cells, its vector. From there on it is all arithmetic on numbers, as in any neural network.

Worked solution
1. The text is cut into tokens: words or parts of words.
2. Each token has a fixed number in the vocabulary.
3. Each number becomes a vector, which the network computes on.
Step 2 of 6
Open this step
Attention
Practice · Predict, then look
Question
After “The keys to the cabinet in the hall” the model has to choose between is and are. Which earlier token will hall give the most attention to?

Correct answer
- Correct keys
Setup, choices and feedback
The Scene at this step
Sentence
The keys…fixed
Attending token
8fixed
Temperature
1
Tokens / Next token
Tokensfixed
Choices
- Correct keys
- Wrong the, the token just before it
  Misconception: Thinks a token attends mostly to the word just before it.
  Feedback from the catalogue
  Attention is not measured in distance. Each token weighs all the tokens before it, and gives the most attention to whichever helps it most to predict what comes next, however far back. After “The keys to the cabinet in the hall”, to choose between is and are you need to know the subject, so hall looks at keys, six words back.
- Wrong cabinet, the nearest noun
  Misconception: Thinks a token attends mostly to the word just before it.
  Feedback from the catalogue
  Attention is not measured in distance. Each token weighs all the tokens before it, and gives the most attention to whichever helps it most to predict what comes next, however far back. After “The keys to the cabinet in the hall”, to choose between is and are you need to know the subject, so hall looks at keys, six words back.
Once the prediction is locked in, the Scene changes
Tokens / Next token
Attentionfixed
The Scene then shows: 2 · The position (1–8) of the token it attends to most/nodes/rack/outputs/lookedAt
A prediction is never marked as a failure: the learner sees the outcome and the explanation either way.
Feedback for a right answer
62% of the attention goes to keys, token 2. The subject of the sentence is the keys, so the verb will be are. The thickest arc leaps over six words.

Worked solution
1. To decide between singular and plural, you need the subject of the sentence.
2. The subject is keys; cabinet and hall only describe where they are.
3. So a head that has learned grammar looks at keys, however far back.
Step 3 of 6
Open this step
Sharp attention
Practice · Reach the target
Question
Find a Sentence and an Attending token where the token gives about 80% of its attention to a single token, and press Check.

Goal
- 0.8 ± 0.05
  At the start of the step: 0.62
Setup, choices and feedback
The Scene at this step
Sentence
The keys…
Attending token
8
Temperature
1fixed
Tokens / Next token
Attentionfixed
Read from the Scene at
The largest share of attention the attending token gives one token, 0..1/nodes/rack/outputs/topWeight
One setting that reaches it
Sentence
My pet zorblat…
Attending token
8
Feedback for a right answer
In “My pet zorblat sleeps. The zor” the second zor gives 80% of its attention to blat, the token that followed the first zor. That is how the model knows to finish a word made up a moment before.

Feedback for a wrong answer
Not yet. In most places the attention is spread over several tokens.

Hint
Look for a word that appears twice.

Worked solution
1. Attention as sharp as that is a sign of copying: the token is looking for something specific.
2. In the zorblat sentence the token zor appears twice.
3. Choose the sentence My pet zorblat… and the last attending token, 8.
Step 4 of 6
Open this step
Copying
Practice · Choose
Question
The word zorblat is made up, and no model met it in training. How does the model know that blat comes after the second zor?

Correct answer
- Correct An attention head finds the earlier zor and copies what came after it
Setup, choices and feedback
The Scene at this step
Sentence
My pet zorblat…
Attending token
8
Temperature
1
Tokens / Next token
Attention
Choices
Shown in a random order.
- Correct An attention head finds the earlier zor and copies what came after it
- Wrong It saw the word somewhere on the web in training
  Misconception: Thinks the model can only continue a made-up word by having met it in training.
  Feedback from the catalogue
  No model met zorblat in training. But one attention head learned a general pattern: “find the last time the current token appeared, and look at what came after it”. When zor comes round again, attention jumps to the blat after the first zor, and the model copies it. Heads like this are called induction heads, and they are part of what lets models learn from the text in front of them.
- Wrong It looks at the token just before it, The
  Misconception: Thinks a token attends mostly to the word just before it.
  Feedback from the catalogue
  Attention is not measured in distance. Each token weighs all the tokens before it, and gives the most attention to whichever helps it most to predict what comes next, however far back. After “The keys to the cabinet in the hall”, to choose between is and are you need to know the subject, so hall looks at keys, six words back.
Feedback for a right answer
That is an induction head: in training it learned a general rule, “what followed this last time will follow again”, not the word itself. That is how a model learns from the text in front of it.

Worked solution
1. The word was not in training, so the answer cannot come from memory.
2. It is in the sentence itself: zor followed by blat.
3. Attention jumps to blat, and the model copies it.
Step 5 of 6
Open this step
The next token
Practice · Predict, then look
Question
After “The Eiffel Tower is in the city of”, Paris has an 88% chance at temperature 1. What will happen to that chance when Temperature goes down to 0.2?

Correct answer
- Correct It will rise, nearly to 100%
Setup, choices and feedback
The Scene at this step
Sentence
The Eiffel Tower…fixed
Attending token
8fixed
Temperature
1fixed
Tokens / Next token
Next tokenfixed
Choices
- Correct It will rise, nearly to 100%
- Wrong It will fall: the model will be less sure
  Misconception: Thinks a lower temperature makes the model's choice more random.
  Feedback from the catalogue
  Temperature stretches or flattens the probabilities: each probability $p$ ⁠ becomes $p^{1/T}$ ⁠, then they are rescaled to add to 1. At a low temperature the power is large, and the leading candidate takes nearly all the probability: the model is predictable and cautious. At a high temperature the differences wash out, and the wheel stops on rare tokens too: more creative, and wilder.
Once the prediction is locked in, the Scene changes
Temperature
0.2fixed
The Scene then shows: 1 · The probability of the likeliest next token at this temperature, 0..1/nodes/rack/outputs/pTop
A prediction is never marked as a failure: the learner sees the outcome and the explanation either way.
Feedback for a right answer
At temperature 0.2 Paris has nearly 100%, and the wheel always stops on it. A low temperature suits factual answers; a high one suits stories.

Worked solution
1. At temperature $T$ ⁠ each probability $p$ ⁠ becomes $p^{1/T}$ ⁠, then they are rescaled.
2. At $T = 0.2$ ⁠ the power is $5$ ⁠: $0.88^5 \approx 0.53$ ⁠, but $0.03^5 \approx 0.00000002$ ⁠.
3. After rescaling, Paris takes nearly everything.
Step 6 of 6
Open this step
Temperature
Practice · Reach the target
Question
After “…in the hall” the leading candidate, are, gets 55%. Change Temperature until it gets about 70%, and press Check.

Goal
- 0.7 ± 0.03
  At the start of the step: 0.55
Setup, choices and feedback
The Scene at this step
Sentence
The keys…fixed
Attending token
8fixed
Temperature
1
Tokens / Next token
Next tokenfixed
Read from the Scene at
The probability of the likeliest next token at this temperature, 0..1/nodes/rack/outputs/pTop
One setting that reaches it
Temperature
0.75
Feedback for a right answer
At about 0.75–0.8. A small drop in temperature sharpens the distribution: strong candidates grow stronger, and the long tail of rare tokens all but vanishes.

Feedback for a wrong answer
Not yet. A lower temperature raises the leader’s chance; a higher one lowers it.

Hint
For the leader to get more, the model must be more cautious.

Worked solution
1. At $T = 1$ ⁠, are has 55%; it needs more, so lower $T$ ⁠.
2. At $T = 0.5$ ⁠ it is already at 85%, too much.
3. In between, at $T \approx 0.75$ ⁠, it is at about 71%.

Across the unit · 15 misconceptions

Misconception catalogue

Each common mistake this unit recognises, in a few words. Open one for what the learner reads when an answer matches it, and where it is asked.

Index

Learning a line

A model

Question

Correct answer

The Scene at this step

Choices

Once the prediction is locked in, the Scene changes

Worked solution

Predicting

Question

Correct answer

The Scene at this step

Wrong answers it recognises

Hint

Worked solution

The error

Question

Goal

The Scene at this step

One setting that reaches it

Worked solution

Bad data

Question

Correct answer

The Scene at this step

Choices

Once the prediction is locked in, the Scene changes

Worked solution

Nothing to learn

Question

Correct answer

The Scene at this step

Choices

Once the prediction is locked in, the Scene changes

Worked solution

Neurons and weights

A neuron

Question

Correct answer

The Scene at this step

Wrong answers it recognises

Hint

Worked solution

Activation

Question

Correct answer

The Scene at this step

Choices

Worked solution

Layers

Question

Correct answer

The Scene at this step

Wrong answers it recognises

Hint

Worked solution

Parameters

Question

Correct answer

The Scene at this step

Wrong answers it recognises

Worked solution

Reading the links

Question

Correct answer

The Scene at this step

Choices

Worked solution

Training a network

Random weights

Question

Correct answer

The Scene at this step

Choices

Once the prediction is locked in, the Scene changes

Worked solution

Training steps

Question

Goal

The Scene at this step