Neurons and weights.

Step 1 of 5

A neuron

A line is a model of one input. When there are several inputs, such as the two numbers that place a point, you use a neuron: a small unit that multiplies each input by a weight, adds them up, and adds a bias:

$z = w_1 x_1 + w_2 x_2 + b$$z =$$w_1 x_1 + w_2 x_2 + b$⁠

A large weight means the input matters; a negative weight, that it argues against. The name comes from the nerve cells of the brain, which sum signals from other cells, but the artificial neuron is just this sum.

Step 2 of 5

Activation

After the sum, the neuron passes $z$⁠ through an activation function. The output neuron of the network in the Scene uses the sigmoid,

$\sigma(z) = \frac{1}{1 + e^{-z}}$$\sigma(z) =$$\frac{1}{1 + e^{-z}}$⁠

which turns any number into a value between $0$⁠ and $1$⁠: a large negative $z$⁠ gives nearly $0$⁠, $z = 0$⁠ gives a half, and a large positive $z$⁠ nearly $1$⁠. The hidden neurons use a similar function that gives a value between $-1$⁠ and $1$⁠.

Step 3 of 5

Layers

One neuron can only split the plane with a straight line. To learn complicated shapes, neurons are arranged in layers: the input layer, hidden layers, and an output layer, with each neuron joined to every neuron in the next layer. The network in the Scene takes the position of a point, $(x, y)$⁠, and decides whether it is blue or orange.

Step 4 of 5

Parameters

The weights and biases are the network’s parameters, like the $a$⁠ and $b$⁠ of the line. They are everything the network “knows”; the structure alone knows nothing.

Step 5 of 5

Reading the links

The network here has already been trained. Each link is drawn as thick as its weight’s size, blue when the weight is positive and orange when it is negative. A point from the data passes through the network, and the neurons light up by their activation.