Fire, Miss, Adjust

An artificial neuron computes $z = \sum_i w_i x_i + b$. Give it a hard threshold and it behaves like the biological cell it was modelled on: it fires (outputs 1) when $z > 0$ and stays quiet (outputs 0) otherwise. A total of exactly $0$ counts as quiet.

A neuron like this can learn its own weights from labelled examples, with no gradients at all, by adjusting itself every time it gets one wrong. Visit the examples one at a time, in order:

1. Compute the neuron's output for example $x$ using the current weights and bias.
2. Compare it with the example's label $t$ (0 or 1). The error is $e = t - \text{output}$, so it is $+1$, $-1$ or $0$.
3. If $e \neq 0$ the neuron made a mistake, and every weight and the bias change straight away:

where $\eta$ is the learning rate. If $e = 0$, nothing changes.

One trip through all the examples is a pass. Training starts with every weight and the bias at $0$. It stops after the first pass in which the neuron makes no mistakes, or after passes passes, whichever comes first.

Task: write train_perceptron(examples, labels, lr, passes).

• examples is a list of equal-length lists of numbers, and labels holds a 0 or 1 for each one.
• Return a tuple (weights, bias, mistakes): the final weights as a list and the final bias, each rounded to 4 decimal places, and mistakes, a list holding the number of mistakes made in each pass that ran.

This rule only ever learns from its errors: a correct answer teaches it nothing. Backpropagation, later in the course, keeps that spirit but swaps the all-or-nothing threshold for something smooth enough to have a slope.