A very small model has a single weight . How badly it performs is measured by a loss function
and training's whole job is to make smaller. It does that by repeatedly nudging using this rule:
Here is the instantaneous rate of change of at the weight , and (the learning rate) controls how big the nudge is. For this run,
The model takes exactly one step.
By how much does the loss decrease? Report a positive number, rounded to two decimal places.