A tiny model has two weights, w1 and w2, and its loss is
L(w1,w2)=(w1−3)2+2(w2+1)2
Gradient descent updates every weight at once using the rule
wnew=w−α∇L(w)
where α is the learning rate, a small positive number controlling the step size.
Starting from w=(5, 1) with α=0.1, what is wnew?