Plain momentum builds up velocity in a consistent direction, which helps roll through noise — but it also overshoots, because it commits to a step before checking what's ahead. Nesterov momentum peeks at where the velocity is about to take it and corrects for that.
Use this formulation exactly:
1velocity = momentum * velocity + grad2param = param - learning_rate * (grad + momentum * velocity)
Task: write nesterov_step(params, grads, velocity, learning_rate, momentum) returning [new_params, new_velocity], rounded to 4 decimal places.
momentum = 0 reduces this to ordinary gradient descent.The look-ahead term is the whole difference. Standard momentum steps along the accumulated velocity and then measures the gradient; Nesterov effectively measures it after the momentum step, so it starts braking before it overshoots rather than after.