Gradient descent with momentum uses
vt=βvt−1+∇L(θt),θt+1=θt−αvtv_t = \beta v_{t-1} + \nabla L(\theta_t), \qquad \theta_{t+1} = \theta_t - \alpha v_tvt=βvt−1+∇L(θt),θt+1=θt−αvt
Take α=0.1\alpha = 0.1α=0.1, β=0.9\beta = 0.9β=0.9, θ0=5\theta_0 = 5θ0=5 and v0=0v_0 = 0v0=0. The gradient comes out as ∇L=2\nabla L = 2∇L=2 at both of the first two steps.
What is θ2\theta_2θ2, the parameter value after two updates?
Select all that apply.
4.424.424.42
4.604.604.60
4.624.624.62
5.585.585.58