A network has two parameters. sits in a busy part of the network and its gradient has held steady at for many steps. is the weight on a rarely-seen feature, and its gradient has held steady at .
Both are trained with RMSProp:
with , , and (ignorable). Because each gradient has been constant for so long, each parameter's has settled to a constant.
Plain gradient descent would move by per step and by per step — a hundredfold gap. Once has settled, how big is each parameter's RMSProp update?
Select all that apply.