AdaDelta (Single Update Step)

AdaDelta removes the learning rate altogether. It keeps two decaying averages — one of squared gradients, one of squared updates — and uses their ratio to set the step size, so the units of the update finally match the units of the parameter.

1accum_grad   = rho * accum_grad + (1 - rho) * grad²
2
3delta        = -sqrt(accum_update + epsilon) / sqrt(accum_grad + epsilon) * grad
4
5accum_update = rho * accum_update + (1 - rho) * delta²
6param        = param + delta

Task: write adadelta_step(params, grads, accum_grad, accum_update, rho, epsilon) returning [new_params, new_accum_grad, new_accum_update], rounded to 4 decimal places.

• Strict order: update accum_grad, compute delta from it, then update accum_update from that delta.
• delta is already negative — add it to the parameter.
• Both epsilons sit inside their square roots.

Notice there's no learning_rate parameter at all. The numerator is a running estimate of how large previous steps were, so the algorithm calibrates its own step size from its own history — which also means the very first step is tiny, since accum_update starts at zero.