AdaGrad (Single Update Step)

AdaGrad gives every parameter its own learning rate by accumulating the squares of all the gradients it has ever seen:

1cache = cache + grad²
2param = param - learning_rate * grad / (sqrt(cache) + epsilon)

Task: write adagrad_step(params, grads, cache, learning_rate, epsilon) returning [new_params, new_cache], all values rounded to 4 decimal places.

• Update the cache first, then use the new cache in the parameter update.
• epsilon sits outside the square root.

The accumulator only ever grows, so the effective learning rate only ever shrinks — and on a long run it eventually reaches zero and training stops making progress. That single flaw is what RMSProp exists to fix, by decaying the accumulator instead of letting it pile up forever.