A diagram of how Adam, a popular training method, joins two ideas: momentum remembers the recent direction, and RMSProp sizes each weight's step to suit it.
From momentum
Remember the recent direction so progress builds up instead of zig-zagging.
From RMSProp
Remember the recent size so every weight gets a step that suits it.
↓
Adam
keeps both memories, and uses them together on every step
