Optimizers

The optimizers.py module implements common modifications to stochastic gradient descent. It includes:

• SGD with momentum (Rummelhart, Hinton, & Williams, 1986)
• AdaGrad (Duchi, Hazan, & Singer, 2011)
• RMSProp (Tieleman & Hinton, 2012)
• Adam (Kingma & Ba, 2015)