Learning Rate Schedulers

The schedulers module implements several common strategies for learning rate decay:

• Constant
• Exponential decay
• Noam/Transformer decay (Vaswani et al., 2017)
• Davis King/Dlib decay (King, 2018)

Plots

<p align="center"> </p>