Activation Functions

The activations module implements several common activation functions:

• Rectified linear units (ReLU) (Hahnloser et al., 2000)
• Leaky rectified linear units (Maas, Hannun, & Ng, 2013)
• Exponential linear units (Clevert, Unterthiner, & Hochreiter, 2016)
• Scaled exponential linear units (Klambauer, Unterthiner, & Mayr, 2017)
• Softplus units
• Hard sigmoid units
• Exponential units
• Hyperbolic tangent (tanh)
• Logistic sigmoid
• Affine

Plots

<p align="center"> </p>