Transfer Functions in Angel

1. Transfer Functions in Angel

Name	Expression	Description
Sigmoid		Turn any real number into propability
tanh		Turn any real number into a number between -1 and 1
relu		Discard the negative part
dropout		Randomly set to zero
identity		Output as it is

The following figure shows some common transfer functions (some of them are not available in Angel):

2. JSON Expression of Transfer Functions

2.1 Parameter-Free Transfer Functions

json

"transfunc": "sigmoid",

"transfunc": {
    "type": "tanh"
}

2.2 Transfer Functions with Parameters

json

"transfunc": "dropout",

"transfunc": {
    "type": "dropout",
    "proba": 0.5,
    "actiontype": "train"
}

Note: since the dropout transfer function is calculated differently in training and testing (prediction), users must specify in which scenario the function is used: train, inctrain or predict.