Activation Function

In the Artifical Neural Networks, the activation function of the neuron defines the output of that neuron given a set of inputs from neurons on the previous layers.
Activation Function comes under the category of hyper parameters.
The weighted sum of inputs is passed to the Activation Function in the neuron. And this activation function will perform some type of operation on the input to transfer the sum to a lower limit or upper limit. Activation Functions are also used to impart a non-linearity and they are one of the important factors which affect the results and accuracy of the models.

There are a couple of activation functions. Let’s look at the few mostly used Action Functions.

A sigmoid function will transform or limit the output between 0 or 1.
Sigmoid functions will be models where we predict the probability as an output. Because the probability value exists somewhere between 0 or 1.

Equation:
f(x)f(x) = 1/(1 + e-x)

Tanh activation is similarly useful to the sigmoid function. The output ranges from -1 to 1.
The activation that works almost always better than sigmoid function is Tanh function also knows as Tangent Hyperbolic function And it is a rescaling of the logistic sigmoid.

Equation:
f(x)f(x) = 2/(1 + e-2x) — 1

RELU transforms the input to either zero (threshold) or the input itself. And it is the most widely used activation function. It is normally used to convert all the negative values to the zero in the Neural Networks.

Equation
f(x)f(x) = max(0,x)

Softmax is also the type of sigmoid function. But it will be very useful when we are dealing with the classification problems.

Related posts:

Ocupando o antigo endereço do campus da Universidade Cruzeiro do Sul, o Senac São Miguel Paulista iniciou suas atividades em janeiro deste ano, quatro anos após o começo da obra. Essa é a maior…

We have all been countlessly told that exercise is good for our health and well-being, but, in what may be a surprise to some, studies have shown that exercise is correlated to increased study times…

After putting together a list of the ten most influential athletes in Chicago sports, I thought I’d do the same thing for New York sports. This isn’t a list of the greatest New York athletes, though…