A Review of Activation Function for Artificial Neural Network

Survey of common and recent activation functions used in neural networks and their role in training.

Published

January 26, 2020

Doi

10.1109/SAMI48414.2020.9108717

Keywords

Machine Learning Engineer, Python, PyTorch, LLM, MLOps, Applied ML, Model Evaluation, Probability Calibration, Affective Computing, NLP

Published in: SAMI 2020
DOI: 10.1109/SAMI48414.2020.9108717
Link: https://ieeexplore.ieee.org/abstract/document/9108717

The activation function plays an important role in the training and the performance of an Artificial Neural Network. They provide the necessary non-linear properties to any Artificial Neural Network. In this work, we provide a review of the most common and recent activation functions used in the hidden layer of an Artificial Neural Network.