Computations, optimization and tuning of deep feedforward neural networks

Abstract

This article presents an overview of the generalized formulations of the computations, optimization, and tuning of a deep feedforward neural network. A small network has been used to systematically explain the computing steps, which were then used to establish the generalized forms of the computations in forward and backward propagations for larger networks. Additionally, some of the commonly used cost functions, activation functions, optimization algorithms, and hyper-parameters tuning approaches have been discussed.