Integrating Dropout Regularization Technique at Different Layers to Improve the Performance of Neural Networks

In many facial expression recognition models it is necessary to prevent overfitting to check no units (neurons) depend on each other. Therefore, dropout regularization can be applied to ignore few nodes randomly while processing the remaining neurons. Hence, dropout helps dealing with overfitting and predicts the desired results with more accuracy at different layers of the neural network like ‘visible’, ‘hidden’ and ‘convolutional’ layers. In neural networks there are layers like dense, fully connected, convolutional and recurrent (LSTM- long short term memory). It is possible to embed the dropout layer with any of these layers. Model drops the units randomly from the neural network, meaning model removes its connection from other units. Many researchers found dropout regularization a most powerful technique in machine learning and deep learning. Dropping few units (neurons) randomly and processing the remaining units can be considered in two phases like forward and backward pass (stages). Once the model drops few units randomly and select ‘n’ from the remaining units it is obvious that weight of the units could change during processing. It must be noted that updated weight doesn’t reflect on the dropped units. Dropping and stepping-in few units seem to be very good process as those units which step-in will represent the network. It is assumed to have maximum chance for the stepped-in units to have less dependency and model gives better results with higher accuracy.

@article{Pansambal2023,
  title     = {Integrating Dropout Regularization Technique at Different Layers to Improve the Performance of Neural Networks},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {14},
  number    = {4},
  year      = {2023},
  publisher = {The Science and Information Organization},
  author    = {B. H. Pansambal and A. B. Nandgaokar},
  doi       = {10.14569/IJACSA.2023.0140478},
  url       = {https://doi.org/10.14569/IJACSA.2023.0140478}
}