Can I read Truncated Cross-entropy: A New Loss Function for Multi-category Classification on EtoBox?

Truncated Cross-entropy: A New Loss Function for Multi-category Classification by Yaolin Zheng; Hongbo Huang; Xiaoxu Yan; Jiayu He is a scholarly article available to read on EtoBox.

What is Truncated Cross-entropy: A New Loss Function for Multi-category Classification about?

Deep learning is one of the hottest research topics which has received a lot of attention in recent years. During the training of deep learning models, the loss function is a critical indicating objective which measures the difference between the predicted value and the distribution of the real data. It is also an important indicator for evaluating the performance of a deep learning model. The most popular loss functions used in deep learning include mean square error (MSE), cross-entropy error, etc. Obviously, the loss function has non-negligible influence to the optimizer. The most common optimizers include stochastic gradient descent method (SGD), minibatch stochastic gradient descent method (MBGD) and Adaptive moment estimation (ADAM). Among them, the MBGD is widely used for its equilibrium between accuracy and speed. However, how to set the batch size is a big challenge. If the batch size is too large, the cost of computation and memory increases accordingly. On the other hand, the gradient descent process could be more oscillated with small batch size. Therefore, this paper proposes a improved loss function named truncated cross-entropy to stabilize the convergence procedure

Author

Yaolin Zheng; Hongbo Huang; Xiaoxu Yan; Jiayu He

Publisher

ACM

Published

2022

Language

EN