Penalized Entropy Loss Function with High Quality Uncertainty Estimation for Pneumonia Diagnosis in Pediatric Patients
D Feng1, X Chen1*, K Wang2, Z Zhou3, (1) Xi'an Jiaotong University, Xi'an, 61, CN, (2) UT Southwestern Medical Center, Dallas, TX, (3) University of Central Missouri, Warrensburg, MISSOURI
Presentations
PO-GePV-M-259 (Sunday, 7/25/2021) [Eastern Time (GMT-4)]
Purpose: In the uncertainty-informed decision system, models may make wrong predictions with high certainty which leads to omission of false predictions. Therefore, we propose to set up a model which could make correct prediction with low uncertainty and wrong prediction with high uncertainty.
Methods: From accuracy and certainty perspective, we categorized the predictions into four kinds, correct and certain (CC), correct and uncertain (CU), wrong and certain (WC), wrong and uncertain (WU). The desirable model is that almost all predictions belong to CC or WU and fewer WC cases. Therefore, we integrated uncertainty estimation into training phase and proposed penalized entropy (PE) loss. PE has two terms, accuracy related and certainty related terms. Accuracy related term is cross entropy. Certainty related term is penalizing WC predictions and is the multiplication of negative likelihood (NLL) and certainty, where certainty is calculated by Monte Carlo dropout. High NLL and high certainty will result in high certainty-related loss which means wrong and certain decision. A hyper-parameter is used for relative weighting of certainty-related term with respect to accuracy-related term.
Results: Validating on the Chest X-ray dataset, we achieved higher quality of uncertainty estimation that accuracy vs. uncertainty (AvU) is 0.89 which outperformed state-of-the-art losses. Furthermore, PE can produce more accurate predictions with accuracy of 0.90 and AUC of 0.96.
Conclusion: In this study, we proposed a penalized entropy loss function in a prediction model for pneumonia diagnosis in pediatric patients. This model achieves better uncertainty estimation that makes correct predictions with high certainty, wrong predictions with high uncertainty, especially avoiding wrong but certain results. Compared to state-of-the-art loss functions, this loss can not only give higher quality uncertainty estimation but also more accurate results. Therefore, model trained with this novel loss function can be used in clinical for precise and comprehensive diagnosis.
ePosters
Keywords
Not Applicable / None Entered.
Taxonomy
Not Applicable / None Entered.
Contact Email