Correlations of Evaluation Metrics in Deep Learning-Based Segmentation
Presentations
PO-GePV-M-12 (Sunday, 7/10/2022) [Eastern Time (GMT-4)]
ePoster Forums
Purpose: Although various forms of evaluation metrics have been explored in deep-learning based approaches towards biomedical segmentation problems, no previous study has been done to compare their performance curves during neural network training. In this work, we examine (i) the mutual relations among the learning curves of various evaluation metrics to determine which measures improve synchronously during network training in a correlated manner (ii) the correlations among their ranking scores in the context of the segmentation of OARs in lung CT via convolutional type neural networks.
Methods: We employ the cross-correlation function as a measure of the correlations among learning curves. Correlograms are used to explore and identify mutual dependencies among the metrics. The Spearman’s rank correlation coefficients are computed and, in particular, are used to identify geometrical counterparts associated with information-theoretic metrics. The neural network architecture employed is Unet and a few other variants.
Results: For each evaluation metric, strongly-correlated ones are identified through visual inspection of correlograms. Some notable
results: (i)the Dice coefficient generally exhibits a learning curve that is strongly-correlated with many other conventional metrics such as precision and sensitivity, but not with the measure of volumetric similarity which does not demonstrate clear correlation with most other metrics (ii) the mutual information metric appears to be a robust metric with scoring performances intimately related to Dice and a Tversky index that interpolates between Dice and sensitivity.
Conclusion: We initiated a study of the mutual dependencies among the learning curves of various forms of segmentation evaluation metrics by examining their cross- correlation functions. Their correlograms characterize the network’s training process and can potentially be related to a study of its capability. Our study of their Spearman’s rank correlation coefficients shows that they can be harnessed to associate geometrical intuitions with abstract metrics such as information-theoretic ones.
Keywords
Taxonomy
IM/TH- image Segmentation: General (Most aspects)
Contact Email