Deep Learning-Guided Iterative Refinement to Improve Label Quality and Consistency
H Zhou1*, J Xiao2, Z Fan2, 3, 4, D Ruan1, 5, (1) Department of Bioengineering, University of California, Los Angeles, CA 90095, (2) Department of Radiology, University of Southern California, Los Angeles, CA 90033, (3) Department of Radiation Oncology, University of Southern California, Los Angeles, CA 90033, (4) Department of Biomedical Engineering, University of Southern California, Los Angeles, CA 90089 (5) Department of Radiation Oncology, University of California, Los Angeles, CA 90095
Presentations
MO-B-BRC-3 (Monday, 7/11/2022) 8:30 AM - 9:30 AM [Eastern Time (GMT-4)]
Ballroom C
Purpose: Data quality is essential in modeling construction. In the biomedical field, manual labels are typically used as the ground truth for model training and assessment. On the other hand, human labels could be noisy or uncertain. The definition of truth is hardly absolute from a biomedical perspective, allowing room for small perturbations within the clinical acceptance range. In this study, a hybrid data-model instead of the typical model-centric approach is taken to improve the manual label quality used for supervised learning.
Methods: We hypothesize that perturbing labels to better conform to parsimonious modeling within biomedical feasibility regime could enhance both model capability and label quality. We take intracranial vessel segmentation as a use case to demonstrate the general idea and illustrate the proposed iterative procedure with a convergence criterion for an end-to-end presentation. In this specific example, a lightweight 2.5D segmentation network is used as the low-dimensional model, and we alternate between fitting the network to human labels and guiding the human observer to review the labels to identify ill-fitted examples and perform modification when appropriate. Iteration stops when the fitting quality converges by a defined equivalence criterion.
Results: We observe enhanced modeling performance and better conformality to clinical insight with the final labels. In this use case, Dice similarity coefficient improved from 0.893±0.108 to 0.938±0.078 for lumen, from 0.806±0.086 to 0.879±0.072 for vessel wall, the total variation in normalized wall index decreased from 0.757±0.181 to 0.586±0.182, and the input-output agreement index increased from 0.523 to 0.556 by the proposed refinement procedure, in a cross-validation assessment.
Conclusion: The proposed refinement produces manual labels of better quality, which is shown by the significantly reduced inter-sample inconsistency, better compliance with the low-dimensional network model, and more physiologically sound quantitative indices. The rationale generalizes to other label settings and tasks.
Funding Support, Disclosures, and Conflict of Interest: This work is supported in part by NIH/NHLBI R01 HL147355.
Keywords
Blood Vessels, Image Guidance, Segmentation
Taxonomy
IM/TH- Image Analysis Skills (broad expertise across imaging modalities): Machine Learning
Contact Email