Building a Bidirectional Encoder Representations From Transformers (BERT) Transfer Learning Framework for Glioma Overall Survival Prediction Using Unstructured Electronic Health Record Notes
H Lin1*, J Barrios Ginart1, H Gong2, Y Interian2, W Chen1, R Luo2, T Upadhaya1, J Lupo1, S Braunstein1, O Morin1, (1) University of California San Francisco, San Francisco, CA, (2) University Of San Francisco, CA
Presentations
SU-F-206-2 (Sunday, 7/10/2022) 2:00 PM - 3:00 PM [Eastern Time (GMT-4)]
Room 206
Purpose: To build a Bidirectional Encoder Representations from Transformers (BERT) transfer learning framework that directly uses unstructured medical notes to predict overall survival (OS) of glioma patients.
Methods: The framework can capture long-range dependencies in numerous and lengthy Electronic Health Record (EHR) notes, and consists of two main components: a BERT model, pre-trained on large-scale clinical notes (Medical Information Mart for Intensive Care III), and a logistic regression (LR) model to fit OS prediction using 14-months as a binary endpoint. For each patient, EHR notes from a single institution were aggregated from the diagnosis date to the initial timepoint of outcome evaluation, and new notes were incorporated progressively for up to one year. Training and testing data include EHR notes of 378 and 97 patients. F1 and the Area Under the Receiver Operating Characteristic (AUROC) were used to compare BERT against an LR model based on WHO tumor grades and a co-occurrence Natural Language Processing model, Term frequency–inverse document frequency (Tf-idf). The data were stratified based on tumor grades and 21 rounds of experiments were run at each timepoint to evaluate the model robustness.
Results: Compared with grade-based LR and Tf-idf, the BERT framework achieved the best performance with F1 scores of 0.72 to 0.81 and AUROC of 0.62 to 0.84 on the hold-out testing set across all timepoints with smaller variations, outperforming Tf-idf and grade-based models in F1 score by 5-18% and 15-33% respectively. We also demonstrated the ability of BERT to update and improve a prognostic OS model over time as an individual’s illness course unfolds.
Conclusion: BERT with the direct use of unstructured EHR notes improved the performance on OS prediction in glioma patients. The framework developed in this study is not limited to glioma and the methodology can be adapted to other cancers and treatment paradigms.
Keywords
Not Applicable / None Entered.
Taxonomy
Not Applicable / None Entered.
Contact Email