A Feasibility Study of Various Data Classifiers On Automatically Identifying Free Text Journal Notes in Radiation Oncology
R Ma1*, L Yu2, P Zhang3, J Xiong4, S Liu5, Y Zhou6, S Berry7, (1) Memorial Sloan Kettering Cancer Ctr. Westchester, West Harrison, NY, (2) Metanovas Biotech Inc, ,,(3) Memorial Sloan Kettering Cancer Center, New York, NY, (4) Memoiral Sloan-Kettering Cancer Center, New York, NY, (5) Memorial Sloan Kettering Cancer Center, West Harrison, NY, (6) Memoiral Sloan-Kettering Cancer Center, New York, NY, (7) Memorial Sloan Kettering Cancer Center, New York, NY
Presentations
PO-GePV-M-334 (Sunday, 7/10/2022) [Eastern Time (GMT-4)]
ePoster Forums
Purpose: Free text entries in medical charts are a barrier to automation. We evaluated the performance of various commonly used data classifiers in categorizing the content of free text notes in an electronic radiotherapy chart’s treatment journal as part of an initiative to automate physics weekly chart checks.
Methods: A total of 2756 journal notes were labeled into two groups – notes indicating “setup check” has been performed (positive) versus notes irrelevant to “setup check” (negative). Eighty percent of data points were randomly selected for training the classifiers, while the remaining 20% were used for assessment. Each result was averaged over 5 repetitions. Five classifiers – deep neural network (DNN), k-nearest neighbors (KNN), logistic regression (LR), random forest (RF), and GaussianNB were tested.The text-based journal notes were vectorized by selected number of “features” or “combination of features” (i.e. ngrams) for training and testing the classifiers. The optimal number of features and ngrams were determined by four output criteria: positive predictive value (PPV), negative predictive value (NPV), specification, and sensitivity.
Results: All classifiers predicted similar NPV scores (median 97.9%). The DNN, KNN, and RF classifiers achieved relatively high sensitivity (median 91.3%) and specificity (median 97.6%) in classifying free text with minimum fluctuations. The bi-gram GaussianNB classifier performed the best in sensitivity (96.4% vs <= 93.8%) but suffer in both PPV (66.4% vs >= 91.7%) and specificity (91.7% vs >= 97.1%), which suggests high number of false positive in prediction. In contrast, the LR classifier produced the worst sensitivity (87.7% vs >= 92.6%).
Conclusion: Performance of mono-gram DNN, KNN, and RF classifiers met the needs of parsing free text journal notes for radiotherapy and can be implemented into our automation tool to improve the efficiency of physics weekly chart checks.
Keywords
Not Applicable / None Entered.
Taxonomy
Not Applicable / None Entered.
Contact Email