Consensus-Based Survey Design: Assessing Formalized Professionalism Training in Medical Physics Residency Training Programs
I Vergalasova1*, K Hendrickson2, H Al-Hallaq3, A Rodrigues4, L Padilla5, L Schubert6, (1) Rutgers, Cancer Institute of New Jersey, New Brunswick, NJ, (2) University of Washington, Seattle, WA, (3) The University of Chicago, Chicago, IL, (4) Duke University Medical Center, Durham, NC, (5) Virginia Commonwealth University, Richmond, VA, (6) University of Colorado Denver, Aurora, CO
Presentations
PO-BPC-Virtual-6 (Saturday, 4/17/2021) [Eastern Time (GMT-4)]
Purpose: Professionalism is one of the CAMPEP standards required for accreditation of medical physics residency training. Given the lack of literature and resources regarding formalized professionalism training and its definition in medical physics, this work aims to design an effective survey to collect data from program directors (PDs) on the current status of formalized professionalism training.
Methods: A systematic approach was implemented during the survey design process. A working group of 5 AAPM-MPRTP/SDAMPP committee members interested in professionalism training was formed. A short pilot survey was sent out to 15 PDs to assess the need (i.e., definition of professionalism, methods for teaching it, skill assessment, and associated challenges). This prompted the working group to design an official survey via a consensus-based approach. Upon completion, the survey was first validated using cognitive pre-testing with an independent reviewer over a qualitative discussion of verbal-probing and think-aloud techniques. Next, quantitative validation was performed by six independent content experts with questions rated on a scale from 1-4 each of increasing relevance and clarity. Metrics known as the reliability agreement (RA), the degree to which interrater ratings agree with each other, and the content validity index (CVI), which measures the collective validity of the expert’s rating per question, were calculated from the ratings, each for relevance and clarity. Based on published guidelines, a CVI>0.8 indicates an effective survey question.
Results: Pre-survey results indicated wide variability in defining professionalism and the need for formalized training and resources. An official survey of 20 questions divided into 6 sections was developed for validation. In regards to relevance, the average RA=85% and CVI=98%. For clarity, the average RA=80% and CVI=97%.
Conclusion: Validation results demonstrated the success of a consensus-based approach for survey question design. Final survey revisions will incorporate validation results and feedback prior to distribution to PDs.
Keywords
Taxonomy
Education: Knowledge of principles and generalizations
Contact Email