Latest Research in Medical Education - Newsletter #19

400+ abstracts reviewed, here are the greatest hits!

Welcome to our 19th newsletter! As with prior newsletters, I’ve highlighted what I find to be the critical pearls within each abstract and called out my #TOPPICK(s)!

I found so much good work in the past two weeks, it was hard to get down to a manageable set of articles… I guess this is a good problem to have 🤷🏻

Learning Science:

1. Electronic Flashcards in Health Professions Education: A Scoping Review #TOPPICK
   Electronic flashcards (EFs) are a widely used learning resource in medical education. This scoping review provides a comprehensive overview of the existing empirical research on EFs and a direction for future research on EF interventions. The authors organized the reviewed articles into 4 nonmutually exclusive categories: development, delivery, utilization, and associated outcomes. The database search was conducted on April 12, 2024, for articles published before this date using the following keywords (as well as other keywords): anki, quizlet, firecracker, osmosis, flashcard, spaced repetition, spaced interval training, spaced interval learning, spaced test, and parallel curriculum. This review of the rapidly increasing body of research on EFs suggests broad use and satisfaction among health professions learners; however, much work remains to be done to systematically understand EF development and delivery. Future research directions may consider more systematic investigations of how decisions made during the development and delivery of EFs affect downstream use and learning outcomes. Addressing these gaps will deepen the understanding of how EFs affect medical education pedagogy and facilitate the informed integration and refinement of these tools within curricula.

genAI/LLMs:

1. Artificial Intelligence in Health Professions Education assessment: AMEE Guide No. 178 #TOPPICK
   Health Professions Education (HPE) assessment is being increasingly impacted by Artificial Intelligence (AI), and institutions, educators, and learners are grappling with AI's ever-evolving complexities, dangers, and potential. This AMEE Guide aims to assist all HPE stakeholders by helping them navigate the assessment uncertainty before them. Although the impetus is AI, the Guide grounds its path in pedagogical theory, considers the range of human responses, and then deals with assessment types, challenges, AI roles as tutor and learner, and required competencies. It then discusses the difficult and ethical issues, before ending with considerations for faculty development and the technicalities of AI acknowledgment in assessment. Through this Guide, we aim to allay fears in the face of change and demonstrate possibilities that will allow educators and learners to harness the full potential of AI in HPE assessment.

2. A mixed-methods study comparing human-led and ChatGPT-driven qualitative analysis in medical education research #TOPPICK
   Qualitative research, used to analyse non-numerical data including interview texts, is crucial in understanding medical education processes. However, it is often complex and time-consuming, leading to an interest in technology for streamlining the analysis. This study investigated the applicability of ChatGPT, a large language model, in thematic analysis for medical qualitative research. Previous research has used ChatGPT to explore the deductive process as a qualitative study. This study evaluated thematic analysis including the inductive process by ChatGPT with reference to human qualitative analysis. ChatGPT scored higher in most aspects but showed variable transferability and mixed depth scores. In the integrated analysis including qualitative data, six themes were identified: superficial similarity of results with human analysis, good first impression, explicit association with data and process, contamination by directions in prompts, deficiency of thick descriptions based on context and research questions, and lack of theoretical derivation. ChatGPT excels at extracting key data points and summarising information; however, it is prone to prompt contamination, which necessitates careful scrutiny.

3. Comparing human-made and AI-generated teaching videos: An experimental study on learning effects #TOPPICK
   In this study, the authors used generative AI tools to develop four short teaching videos for a management course and then compared them with human-generated videos on the same subjects in an online experiment. In an across-subject experimental design, 447 participants completed two treatment conditions presenting different mixes of AI-generated and human-made videos. The findings show that human-generated videos provided a statistically significant but small advantage to participants in terms of learning experience, indicating that the participants still prefer to be taught by human teachers. However, a comparison of exam results between the experimental groups implies that the participants eventually acquired knowledge about the content to a similar degree. Given these findings and the ease with which AI-generated teaching videos can be created, this study concludes that AI-generated teaching videos will likely proliferate.

4. Artificial Intelligence and ChatGPT in Medical Education: A Cross-Sectional Questionnaire on students' Competence
   Artificial intelligence is rapidly transforming the field of health science and medical education, but less is known about the students´ competencies related to knowledge, skills and attitudes towards the application of AI tools like ChatGPT. Therefore, a unicentric questionnaire-based cross-sectional study was applied to students in the medical field (n = 207). The data revealed that while most students were familiar with ChatGPT (66.7%), other AI tools were significantly less known or utilised for study purposes. Students approached AI tools rather informally, often preferring to use them as a simple search engine. More than half of the students admitted that they were not sufficiently informed about the underlying technology of AI. Overall, the majority of respondents showed interest in and positivity towards the introduction of AI. However, they did not feel adequately prepared to handle AI confidently, leading many to express interest in further training. This training should be directly related to students' professional roles, e.g. as a physician. Notable subgroup differences related to the students" gender or self-assessed study performance were observed and should be considered in future research.

5. IAMSE Artificial Intelligence Meeting Survey: AI's Impact on Medical Education Faculty
   This report describes the results of an AI Needs Assessment survey conducted during the International Association of Medical Science Educators (IAMSE) Annual Meeting in June 2024. It assessed IAMSE members' use of AI, implementation challenges, their organization's AI initiatives, and internal and external resources needed for effective AI adoption. Although many respondents were comfortable with AI (53%), they reported substantial challenges in adopting AI at the personal and organizational levels. A high percentage of institutions were at the initial stages of integrating AI (86%). Most respondents requested online training and a certificate course (53%).

#socialQI/Social Media

1. Irritability and Social Media Use in US Adults #TOPPICK
   Efforts to understand the complex association between social media use and mental health have focused on depression, with little investigation of other forms of negative affect, such as irritability and anxiety. This survey study analyzed data from 2 waves of the COVID States Project, a nonprobability web-based survey conducted between November 2, 2023, and January 8, 2024, and applied multiple linear regression models to estimate associations with irritability. Survey respondents were aged 18 years and older. Across the 2 survey waves, there were 42 597 unique participants. In total, 33 325 (78.2%) of the survey respondents reported daily use of at least 1 social media platform, including 6037 (14.2%) using once a day, 16 678 (39.2%) using multiple times a day, and 10 610 (24.9%) using most of the day. Frequent use of social media was associated with significantly greater irritability for more than once a day vs never; for most of the day vs never; for more than once a day and; for most of the day. Irritability represented another correlate of social media use that merits further characterization, in light of known associations with depression and suicidality.

2. The Assessment of TikTok as a Source of Quality Health Information on Human Papillomavirus: A Content Analysis
   ackground Various studies have evaluated the quality of health-related information on TikTok (ByteDance Ltd., Beijing, China), including topics such as COVID-19, diabetes, varicoceles, bladder cancer, colorectal cancer, and others. However, there is a paucity of data on studies that examined TikTok as a source of quality health information on human papillomavirus (HPV). This study, therefore, evaluated the quality of health information on HPV on TikTok. Methods The terms "HPV" and "human papillomavirus" were searched on TikTok on a single day in August 2024, and 200 videos were retrieved. Relevant user metrics were collected for each video, including the number of likes, shares, and followers, the video length, and the uploader type. The overall quality of HPV videos uploaded on TikTok is very poor and not acceptable, thus failing to satisfy public health needs. Healthcare professionals must enhance their social media presence, produce reliable and substantive material, and collaborate with social media platforms and high-engagement accounts to facilitate users' access to high-quality data. TikTok users must recognize that material regarding HPV may lack medical accuracy and should consistently consult healthcare providers for medical guidance.

3. Development, Implementation, and Evaluation of a Virtual MedTwitter Curriculum for Medical Students
   Social media has numerous academic and professional benefits and is increasingly valued within healthcare. MedTwitter is an online community of medical professionals on the X platform (formerly Twitter). Despite MedTwitter's numerous benefits and far-reaching users, few medical schools teach students about this resource. We addressed this educational gap by implementing a MedTwitter curriculum designed for medical students. 43 pre-clinical students voluntarily registered; only 26% had used MedTwitter previously. Pre- and post-course surveys showed significantly increased perception of MedTwitter as a valuable resource, intent to increase MedTwitter utilization, and significantly higher comfort with MedTwitter skills. Learning objectives were achieved, determined by survey results, group discussion responses, and products of course activities. Networking and learning were frequently cited benefits. Most responded positively to the self-paced, virtual design.

General Themes:

1. Creating pragmatic, rapid-cycle, evidence-based innovation: The Kaiser Permanente Northern California Delivery Science and Applied Research (DARE) program #TOPPICK
   This article describes one large-scale, generalizable model: The Permanente Medical Group's (TPMG) Delivery Science and Applied Research (DARE) program within Kaiser Permanente Northern California. This organization-level initiative was designed to: (1) elicit clinical innovation and quality concerns across clinical specialties; (2) prioritize these questions for next-step evaluations; (3) create dedicated analytic, research, and clinical expertise to rapidly study the questions generated; (4) develop communities of clinician-researchers, embedded within their specialties' clinical operations, who gather ideas for evaluation, generate evidence, and facilitate implementation of research results (a.k.a. evidence-based innovators); and (5) broadly disseminate findings, to connect results with potential next-step implementation. #RapidLearningHealthcareSystem

2. The Complete Inpatient Record Using Comprehensive Electronic Data (CIRCE) project: A team-based approach to clinically validated, research-ready electronic health record data #TOPPICK
   The rapid adoption of electronic health record (EHR) systems has resulted in extensive archives of data relevant to clinical research, hospital operations, and the development of learning health systems. However, EHR data are not frequently available, cleaned, standardized, validated, and ready for use by stakeholders. We describe an in-progress effort to overcome these challenges with cooperative, systematic data extraction and validation. A multi-disciplinary team of investigators collaborated to create the Complete Inpatient Record Using Comprehensive Electronic Data (CIRCE) Project dataset, which captures EHR data from six hospitals within the University of Pennsylvania Health System. Analysts and clinical researchers jointly iteratively reviewed SQL queries and their output to validate desired data elements. The CIRCE Project represents a novel cooperative research model to capture clinically validated EHR data from a large diverse academic health system in the greater Philadelphia region and is designed to facilitate collaboration and data sharing to support learning health system activities. Ultimately, these data will be de-identified and converted to a publicly available resource. #RapidLearningHealthcareSystem

3. Comparing Virtual Reality-Based and Traditional Physical Objective Structured Clinical Examination (OSCE) Stations for Clinical Competency Assessments: Randomized Controlled Trial
   Objective structured clinical examinations (OSCEs) are a widely recognized and accepted method to assess clinical competencies but are often resource-intensive. This study aimed to evaluate the feasibility and effectiveness of a virtual reality (VR)-based station (VRS) compared with a traditional physical station (PHS) in an already established curricular OSCE. Apart from some hesitancy toward its broader application in future practical assessments (mean 3.07, SD 1.37 for VRS vs mean 3.65, SD 1.18 for PHS; P=.03), there were no other differences in perceptions between VRS and PHS. o9isaz]\ Regarding cost-effectiveness, initial development of the VRS can be offset by long-term savings in recurring expenses like standardized patients and consumables.

4. Development and Outcomes of a Provider-Driven, Online Continuing Education Program on Integrative Palliative Care: Randomized Controlled Trial

   Objectives: (1) Elicit input from multidisciplinary PC providers on needs for CE/CME content on integrative care, and indicators of implementation for use in impact assessment. (2) Produce an online CE/CME program responsive to provider input. (3) Assess program impact on PC providers in a randomized controlled trial. Qualitative analysis then informed development a 9-hour CE/CME program, and identified candidate items for an outcome measure (Integrative Practice Assessment) to assess impact. Validation testing followed with 63 new subjects. 213 subjects were randomized and 170 provided follow-up data on program impact. Subjects' confidence (10-point scale) understanding safety considerations, recommending modalities, and explaining modalities increased. Direct actions promoting integrative care in the last 10 patient encounters increased. Indirect actions taken in the work setting increased in the past month to advocate integrative therapies and to promote organizational change. All outcomes P < .001. This provider-driven CE/CME program led to significant positive changes in practitioners' self-efficacy and implementation of integrative care practices in PC settings. The results indicate that CE/CME can have measurable impacts that benefit providers and may potentially impact patients, families and the culture of care.

5. Experience of Medical School Faculty in the Transition From Lectures to Active Learning: A Phenomenographic Study
   Although there is a robust literature on the benefits and outcomes of active learning in medical education, little is known about the faculty experience of transitioning from lecture-based teaching to active learning in the preclinical, foundational science curriculum. The authors explored how faculty describe changing from lecture to active learning and how that change relates to the loci of control and basic psychological needs of faculty. The 15 participants described the change in instructional focus from lecture to active learning in 2 fundamentally different ways, defining the outcome space as 2 parallel sets of categories of description. A process set of categories describes the change in instructional focus from lecture to active learning as a process of instructional change, change in role, and socialization with other educators. The other categories correspond to motivation or demotivation via external and internal regulations related to self-determination theory. This study offers valuable insights into the interconnected aspects of this educational transformation, providing perspectives that can guide future efforts in curriculum redesign and faculty support programs.

---

As always, don’t forget to review and subscribe to our JCEHP podcast: Emerging Best Practices in CPD.