Transgender and gender non-binary (TGNB) patient care is not routinely taught in medical training. As a result, clinicians frequently lack knowledge regarding gender-affirming practices, surgeries and medications. TGNB-specific health knowledge and care delivery are further negatively impacted by pervasive societal transphobia. Virtual reality (VR)-based instruction may provide learners with opportunities for perspective taking, empathy building and attitudinal shifting to improve care delivery to TGNB patients.
This was a feasibility trial that compared passive VR and webinar in changing learner attitudes, knowledge and medical decision-making (MDM). Residents were recruited from five US emergency medicine (EM) residencies and were randomized to either VR or webinar educational intervention arm. Pre- and post-intervention assessments were matched using a unique identifier. Participants provided feedback about platform usability.
From August 2020 to October 2021, 206 resident participants were randomized into a case-based VR experience or webinar intervention on TGNB-specific EM medicine; 52 residents completed all three steps of the study. Webinar participants demonstrated more changes in attitudinal statements than VR participants. No improvements in knowledge or MDM were appreciated in either intervention arm. Participants in both arms shared positive and critical feedback about the intervention they accessed, with convenience and ease of use being valued for asynchronous webinar learning.
The webinar arm demonstrated a higher increase in empathy or change in attitude compared to VR learners. As VR becomes more technologically agile, accessible and reliable, more work is needed to see if this novel format is an educational tool that closely approximates the standardized patient encounter.
What this study adds:
There are approximately 1–1.3 million transgender and gender non-binary (TGNB) individuals in the United States [1–3], many of whom experience significant discrimination and health inequities compared to their cisgender peers [4–5]. This includes disproportionate rates of sexual abuse, suicide and eating disorders, to name only a few [6,7]. Although TGNB individuals often seek gender-affirming medications or surgeries, studies across various medical specialties, including emergency medicine (EM), consistently demonstrate that TGNB patients have negative interactions and experiences with healthcare practitioners and institutions [8–15]. Poor experiences with healthcare practitioners are due to a combination of factors including gaps in clinician training and knowledge, as well as negative attitudes towards TGNB patients.
TGNB patient care is not yet routinely taught in medical education [16,17]. For example, in a 2023 investigation of medical imaging curricula, only 19 programmes included ‘TGD’ (transgender and gender diverse) content [18]. As a result, clinicians commonly lack fundamental knowledge regarding gender-affirming practices, surgeries and medications, and their potential side effects and complications. Instruction on TGNB care must be attuned to individualized nuance, diversity and evolving clinical evidence. This poses a challenge for EM clinical educators, few of whom have expertise, training or experience on this topic [9,19]. A recent trend towards asynchronous learning may be a way to leverage topic expertise across institutions where content expertise is not available.
TGNB-specific knowledge and patient care are negatively impacted not only by societal transphobia, but also transphobia within the medical community [20]. Transphobia negatively impacts TGNB-specific education by (1) decreasing the likelihood clinicians will participate in TGNB medical education activities and (2) restricting knowledge uptake by uncoupling emotional stimulation and the learning experience [21,22]. Experiential learning (also known as ‘simulation’) that uses standardized patients (SP) has demonstrated an impact on the complete learning spectrum, including in teaching TGNB healthcare content: improving knowledge, enhancing attitude and empathy, and producing measurable behaviour change [23–32]. Ideally, learners would have an opportunity to work with in-person TGNB SPs. However, this approach is resource intensive and requires both expert instructors and SPs who may not be available for or safe doing the work [33]. For these reasons, asynchronous learning may be particularly useful in teaching clinicians-in-training how to best serve a structurally marginalized population, like TGNB patients.
In passive virtual reality (VR), the learner can move anywhere in the virtual space; in active VR, the learner can also take actions that impact the scenario. A few studies have demonstrated the ability of VR to impact users’ emotional states [34–36] and it possesses the potential to generate meaningful attitudinal shifts and build clinician empathy for patient populations [37]. VR-based instructional experience might mirror SP-based training (impacting attitudes, empathy, knowledge and behaviour) in a scalable and affordable format through an immersive experience with opportunities for perspective taking, empathy building and attitudinal shifting; more information is needed.
To the best of our knowledge, no investigations in TGNB-specific medical education have compared asynchronous intervention formats, head-to-head, with respect to changes in empathy/attitude, knowledge and behaviour. Commonly, research on medical education interventions focuses on one format [26–32]. Given the increasingly virtual nature of education and the improved capacity of technology to facilitate education, a unique opportunity may exist for educators to teach clinicians-in-training how to best serve structurally marginalized patients using VR. VR, with the possibility to positively affect clinical behaviour, may be especially important for marginalized populations like TGNB patients. While simulation may offer ‘the best’ opportunities for learning, it may not be safe for TGNB people in many communities to do this work, and most clinical educators lack the expertise to facilitate effective instruction.
In this study, we compare passive VR instruction to a similarly scalable format of electronic, asynchronous education: webinar. Both require a one-time gathering of resources and experts and are then available, via internet access, to learners. But in comparison to VR, the webinar is not expected to impact users’ emotional states, so while it is scalable, it may be less impactful than VR on emotional stimulation, limiting the learning experience as described above. TGNB Research Applying Novel Strategies in Education (TRANS ED) evaluates whether a VR-based experience can increase empathy and explores VR’s impact on learning compared to a traditional webinar format.
TRANS ED sought to compare the effectiveness of a passive VR educational platform and a traditional online webinar in changing learner attitudes towards TGNB patients. Secondary objectives were to compare VR with conventional webinar instruction involving the capacity to transmit knowledge and change medical decision making (which documents intended clinical action), and to examine the usability of each modality.
U.S. emergency medicine residents were recruited, for participation, from five residency programmes (The Mount Sinai Hospital; Mount Sinai Morningside-West; Brown University; Kings County/SUNY Downstate; Baystate) from August 2020 to October 2021. Study sites included 3- and 4-year academic EM residency programmes and a 5-year EM/IM (emergency medicine – internal medicine) residency programme in north-east urban centres. Participants were randomized into either the VR or the webinar arm. Randomization, for each residency site, was performed by computerized randomized block design, with each post-graduate year (PGY) comprising a block. Each participant was assigned a unique, participant-generated, non-identifying ID. A pre-intervention attitude, knowledge and clinical scenario medical decision-making (MDM) assessment was completed prior to the intervention (VR or webinar) and the same assessment was completed after the conclusion of the education module. Pre- and post-intervention assessments were matched using the unique study ID and investigators and assessment raters were blind to participant identities.
All residents received an initial automated participation invitation via email from REDCap, and assessments were completed directly in REDCap [38]; on a weekly basis for at least 6 weeks, reminder invitational emails went out to all residents who had not yet participated for each residency. Participants who completed the study received asynchronous learning credit from their residency leadership and a $15 gift card. The Institutional Review Boards of (The Icahn School of Medicine; Rhode Island Hospital and the Miriam Hospital; SUNY Downstate Health Sciences University; Baystate Health) all approved the study or deemed it exempt.
Both educational interventions (VR and webinar) were developed by a gender-diverse study team with nationally recognized expertise in TGNB healthcare, EM and medical education; multiple team members have both academic and lived experience expertise. Both the VR module and the TGNB webinar covered the same background information, terminology and case-based scenarios. The VR and cases are discussed, and video links are shared in Baluyot et al. [39]. Case 1 centres a trans man presenting with abdominal pain [40]; case 2 centres a trans woman presenting with unilateral leg swelling [41] and case 3 centres a non-binary person presenting with shortness of breath [42]. The webinar was an audio-recorded lecture by two presenters using a slideshow presentation lasting 32 minutes. The webinar was viewed online by participants at their convenience [43].
The VR modules were three video-recorded SP encounters between a clinician and TGNB SP actors, lasting 27 minutes in total [40–42]. The VR modules each began from the patient’s perspective and then transitioned to the perspective of an observer within the room. The VR script used the principle of affective cognition, which asserts that humans use cognitive reasoning to interpret the words and behaviours of others to try to understand others’ emotions [44,45]. Incorporating affective cognition, the VR script guided TGNB actors in the VR module to create verbal (e.g. SP actor speaking to kin about concern for transphobia at the start of the visit) and non-verbal cues (e.g. was the SP actor was instructed on how to demonstrate pelvic pain during examination) aimed at increasing learner empathy towards TGNB patients and eliciting empathetic responses, which could improve retention of TGNB-specific knowledge. VR modules were viewed using Oculus VR (Meta, Irvine, CA) headsets, resulting in an immersive experience simulating in-person attendance by the viewer. The modules were passive, meaning the viewer watched the encounter and was not able to interact with or engage the TGNB SPs or instructors. Study participants accessed the VR headsets for module viewing at a centralized location at each study site.
The webinar was adapted from team members’ previous national, case-based presentations on TGNB EM care [authors MCS, AD, and EAS] and recorded by [authors AD and EAS]. The VR content was based on the webinar, with script development and recording led by [authors MCS, RG, MB, and ZG]. In addition to our gender-diverse TRANS ED team, gender-diverse patient actors were recruited for the VR format from professional actor networks and TGNB-service community-based organizations. They reviewed the scripts, revised the non-medical content based on their lived experience expertise and were paid a stipend for their participation. Medical students and clinicians acted as student learners, residents and supervisory clinicians in the VR scenarios.
The assessment instrument (see Supplementary Appendix) was developed by the research team which is comprised of experts in education assessments, research design and the care of TGNB patients. The instrument was largely based upon a pre-existing transphobia assessment tool [20], which included questions about participants’ political views as well as other demographics and extant literature. The instrument was reviewed by non-study team TGNB content experts and pilot tested in a small sample of ED clinicians (who were not study-eligible) to evaluate question comprehensibility. Consequent revisions resulted in the final instrument (see Supplementary Appendix).
The pre-/post-intervention assessments included questions about participant attitudes (13 questions, 7-point Likert scale), knowledge (3 true/false questions) and 3 TGNB clinical case prompts in which participants were instructed to document a patient history, a differential diagnosis and MDM. MDM responses were randomly assigned to and, using a standardized scoring rubric, scored by study team members (authors MCS, CH, MPJ, EAS, and FS). The pre-intervention assessment also included demographic questions and items about participants’ perceptions of their own structured medical training on TGNB care and interaction with TGNB people. The pre-intervention assessment could be taken at any time prior to the intervention. The post-intervention survey collected answers to questions on the webinar or VR format itself including items pertaining to ease of use, technical issues encountered and open-ended questions about their experience with the educational format. In the webinar arm, the post-intervention survey could be taken at any time, and we did not collect data on this timing. In the VR arm, participants completed the post-intervention assessment immediately after the VR experience.
This was a pre- and post-intervention study with two assessments that measured survey items, including usability, on an ordinal scale for the two educational modalities (study arms). To compare the pre- and post-intervention assessments within each modality, the differences between the pre-/post-pairs (gain scores) were evaluated using the Wilcoxon signed-rank test. The Mann–Whitney U test was used to compare respondent differences between modalities. An agreement study was conducted to assess the closeness of repeated assessments among the five raters, who scored the assessments’ MDM. Each rater evaluated 13 targets (10% of the data) and the degree of agreement was calculated using Gwet’s Agreement Coefficient (AC1). All statistics were computed using Statistical Analysis Software version 9.4 (SAS version 9.4). The usability assessment also included free text options that were not required for the assessment to be considered complete; this text was collectively reviewed by the team for salient and common feedback.
From August 2020 to October 2021, there were 206 resident participants from the five emergency medicine residencies (58% of the eligible 353). After randomization, 128 (62%) residents completed the pre-intervention survey (58 in the webinar arm, 70 in the VR arm). As indicated in Figure 1, 52 (42%) completed the intervention and the post-intervention survey (31 in the webinar arm, 21 in the VR arm). The demographics of residents that completed participation are found in Table 1; 69% of the participants were White, while 51% of eligible residents identified as White upon entering their programme. There were no differences in study arms with respect to gender, religion, religiosity, political leaning (83% identified as ‘liberal’), experience or training in TGNB health care, or knowing a TGNB person. MDM scoring for case 1 had a Gwet’s AC1 of 0.6 (substantial), case 2 had a Gwet’s AC1 of 0.7 (substantial) and case 3 had a Gwet’s AC1 of 1 (almost perfect).
| Webinar (n = 31) | Virtual reality (n = 21) | |
|---|---|---|
| Post-graduate year | ||
| PGY 1 | 48% (15) | 24% (5) |
| PGY 2 | 13% (4) | 24% (5) |
| PGY 3 | 23% (7) | 43% (9) |
| PGY 4 | 16% (5) | 10% (20) |
| Gender | ||
| Woman (cis or trans) | 65% (20) | 38% (8) |
| Man (cis or trans) | 35% (11) | 62% (13) |
| Race (can select ≥1) | ||
| African/American or Black | 6% (2) | 0 |
| Asian/American | 16% (5) | 38% (8) |
| Multiracial | 3% (1) | 0 |
| White | 74% (23) | 62% (13) |
| Ethnicity (can select ≥1) | ||
| Latinx/Hispanic | 10% (3) | 5% (1) |
| Arab/Persian/Middle Eastern | 3% (1) | 5% (1) |
| Religion (can select ≥1) | ||
| Agnostic | 26% (8) | 14% (3) |
| Atheist | 19% (6) | 19% (4) |
| Christian | 35% (11) | 33% (7) |
| Hindu | 0 | 14% (3) |
| Jewish | 16% (5) | 19% (4) |
| Other | 3% (1) | 0 |
| Religiosity | ||
| Not at all religious | 61% (19) | 67% (14) |
| Slightly religious | 23% (7) | 14% (3) |
| Moderately religious | 10% (3) | 10% (2) |
| Very religious | 6% (2) | 10% (2) |
| Political identity | ||
| Conservative | 6% (2) | 0 |
| Moderate | 13% (4) | 14% (3) |
| Liberal | 81% (25) | 86% (18) |
| TGNB experiences | ||
| Ever met a trans person? | 100% (31) | 95% (20) |
| Ever cared for a trans person? | 97% (30) | 90% (19) |
| Cared for a trans person in the past year? | 84% (26) | 81% (17) |
| Cared for a trans person in the ED? | 87% (27) | 90% (19) |
| Any medical education on trans care? | 84% (26) | 86% (18) |
| Any residency training on trans care? | 39% (12) | 38% (8) |
TRANS ED resident participants.
At baseline, 90% (28) of participants who completed the webinar arm were more comfortable (slightly agree, agree and strongly agree) taking care of TGNB patients in comparison to those who completed the VR arm (81%, 17; p < 0.01). Likewise, at baseline, 75% (23) of the webinar arm participants felt more capable of taking ‘very good care’ of TGNB patients; in comparison, 48% (10) in the VR arm expressed this sentiment (p < 0.05). In the webinar arm, PGY1s were 9.4 times more likely to complete all three steps than PGY2s (p = 0.03, confidence interval [CI] 1.3, 67.6). In the VR arm, there were no differences between residents who only completed the pre-survey and those who completed all three steps (pre-survey, intervention and post-survey).
Neither educational format resulted in a change in knowledge or MDM (Table 2). Empathy or attitudes improved in both study arms (Table 3). In the VR arm, Figure 2 demonstrates that participants’ attitudes about gender moved towards the ‘disagree’ side of the Likert spectrum with respect to this statement, with none ‘agreeing’ after the intervention: ‘A person’s genitalia defines their gender’ (p < 0.05). For the same statement, webinar participants also moved towards ‘disagree’, and again, none ‘agreed’ with the statement after the intervention (p < 0.02). For the statements, ‘When I meet someone, it is important for me to be able to identify them as a man or a woman’, and ‘I believe that the man/woman dichotomy is natural’, webinar participants moved towards ‘disagree’ (p < 0.002 and p <0.05, respectively). Due to a technological transcription error, the former statement was missing from the post-intervention assessment for VR participants; no attitudinal change was demonstrated in the VR arm for the latter question (Table 3). A positive shift in attitude, representing increased empathy, was noted in both intervention arms but more so in the webinar arm. Given the small sample size, sub-group analyses based on participant demographics were not planned or conducted.
| Pre-intervention mean score | Post-intervention mean score | p-Value | |
|---|---|---|---|
| Knowledge | |||
| Webinar (max points 3) | 2.9 | 2.9 | 1.0 |
| VR (max points 3) | 2.7 | 2.8 | 0.48 |
| Behaviour | |||
| Webinar | |||
| Case 1 (max points 12) | 7.2 | 7.1 | 0.81 |
| Case 2 (max points 9) | 5.4 | 5.9 | 0.46 |
| Case 3 (max points 9) | 4.9 | 5.8 | 0.08 |
| VR | |||
| Case 1 | 7.4 | 7.7 | 0.77 |
| Case 2 | 5.8 | 5.8 | 1 |
| Case 3 | 4.8 | 5.5 | 0.36 |
| ‘A person’s genitalia defines their gender’ | |||
|---|---|---|---|
| Pre-intervention mean | Post-intervention mean | p Value | |
| Webinar | 2.9 | 2.3 | 0.02 |
| VR | 2.2 | 1.7 | 0.05 |
| ‘When I meet someone, it is important for me to be able to identify them as a man or a woman’ | |||
| Webinar | 3.2 | 2.5 | 0.002 |
| VR | 3.2 | missing | n/a |
| ‘I believe the man/woman dichotomy is natural’ | |||
| Webinar | 2.9 | 2.3 | 0.05 |
| VR | 3.4 | 2.9 | 0.18 |
Questions answered on a Likert Scale from 1/Strongly Disagree to 7/Strongly Agree. Comparisons made using Wilcoxon signed-rank test.
Attitudes before and after TRANS ED educational interventions.
In comparison to VR participants, participants reported the webinar was easy to set up (p < 0.001), convenient (p < 0.009) and all said it had few technical difficulties (p < 0.0001). In comparison to webinar participants, VR participants reported it was more fun than other ways of learning (p < 0.0009). Participants in both groups reported appreciating the intervention type and preferring to learn in a different, non-conference lecture way (Table 4). Some webinar participants appreciated being able to control the speed of information, ‘Could speed up the playback to get through it quickly, and it was case based which helps me retain information’ (R13). Many recognized the value of webinar’s convenience, ‘Less interactive but more convenient, easy to access later for a refresher’ (R8). A few offered some version of this sentiment, ‘[Webinar] will work for some, will be glossed over by others. No learning tool is perfect for all people’ (R33).
| Intervention arm | Positive feedback | Mixed feedback | Critical feedback |
|---|---|---|---|
| Webinar | ‘Loved the timing and structure – well presented (R4)’ | ‘Would like more details about exactly how to examine transgender patients and more examples of what language to use. Would like to learn this way again (R1)’ | ‘About the same as ppt (R7)’ |
| ‘Could speed up the playback to get through it quickly, and it was case based which helps me retain information. I don’t retain information well in a pure lecture style webinar versus other interactive formats (R13)’ | ‘Worked well, not my favorite way to learn (R2)’ | ‘I prefer to read rather than watch/listen (R12)’ | |
| ‘It was nice. I would recommend it. Especially for early education in gender care, for those who have less experience with this population (R16)’ | ‘More boring than in person learning, less engaging, more convenient (R10)’ | ||
| ‘I like webinars especially when you can speed them up and still get all the information (R20)’ | ‘Easy to use but not engaging (R18)’ | ||
| Virtual reality | ‘Better than online videos and comparable to in person simulation (R40)’ | ‘More immersive than standard video, but could be more useful for other non-video applications Where the remotes are used to interact in a virtual environment, i.e. procedures. Cool unique experience that is better than a standard video, but slightly inconvenient to set up and use (R41)’ | ‘For teaching not involving procedures or physical involvement, I would prefer to stay at home and read/watch videos on subject matter instead (R45)’ |
| ‘I was likely more engaged. Felt like I was seeing real patients and made me reflect on my own practice. I think this was great and likely brought up key facts and things I had not yet been doing but will likely stick with me more than if I just watched a regular video (R47)’ | ‘Being put in the patient’s position at the beginning, however it quickly changed so that you were basically watching a video, would have been better had you played the patient throughout to get that experience. hard to coordinate logistically being on campus, a bit onerous to get started, didn’t really seem to add much given mostly just watching videos you could do on a different platform (R42)’ | ‘I think it would be more worthwhile if it was interactive. Or if we can do the observational part and then do simulation cases to apply the skills we learned to actual patient scenarios (R46)’ | |
| ‘There was no other option for distractions. you had to watch it and only it since you wore the goggles (R26)’ | ‘Allows suspended disbelief a lot more than normal sim sessions, lacked opportunity for interaction. ‘Enjoyed experience, gave [me a] headache due to pixilation, would do it again (R49)’ | ||
| ‘Mildly physically nauseating. Cases were informative. would benefit from debrief sessions with specific learning points/ takeaways (R50)’ |
In the VR arm, many appreciated the interactive nature of the intervention and wanted more interaction, ‘It was nice, but I think it could have been fleshed out more! Maybe a choose-your-own-adventure kind of style to it. It was excellent and I look forward to any future additions to this kind of learning’ (R37). Others wanted more of the ‘patient perspective’ part of the VR experience suggesting, ‘Continue in role of patient through experience’ (R39). One participant found VR useful because of its specificity, ‘Helpful to be reminded of specific questions to ask the trans community from a medical standpoint and to do so in an empathic and non-judgmental tone. Helps to ask in a way to demonstrate understanding or hope to understand’ (R43). Some VR participants found it inconvenient to access the headsets (secured by residency administration staff on hospital campuses) and that the passive nature of this VR did not enhance their learning, ‘Just a novelty. Seems too clunky and [a] waste of [money] to use with current tech’ (R38). A few participants found the headset to be weighty and/or nauseating, ‘the headset was very heavy and painful. Watching 3 scenarios on it gave me a headache’ (R48).
This was a small, randomized, feasibility trial comparing the effects of VR and webinar educational interventions on EM resident attitudes, knowledge and MDM about TGNB patient care. There was a small improvement in empathy or attitude in both the VR and webinar groups with more improvements in attitudinal statements in the webinar arm (three statements) than in the VR arm (one statement). There was no significant change in knowledge or MDM post-intervention in either the webinar or VR arm.
To the best of our knowledge, there is no other literature demonstrating webinar’s capacity to impact empathy or change attitude; given the existing data about simulation [23–32], the findings of this study were not expected. VR is theoretically more akin to simulation than webinar and VR has been linked with emotional activation [34–37], so we hypothesized that, as an educational method, even passive VR would affect empathy and attitude more than the webinar intervention. In this study, participants watched a scenario from two perspectives, though they could ‘move’ throughout the room at will (passive VR). There may be increased learner empathy in an interactive VR format. Indeed, in the post-assessment feedback, several participants stated that an interactive format would have enhanced their experience. High-fidelity, interactive VR experiences may be a promising alternative when encounters with live SPs are not feasible (see Introduction) [33]. Importantly, while empathy has been linked to knowledge and behaviour in other work [20], that linkage was not found in this study. Here again, the expected finding may have occurred if interactive VR had been possible. Future educational intervention studies should investigate webinar in comparison to interactive VR, how both affect empathy, and whether increased empathy is then linked with differential knowledge and clinical behaviour.
At the start of each scenario, we included a brief ‘first-person’ perspective of a TGNB patient on the phone with a loved one. The goal was for VR learners to have an opportunity to hear some of the concerns that TGNB patients often have when seeking EM care. While a few residents reported wanting to continue the VR from the patient’s perspective, this feedback should be implemented with caution. Although the development of empathy is coupled with knowledge [20], it is important to critically consider having non-TGNB people incorrectly think they know what it feels like to be TGNB, even in a clinical setting. Commonly, in educational simulations, health professionals are practising clinical positions they would hold when providing EM care (e.g. being team leader). The goal is for health professionals to learn the skills they need to provide high-quality care to certain populations or in specific clinical scenarios. It may be harmful for people without lived experience of structural exclusion or marginalization to think they have more experiential understanding than they do. At a minimum, such role play may be offensive as has been seen when simulations have people who do not experience disability play at ‘experiencing’ disability [46,47].
In the process of studying these elective asynchronous educational interventions, we found learners reported trade-offs in the delivery mechanism. Despite the novelty of the VR format, participants in both the webinar and VR groups favoured ease of use as an important quality for an educational intervention. For the VR sessions, participants had to schedule an appointment and arrange to be on site. In contrast, the webinar, which could be watched anywhere, at any time and at any speed, may have had an advantage. With VR headsets becoming more affordable, the ability to engage in VR from the comfort of home may soon be a universal reality, which might increase the attractiveness and reach of the VR learning format. Briefly, there were devices called ‘Google Cardboard’ available for less than $10, which turned a smartphone into a simplified VR headset; third-party companies now sell them [48]. These were not available at the design of our study but in future investigations using this technology may improve learner participation and experience. In our study, a few learners found the physical experience of the VR headset to be uncomfortable, so the option of participating from home with the ability to take breaks might improve their experience.
This study developed a protocol that helped investigators explore outcome variables and instruments used to measure them. Well-funded and larger studies are needed to compare interactive VR, passive VR and webinar educational interventions. If evidence-based VR and/or webinar modules collaboratively created by people with lived experience are freely and broadly shared (e.g. national repository of modules), the reach and impact could be especially valuable to improving the capacity of clinicians to empathetically and aptly care for marginalized patients.
This study struggled with enrolment and completion despite explicit resident requests to residency leadership for TGNB content and reportedly strong interest in the VR format. Only a quarter of invited residents completed the study, with the VR arm experiencing a lower completion rate. In addition to the weekly reminder emails via REDCap, PGY champions and residency leadership encouraged all residents to participate by reminding residents at weekly conference, and informally while on shift or in other routine meetings. Given the timing of this study which coincided with the early portion of the U.S.-based COVID-19 pandemic, high stress burden on resident learners unfamiliar with prolonged disaster medicine may have contributed to low participation at a time when asynchronous learning opportunities might otherwise have been attractive.
A larger sample size is needed to further explore the role of VR in healthcare education, especially on topics specific to marginalized populations. Besides recruitment and retention difficulty, this feasibility pilot was limited by the size of participating residencies and the need to randomize participants by residency site and year. More work is also needed to explore webinar’s role in increasing empathy. At baseline, the webinar arm participants were more confident in their ability to and comfortable taking care of TGNB ED patients; this may have contributed to the post-intervention increase in empathy in the webinar arm participants. Additionally, our study could only support a passive VR experience. It is possible that an interactive VR experience may yield different results but requires substantial funding.
All participants who accessed the VR format completed the post-intervention assessment, but we could not learn about the impact of the intervention on participants who viewed the webinar and did not complete the post-intervention assessment. It is possible that such learners had changes in their attitudes, knowledge and/or MDM. Future investigations should explore ways to improve post-assessment completion for the webinar arm. There is also the potential for desirability bias. Participants may have answered questions based on what they perceived as the ‘correct answer, rather than how they felt. One of the attitudinal questions was missing from the post-VR assessment, so VR may have had more impact on attitude (though still less than webinar) than we were able to measure.
TRANS ED study participants may have been more likely to have a higher baseline of empathy towards TGNB patients and/or knowledge about TGNB-specific EM care than residents who did not participate in the study; this would limit their ability to demonstrate the intervention’s intended positive change in attitude. Multiple other sources of literature, as well as a review of medical curricula, demonstrate that TGNB care is not yet a standard part of medical education [16–19]. TRANS ED study participants, many of whom reported receiving medical education in this topic, may have been outliers as they were willing to participate in a three-part study that centres TGNB health care. Such resident participants may also have been more inclined to attend professional schools with TGNB course content. Future investigations should study required educational interventions (of interactive VR and webinar) to mitigate this study’s high baseline of empathy. A differently structured investigation that allows for probing questions would be helpful in qualitatively exploring intervention usability and how residents want to learn.
In this study of emergency medicine residents, passive VR did not demonstrate an advantage over webinar in producing an empathetic change towards TGNB people. In fact, for the first time in medical educational literature, webinar demonstrated more attitudinal changes than passive VR. While many participants found the VR experience to be enjoyable, the webinar was convenient, easy to use and had fewer technical difficulties. As VR becomes more technologically agile, accessible and reliable, more work is needed to see if this novel format is an educational tool that closely approximates the standardized patient encounter.
This study is funded by the Icahn School of Medicine at Mount Sinai’s Institute for Medical Education. We are deeply appreciative of the study participants for their willingness to engage in this investigation. Data collection would not have been possible without the following people: Laura Berluti; Benjamin Asriel, MD; Carolina Camacho-Ruiz, MD; Lee Ann Jordison Keeler, MSW; Jane Kim, MD EdD; Allison Lockwood, MD; Michael Norris; Kye Poronsky; Eleanor Roberts Aluise, MD; Sophia Spadafore MD; Angela Sullivan. Thank you to the following people served as actors in the VR scenarios: Danish Ahmad, MD; IV Castellanos; Octavia Lewis; Emmett Jack Lundberg; Carina Seah; Trent She, MD; Jonathan Yeo, MD. We are also grateful to the following people whose support made this investigation possible: Heidi Baer, MD; Chen He, MD; Andy Jagoda, MD; Jared Kutzin, DNP, MS, MPH, RN, FSSH; Trevor Pour, MD; Elaine Rabin, MD; Lynne D. Richardson, MD; Scott Robertson, MSc, PhD. And thank you to Marc Probst, MD MS who helped creatively name this project. This work was supported in part through the computational and data resources and staff expertise provided by Scientific Computing and Data at the Icahn School of Medicine at Mount Sinai and supported by the Clinical and Translational Science Award (CTSA) grant UL1TR004419 from the National Center for Advancing Translational Sciences.
All authors give final approval for the submitted manuscript and are accountable or all aspects of the work. MCS: study conception, design, acquisition, data interpretation, manuscript drafting. AD: study design, data interpretation, manuscript drafting. SM: study conception, design, manuscript drafting. RG: study design, manuscript revisions. CH: study design, manuscript revisions. MB: study design, data interpretation, manuscript revisions. MC: study conception, design, data interpretation, manuscript revisions. CC: study design, data acquisition, manuscript revisions. GL: study design, data analysis, manuscript drafting. MPJ: data acquisition, manuscript revisions. EAS: study conception, design, manuscript revisions. LW: data acquisition & interpretation, manuscript revisions. FS: data acquisition, manuscript revisions. ZG: study conception, design, data interpretation, manuscript revisions. MCS takes responsibility for the paper as a whole.
Grants funded by Icahn School of Medicine at Mount Sinai’s Institute for Medical Education.
None declared.
None declared.
The authors report there are no competing interests to declare.
This study recruited from emergency medicine residencies in 3 US cities: New York City, New York; Providence, Rhode Island; and Springfield, Massachusetts.
18 September 2021, 13th Annual International Pediatric Simulation Symposia and Workshops (IPSSV2021; a virtual conference).
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
44
45
46
47
Facebook
Twitter
Linkedin
Whatsapp
Tweets