International Journal of Healthcare Simulation

Introduction

Simulation-based education (SBE) is a well-accepted component of medical education. SBE promotes skill acquisition and competence by practice in a safe learning environment [1]. This environment allows for provision of expert feedback which, combined with the principles of deliberate practice and mastery learning, facilitates patient safety by obtaining competency prior to patient contact [1–5]. SBE facilitates clinical experience in an era of decreasing exposure to meaningful clinical encounters [6,7].

Simulation encompasses a broad range of educational approaches and techniques. It may involve any activity that replaces or amplifies real experiences with guided experiences to evoke or replicate aspects of the real world in a fully interactive fashion [8]. SBE utilizes various modalities depending on the nature of the task, clinical scenario or objectives. Examples of these include role-play, simulated patients, part-task trainers (synthetic, animal or cadaveric), simulated scenarios, virtual reality simulation, laparoscopic bench trainers and combinations of these examples [1,9]. This allows application across a breadth of healthcare professions and levels of experience [10–12].

Despite the ubiquity of SBE in medical education and assessment, the broader attitudes of learners towards SBE are not commonly assessed formally or systematically at an institutional level (versus for individual activities). Learner feedback is not routinely being sought in a field of education that is inherently learner centred. Whilst analysis of specific interventions on learning outcomes is of benefit, we propose that analysis of students’ attitudes towards SBE is also of benefit by means of identifying areas of improvement to enhance students’ experience and engagement in their education.

Current literature largely focuses on particular simulated tasks or scenarios, analysing students’ performance and/or feedback for a specific simulation [11–19]. On the contrary, there is little research that assesses students’ attitudes towards SBE in its entirety. Joseph et al. [20] and Agha et al. [21] have published cross-sectional studies describing Indian and Saudi Arabian medical students’ (respectively) attitudes towards SBE. To our knowledge, there is no such study in the Australian context.

McGaghie et al. [22], in describing 12 components of SBE best practice, outlined that SBE should feature feedback, outcome measurement, instructor training and educational/professional context. Our study directly and indirectly assesses the application of these principles.

We aimed to gain insight into student experiences of SBE, particularly in regards to (1) current provision of SBE, (2) learning opportunities and (3) areas for improvements, innovation and focused initiatives. Consequently, we can create a platform by which focussed initiatives can assess and address specific needs to improve SBE for learners and educators alike.

Methods

We aimed to gain insight into students’ exposure to and attitudes towards SBE. Particular areas of interest were:

• Gaining insight into the nature of the current provision of SBE.
• Identifying student opportunities for learning through SBE.
• Identifying student attitudes towards SBE (including what students value).
• Identifying specific areas for improvement.
• Identifying specific areas for innovation and focused initiatives.
• Screening for negative emotional reactions to SBE.

Results

Participant demographics

Two hundred and forty-six medical students participated, with 76.0% (187/246) completing all questions. Participants were predominantly female (66.8%, 163/244) and median age 22 [21–23] years. Demographic characteristics are presented in Table 1.

Table 1:

Participant demographic characteristics

	% (n/N *)
Gender
Female	66.8% (163/244)
Male	33.2% (81/244)
Year level
Year 1	3.3% (8/245)
Year 2A	5.7% (14/245)
Year 3B	10.6% (26/245)
Year 4C	59.6% (146/245)
Year 5D	20.8% (51/245)
Undergraduate or graduate
Undergraduate	82.9% (204/246)
Graduate	17.1% (42/246)
Metropolitan or rural
Metropolitan	76.6% (180/235)
Rural	23.4% (55/235)
Domestic or international
Domestic	88.5% (201/227)
International	11.5% (26/227)

*N denotes the number of participants who responded to each question.

SBE previous experience

All but two students* reported previous participation in SBE (99.2%, 235/237), occurring on average every 4 [1–8] weeks. Both students who had not participated in SBE were in the preclinical phase of their medical education when there is a greater focus on the basic sciences. Most students (86.4% 203/235) had experienced SBE in an extracurricular environment, largely through university clubs (54.9%, 129/235), self-practice (48.9%, 115/235) and external skills-based courses (35.7%, 84/235). 43.2% (102/236) of students reported having paid for a simulation activity, with a median bracket cost of AUD$20–39 [$0–19 to $40–59].

Students participated primarily in simulated patients/role-play (97.9%, 230/235) or scenario-based simulations (92.3%, 217/235). Students also reported these methods as the most beneficial (Table 2). Synthetic part task trainers were beneficial (58.3%, 77/132), though few (6.1%, 8/132) students reported them as the most beneficial. There was perceived limited utility of augmented/virtual reality trainers (25.5%, 60/235) and online simulation (20.9%, 49/235) in comparison to more tactile forms of SBE.

Table 2:

Student exposure to various forms of SBE and debriefing

Modality of SBE	Participant exposure, % (n/N *)	SBE utility (rated first three preferences), % (n/N *)	SBE utility (rated first preference), % (n/N *)
Simulated patient/role-play	97.9% (230/235)	80.3% (106/132)	39.4% (52/132)
Scenario-based simulation	92.3% (217/235)	87.1% (115/132)	40.2% (53/132)
Part task trainer – synthetic	83.8% (197/235)	58.3% (77/132)	6.1% (8/132)
Part task trainer – animal	59.2% (139/235)	31.8% (42/132)	4.6% (6/132)
Part task trainer – cadaveric	49.4% (116/235)	20.5% (27/132)	4.6% (6/132)
Augmented/virtual reality trainer	25.5% (60/235)	4.6% (6/132)	3.0% (4/132)
Online simulation	20.9% (49/235)	1.5% (2/132)	0.8% (1/132)
Laparoscopic bench trainer	43.4% (102/235)	12.9% (17/132)	1.5% (2/132)
No debriefing	8.9% (19/214)	0% (0/102)	0% (0/102)
Video-assisted debriefing	6.5% (14/214)	2.0% (2/102)	1.0% (1/102)
Peer feedback	67.8% (145/214)	77.5% (79/102)	7.8% (8/102)
Instructor feedback	89.3% (191/214)	98.0% (100/102)	82.4% (84/102)
Self-appraisal (to group)	44.9% (96/214)	29.4% (30/102)	2.0% (2/102)
Self-appraisal (individual)	36.4% (78/214)	13.7% (14/102)	0.0% (0/102)
Written feedback (including mark/grade)	23.4% (50/214)	23.5% (24/102)	1.0% (1/102)
Informal opportunistic feedback	37.9% (81/214)	30.4% (31/102)	5.9% (6/102)

Notes: Students were instructed to select all options that applied and then select the form(s) of SBE/debriefing which they found to be (A) beneficial and (B) single most beneficial.

*N denotes the number of participants who responded to each question.

Student attitudes

Students rated their degree of agreement with 15 statements regarding the utility of SBE. A Cronbach’s alpha value of 0.89 demonstrated a high level of internal consistency among students’ responses.

Students were very much in favour of SBE, with 90.1% (191/210) of participants stating that SBE benefits their learning. When asked to rate aspects of SBE out of 10 (0 – not at all useful, 10 – very useful), SBE was viewed to benefit students’ learning (9 [8–10]), make their learning more engaging (9 [8–10]), and aide the transition from knowledge to application (9 [8–10]). While still in favour, students placed less value on aspects of SBE that involved interpersonal communication (score 7 [6–8]) and the development of leadership skills (score 7 [6–9]).

Students reported the most beneficial aspects of SBE to be learning a new skill under supervision (90.3%, 187/207), applying prior knowledge to a clinical scenario (73.4%, 152/187) and identifying gaps in knowledge/skill (73.4%, 152/187).

Qualitative analysis revealed areas of student concern regarding SBE, including ‘if the groups are too big it can waste time and become disengaging’, ‘self-appraisal with no guidance/lack of personalised feedback’, ‘when the scenario is not realistic or relevant’ and ‘anxiety and feelings of judgement’. Conversely, ‘working as a team in emergency scenarios’, ‘getting technique revised and critiqued by professionals’, and ‘practising while being able to make mistakes’ were all highly valued aspects of the SBE curriculum.

Students’ demographic characteristics and experiences during their medical degree did not affect the perceived utility of SBE (Table 2).

Overall, the majority of students (97.1%, 202/208) felt that SBE is a useful component of their education (9 [8–10]) and that it should be incorporated to a greater extent (8.5 [8–10]). There was no demographic group that found SBE to be more beneficial than others (Table 3).

Table 3:

Comparison between student perceived utility ratings of SBE according to demographic characteristics

	Please rate how useful you find simulation on a scale from 0 to 10. Median score [IQR]	p-Value
Gender
Female	9 [8–10]	0.5*
Male	9 [8–9]
Year Level
Year 1	8 [7.5–9]	0.08†
Year 2A	9 [7.5–10]
Year 3B	9 [8–10]
Year 4C	9 [8–9]
Year 5D	9 [8–10]
Undergraduate or graduate
Undergraduate	9 [8–10]	0.6*
Graduate	9 [8–10]
Metropolitan or rural
Metropolitan	9 [8–10]	0.8*
Rural	9 [8–10]

Notes: p-Value represents comparison of rating scores within each demographic category (e.g. median score of male versus female participants).

*Mann–Whitney U-test.

^†Kruskal–Wallis test

Discussion

SBE is an established and effective component of the modern medical curriculum. When utilized as a supplement to patient interactions, SBE provides a safe environment by which learners can acquire new skills or achieve greater competency [9,24]. While participant feedback regarding particular SBE interventions is published, it is uncommon for cohort-wide perspectives to be sought [11–18,25]. This is to the disadvantage of learners and educators, alike, as informed student feedback provides valuable information in order to implement specific initiatives and prioritize learner-centred needs.

This study aimed to gain insight into medical students’ exposure to and attitudes towards SBE as a means of identifying potential areas of improvement, and to facilitate further assessment and intervention. Two previous studies, Joseph et al. [20] and Agha et al. [21], have surveyed medical students’ perspectives on SBE, though only one recruited a similar sample size to our study. Our study is unique because it is the first to investigate broader student perspectives in a Western educational setting and also in that it assesses negative emotional reactions to SBE. Although our study was conducted at a single university, unlike the previous studies, our cohort spans multiple year levels and includes domestic and international undergraduate and graduate-entry students from metropolitan and rural clinical schools. This is also unique in comparison to previous literature.

Agha et al. [21] performed a cross-sectional survey of 115 Saudi Arabian medical students, using a 5-point Likert scale to investigate their satisfaction with SBE. They found that despite challenges such as lack of staff/facility availability, instructor knowledge, and student cooperation the majority of students were satisfied with SBE (85%, 98/115) and felt that it improved their knowledge and skills. Similarly, Joseph et al.’s [20] cross-sectional survey of 247 Indian medical students reflected a general (though to a lesser degree) favour toward SBE (72.5%, 179/247), whereby SBE was viewed as a positive means of creating a varied learning experience and addressing the perceived problem of limited student access to patient encounters.

Figure 1:

Students’ responses to the statement ‘simulation has helped me to improve my…’. Students were asked to select all options that applied

Figure 2:

Students’ responses to how often they have had negative experiences during SBE using a 5-point Likert scale (0 – Never, 4 – Always). Results are presented as: ‘Yes’ – Likert scale 0–1; ‘Sometimes’ – Likert scale 2; ‘No’ – Likert scale 3–4

Our study demonstrated a high proportion of participants who favoured SBE as a useful component of medical education (97.1%), with 90% believing that SBE directly improved their learning. One potential factor reflecting the high satisfaction rate in our survey is that the majority of participants (99.2%) had previously participated in SBE, compared to 67% of Agha et al.’s [21] cohort.

An indirect measure of students finding benefit in SBE is that 86.4% of students experienced SBE in an extra-curricular environment, 43.2% having paid to do so. This highlights the fact that students desire and actively seek out opportunities for SBE as a means of further education and advancement. It should be noted, however, that we do not advocate for students being unduly charged for SBE experiences.

Figure 3:

Students’ responses assessing medical student perception and attitudes towards SBE (11-point Likert scale)

Although debriefing is common and a well-recognised component of SBE [22], (pre-)briefing is also essential for reflective learning. In our study, approximately one-third of participants respectively experienced pre-briefing ‘most of the time,’ ‘about half of the time,’ or ‘rarely.’ The reason for this is beyond the scope of our findings the reason for this and will require further investigation with qualitative analysis. We argue that pre-briefing should occur more often, as even a four-minute preparatory briefing before a simulation can statistically improve clinical performance and cognitive appraisal ratios (perceived resources versus perceived demands) for learners [26]. Briefing is also reported to improve the learning experience for even the observer, promoting them from a passive to an active participant (and thereby learner) in the simulation [27]. Notably, students perceive the importance of preparation for SBE, with three-quarters of our sample reporting that pre-reading is a beneficial component of their preparation for SBE. It is not clear from our study whether pre-reading achieves objective improvements in performance, or rather a greater sense of comfort via an improved cognitive appraisal ratio; this should be investigated further.

In a workplace culture that places value on confidence and technical ability, the role of SBE as a means to encourage learning through mistakes and reflection has the potential to create negative experiences if not implemented appropriately [28]. It is unsurprising that SBE sometimes triggers an emotional response from the learner and although this can often be beneficial, this is only to a certain point where negative emotion can reduce the capacity of one’s working memory [29]. A quarter of participants in our study (23.5%) reported feelings of anxiety associated with SBE and one-fifth (20.8%) had reported a previous negative experience during SBE. Despite this, only 1.5% ever reported having felt unsafe (albeit this number should be reduced further). It must be noted that the exact definition of ‘unsafe’ may be interpreted differently by students and educators, thus requiring further qualitative analysis to in order to better understand the intricacies of this statement. While it is clear that there is room for improvement, reassuringly over three-quarters of students reported looking forward to participating in SBE. Our findings are in keeping with qualitative analyses regarding students’ anxiety to perform in SBE [30]. In SBE environments, students may find learning enjoyable even if they experience a degree of negative emotion associated with it, described by Keskitalo as ‘hard fun’ [31]. Ng et al. [28] state that faculties are essential in creating a safe environment where mistakes can be made by learners without having to ‘constantly save face and appear certain’ – reassuringly <1% of our participants ever felt criticized by instructors or peers. While the Australian Medical Council (AMC) offers guidance regarding support services for students in need, they do not explicitly state that a student should feel safe in their learning environment [32]. Although most students in our survey reported feeling safe, perhaps the proportion of students reporting negative experiences or anxiety could be better addressed with specific initiatives if an element of safety were to be included in AMC Standards.

This single-centre study provides a broad insight regarding the current medical student experiences of SBE. Whilst we recognize that the goal of improving learner outcomes should be improved by implementation and evaluation of specific interventions, the benefit of our study is that by understanding and appealing to the experiences of the general student cohort, new interventions can be better tailored to address gaps within the current student experience and, thus, improve learner outcomes. For example, our findings suggest that while students see the benefit of SBE and wish to engage with it further, new interventions could include an element of formal (pre-)briefing given the disproportionate perception of high value versus limited participation. Given our broad sample and high rates of survey completion, and high Cronbach’s alpha values for internal consistency, we believe our results to be a valid representation of our population. However serial assessments over time and among different cohorts are needed to assess the reliability of our findings.

One key finding of our study to highlight is regarding the negative experiences of SBE. While we recognize that the majority of students enjoy SBE, no single experience will appeal to 100% of a cohort, and that student ‘enjoyment’ is not the ultimate goal of SBE, our findings suggest that there is a cohort of students who feel anxious and vulnerable while participating in SBE – this must be addressed. Given that these experiences were present in a population that reported higher levels of satisfaction than previous studies, one could infer that such negative experiences are not limited to our cohort, though being a single-institutional study, this could not be generalized. We, therefore, recommend that further analysis across multiple centres be conducted in order to better understand the root causes of these negative experiences, whether negative experiences have objective impacts on SBE learning outcomes, and whether focussed initiatives can be implemented to address this.

Our study is limited in generalizability given that it is a single centre (despite capturing varied educational programs within this). In addition, given that the survey was conducted using a convenience sampling method over a 21-month period, it is likely that students’ exposure to SBE could have changed. However, recruiting students at various stages across the academic year would likely reduce the effect of certain periods being associated with higher/lower likelihoods of SBE exposure.

Given that this study has been developed by our research team in order to answer specific questions, it has not yet been validated across a greater cohort.

Conclusions

SBE is a highly valued and beneficial aspect of medical education across many settings and all year levels. In addition, students have ample access, experience and provision of SBE. Students stated a preference for SBE to support the development of ‘technical’ skills. While the majority of students look forward to SBE and wish for it to be incorporated to a greater degree in the medical school curriculum, a proportion of students have negative experiences which should be further assessed and addressed. We encourage medical education providers to actively seek feedback regarding student experiences of SBE. By analysis of students’ feedback regarding experiences, opportunities and areas for improvement, targeted areas for further research and focussed initiatives (from policy to individual practice) can be implemented to further improve the efficacy, participation and enjoyment of learner-centred SBE.