International Journal of Healthcare Simulation

Introduction

One in four individuals will experience an issue with their mental health each year in the UK. Consequently, it is estimated that 16 million people suffer from poor mental health nationally at any given time ^[1]. Individuals who experience serious mental health issues are known to have a shorter life expectancy, experience more physical illnesses and receive worse medical care than those in the general population ^[2]. Therefore, high-quality evidence-based training for healthcare workers is crucial to provide the best possible care and tackle these health inequalities.

Simulation-based education has historically been utilized in healthcare-based training, with systematic reviews demonstrating its positive impact on the medical profession ^[3,4]. Typically, manikins – models of humans that are used to represent patients – are used in order for trainees to practice physical health skills and procedures ^[5]. High-fidelity simulation in mental health care, however, is a type of simulation training that takes an alternative approach to healthcare education. High-fidelity simulation involves the use of simulated patients (SPs) experiencing symptoms of mental health conditions. This is conducted in an artificial but carefully constructed environment, followed by a detailed debrief to consolidate learning. Scenarios used are highly flexible, as SPs follow a loose narrative and have the ability to adjust their stories and presentation to match the needs of the participant ^[6]. Moreover, the opportunity to repeat scenarios allows for continuous rehearsal, providing participants with the potential to try new techniques where previous techniques fail, and thus further engaging with the experiential learning cycle. This rehearsal allows participants to more easily recall cognitive processes, resulting in increased confidence ^[7]. There is mounting evidence to suggest the beneficial effects of simulation as a form of training, both for psychological and physical skills, through engaging in pedagogical models such as experiential and reflective learning ^[8–10]. Simulation training has been found to be effective in addressing personal resilience, self-efficacy, self-reflection, wellbeing and clinical skills in medical and healthcare settings ^[11–13].

However, training in mental health care can only be as effective as the participant’s ability to transfer what they have learned into their workplace when evaluating efficacy at an organizational level ^[14]. Drawing on social-cognitive theories ^[15], one’s cognition is thought to be crucial in learning, with self-efficacy playing a key role in learning and knowledge transference ^[16]. Self-efficacy refers to the belief that one is able to successfully perform a task ^[17]. Bandura suggested that high self-efficacy leads to more goal setting, better performance in training, and higher motivation to transfer learned skills into the workplace ^[15]. Succeeding research has affirmed this, finding higher confidence ratings and skill transfer motivation ^[18]. In the context of healthcare, Schindel et al ^[19] conducted a Randomized Control Trial (RCT) to investigate the effects of enhancing self-efficacy in healthcare staff on outcomes in dementia care for patients. The authors found that following enhancement, patients reported a significant positive impact, including the implementation of person-centered approaches to work. While self-efficacy is not a direct measure of clinical training outcomes, it can provide valuable insight into knowledge transference which is directly linked to task performance, and thus merits research.

High-fidelity simulation training for mental health care has been implemented within a variety of settings and outcomes, notably in nursing training. For example, training courses have been developed to address clinical skills, anxiety and assumptions for nurses in mental health wards ^[20,21], and confidence, knowledge and communication for nurses working with individuals at risk of suicide ^[22]. However, to the authors’ knowledge there has been no large-scale evaluation of its impact on self-efficacy in trainees, or the potential effects that age and gender may have on subsequent findings. Further, it is possible that there are differential effects across sub-groups of trainees. The relationship between age and self-efficacy has been well studied, and results suggest that findings are heterogenous: some studies have shown reduced self-efficacy in older trainees ^{[23, 24]}, whereas others have shown that self-efficacy only becomes lower in older trainees if the training context is perceived as being a ‘younger learners’ domain’ ^[25]. Research has also conversely shown that years of experience in a job role can lead to higher self-efficacy ^[26]. Therefore, it is possible that differences in outcomes relating to age may be mediated by career stage, as the majority of students in the UK are under 30 ^[27]. With a broad range of clinical staff at the frontline of mental health, from trainees to senior clinicians, understanding where training is most successful and unsuccessful could allow for tailoring in order to create optimal learning.

Gender has also been indicated as a factor in training and self-efficacy, with self-estimates of one’s own capability being suggested to differ between men and women ^[28]. Additionally, it has been suggested that there are significant differences between males and females in regard to preference of multimodal information presentation ^[29,30]. This is important to note due to females being the overall majority of the entire NHS workforce (77%) ^[31]. As simulation training uses a blend of learning modalities, such as active participation alongside didactic teaching and reflection, differences in self-efficacy may appear when comparing across genders.

The preceding research has limitations when applying findings to simulation training, as only non-simulation training scenarios have been considered in research, such as classroom-based and online video sessions. Indeed, improvement outcome differences between simulation and non-simulation training have been found, with simulation appearing superior in comparison ^[32–34]. The efficacy and application of training in mental health care has a direct impact on patient care and outcomes, and thus understanding what is and is not effective can assist in the development and delivery of high-quality care.

Therefore, this study aims to investigate whether high-fidelity simulation-based training is effective in improving self-efficacy in trainees. Further, the impact of age and gender will be explored. It is hypothesized that there will be improvements in self-efficacy across all participants. It is further hypothesized that, within this improvement, there will be a difference between age groups and gender in self-efficacy, and any differences in age will be mediated by subsequent career stage (student or qualified).

Methods

Results

Participant inclusion

A total of 890 participants attending simulation courses running between February 2018 and August 2019 in South London completed pre- and post-course questionnaires. Participant outliers in pre- and post-course scores were screened using Mahalanobis Distance, which has been shown to be an effective and reliable tool for outlier identification ^[39,40]. There were two degrees of freedom, equating to a critical Chi-Square value of 13.82 at p < .001. Cases (N = 12) were excluded for exceeding the critical value. Incomplete data of the HFSHI pre- and post-data were also excluded (N = 47). Data from participants in the <20 years category were also excluded due to small sample size (N = 2), as this had the potential to impact the validity of the findings ^[41] . Thus, the total sample size for analysis was 829 participants. See Table 1 for a demographic breakdown of participants.

Table 1:

Participant demographics

Profession	Number of participants	Gender (male/female/other)	Career stage (qualified/student/other)
Nurses	362	86 / 269 / 7	276 / 78 / 8
Midwives	23	0 / 23 / 0	22 / 1 / 0
Doctors	269	86 / 178 / 5	262 / 5 / 2
Allied health professionals	81	26 / 52 / 3	77 / 3 / 1
Non-specific clinical	41	12 / 27 / 2	33 / 2 / 6
Non-specific non-clinical	54c	22 / 31 / 1	29 / 5 / 20
Total	Clinical = 776 (93%) Non-clinical = 54 (7%)	232 / 580 / 18	699 / 94 / 37

Levene’s test was used prior to statistical analysis to ensure that the assumption of homogeneity was met, F(1, 822) = 1.831, p = .104.

There was a minimum change of -46 and a maximum of 49 (M = 2.74, SD = 13.6) in HFSHI scores across all participants. A significant difference was found between pre-course scores (M = 95.7, SD = 12.3) and post-course scores (M = 98.3, SD = 15.3), t(828)= -5.802, p < .001, [95% CI = -3.67, -1.83], with a small effect size of d = .20.

Age analysis

A One-Way ANOVA was conducted to investigate the effect of age on HFSHI score differences (see Table 2). A significant effect was found, F(5, 822) = 2.963, p = .012. Post-hoc tests (Tukey HSD) revealed a significant difference between the categories 25–29 and 35–45 (p = .010). All other comparisons were non-significant (p >.05) (Graph 1).

Table 2:

HFSHI scores across age

Age category	N	Mean	Standard deviation	Minimum	Maximum
20–24	66	3.30	13.8	−35	34
25–29	248	0.19	14.6	−48	28
30–34	154	2.62	12.9	−49	38
35–45	183	4.63	14.1	−46	46
46–55	127	4.31	11.0	−31	28
>55	50	4.12	13.1	−37	27

Graph 1:

The mean sum differences in HFSHI scores across all age groups

Mediation analysis

Linear regressions to assess mediation were conducted in line with Baron and Kenny ^[42]. The total effect was significant, with age predicting HFSHI scores, F(1, 828) = 8.145, p = .004, R = .099, R2 = .01. Age also significantly predicted the mediator (career stage), F(1, 791) = 19.493, p < .000, R = .155, R2 = .024.

However, career stage did not significantly predict HFSHI scores, F(1, 791) = .020, p = .889, R= .005, R 2 = .000. As there was no significant relationship found in step 3 of Baron & Kennedy’s model, the direct effect of age on the HFSHI scores was not calculated, as career stage could not act as a mediator. See Figure 1 for coefficient values.

Figure 1:

Mediation analysis coefficients

Discussion

The results presented suggest that simulation-based mental health care training is an effective method of improving self-efficacy. As previously suggested, a significant change in self-efficacy across participants is believed to lead to improved clinical healthcare skills and practice. This study supports the growing evidence based on the effectiveness of simulation training in mental health care as a tool to support professional and workforce development.

Age differences were detected in the results, although significant improvements were found across all age groups. The data showed an increasing trend in age and self-efficacy. Specifically, individuals within the ages of 25–29 showed significantly fewer mean increases in self-efficacy than those between 35–45. The mechanisms underlying these results are not immediately apparent. Self-efficacy relies on an individual’s belief in their ability to perform a task – it is possible that this belief is reflected in one’s age and increases with time. This finding is of particular importance when considering the needs of young professionals entering the mental health workforce, and how a focus on promoting self-efficacy may enhance performance and subsequent care for service users. Training providers may wish to include self-efficacy as a component to improve in training, in order to build the foundations of enhancement throughout one’s career.

Career stage did not seem to account for the changes within age groups. However, it is recognized that career stage is not the perfect proxy for job experience in regard to mental health care training; acute nurses may have less experience and thus more need for training within the mental health care sector than a mental health nurse, for example. Future studies would benefit from measuring job experience in this respect to better understand the relationship between age and self-efficacy within simulation training.

Interestingly, scores on the HFSHI were not found to differ between males and females. This contrasts with Bandura’s theory, and in experimental work regarding preference of learning modality ^[29,30]. However, simulation training is often diverse in both practical and theoretical learning, with participants being able to actively engage in both simulated scenarios and the discussion of the preceding situation in the debriefs after each scene. Simulation training also offers the opportunity for social and experiential learning and reflection which is weaved within these learning styles and modalities, and as previously discussed has been shown to have a positive impact on self-efficacy in learners. It is possible that the variety of learning styles and delivery mode represented in this type of training and learning also have an impact, such as structure, repetition and multimodal information presentation. Indeed, research has shown that, while there may be preferences between the learning styles of males and females in clinical teaching, there remains a dominant overarching preference for multimodal learning that is shared between both groups ^[43]. This preference in turn may influence self-efficacy as one becomes more comfortable and confident in the training environment. It is important to note that gender is more diverse and complex than the binary definitions of ‘male’ and ‘female’. While previous literature has traditionally contrasted these two genders, succeeding literature should recognize the increasing understanding of gender across a spectrum. This may be reflected in an increased effort to gain larger sample sizes of non-binary participants in order to gain a wider representation and understanding of self-efficacy research across genders.

Within this study, there is the possibility of further confounding variables that were not measured in the course questionnaires. For example, certain age groups may be represented more in certain courses, such as older professionals within courses that focus on management or leadership. Depending on the variability of the success and relevance of the course, this may skew results. Further, some participants may have attended simulation courses with SPs before, and thus previous experiences may also affect performance and perceptions of the training. This is important in the context of self-efficacy, as one may have a stronger belief in their capabilities if they have higher confidence in the training itself. Thus, future studies may wish to account for simulation experience when analysing data. Finally, participants were opportunity sampled in that only those who agreed to data collection and completed the HFSHI both pre- and post-training were included in the analysis.

For some participants, scores on the HFSHI were high pre-course and saw a small drop post-course. It is possible that participants overestimated their abilities prior to training, and this was adjusted for after going through the simulated scenarios; indeed, previous research using the HFSHI for simulation training has suggested that learners may rate themselves more highly in pre-course questionnaires due to experience before training ^[10]. However, as this study intended to measure self-efficacy – which is by definition a self-reported understanding of one’s abilities – this overestimation need not be a limitation, but rather may aid our understanding of how participants view themselves and their abilities. Longitudinal follow ups may provide valuable insight into whether any changes or improvements to self-efficacy occur long-term after training.

Due to the heterogeneity of the training courses used within this study, comparing clinical improvements across courses is difficult, and therefore self-efficacy can act as a common measurement. Nevertheless, focusing on self-reported self-efficacy does have its limitations. While self-efficacy is implemented in knowledge transference, it is not a direct measure of clinical improvement. Future studies may wish to compare self-efficacy changes with quantified clinical improvements in order to assess whether there is a direct relationship between the two. It is interesting to note that a recent scoping review revealed that no studies regarding mental health care simulation targeted mental health outcomes in patients ^[44]. Thus, future studies may wish to consider incorporating this measure.

In conclusion, simulation is an interactive training method for mental health care that has shown to be effective in increasing self-efficacy across a variety of topics and professions. This paper adds to the literature surrounding the factors affecting training outcomes for simulation in mental health care. There appears to be no gender discrepancy in the development of self-efficacy via simulation training as a mode of learning. However, there does appear to be an increasing trend when comparing age groups, with a specific disparity between those in their mid-20s and their mid-30s. More research should be conducted to investigate why there may be differences in learning outcomes for different ages with regard to simulation, and to identify any other confounding factors that may influence these results. Such results could in turn inform future course development in order to optimize mental health care across organizations and enhance the quality of patient care.