This article describes introduction of a comprehensive laparoscopic simulation education program to enhance operative efficiency in gynaecological procedures. We describe our approach to educational design, clinical integration and evaluation using an outcome logic model.
The study was conducted at Gold Coast Hospital Health Service (GCHHS) following the purchase of a laparoscopic virtual reality (VR) laparoscopic simulator (LAPSIM® VR), and development of a training program for obstetrics and gynaecology (O&G) trainees. In 2021, a surgical laparoscopic credentialling simulation program was introduced, requiring trainees to achieve a ‘pass’ on the LAPSIM® VR simulator prior to operating on patients. The evaluation approach used an outcome logic model to document program objectives and outcomes. Data analysis involved multivariate linear regression to ascertain the impact of the laparoscopic simulation program on procedure length in the operating theatre.
Inputs included faculty and trainee time, space, LAPSIM® VR equipment and take-home laparoscopic box trainers. Activities involved online instructional modules, training and surgical learning sessions. Outputs included trainee utilization of LAPSIM® VR and credentialling. Outcomes included surveys, retrospective reviews of patient laparoscopic salpingectomies and primary surgeon operator rates. LAPSIM® VR credentialling was performed by 81% of the GCHHS gynaecology registrar cohort in 2021. Trainees completed 234 VR salpingectomies. Introduction of the LAPSIM® VR program was associated with a significant reduction in mean operative time for all ectopic pregnancies in all primary surgeon groups. For uncomplicated ectopic procedures, there was a reduction in operative time of 14 minutes between 2020 and 2021 (p = 0.001, 95% CI: 9–19 minutes) after adjusting for instrument and surgeon level. Credentialled trainees were no more likely to be a primary operator than those who were not credentialled. Survey results revealed that trainees felt the LAPSIM® VR program improved their technical skills, but that other factors influenced their likelihood of being primary operator.
Introduction of a laparoscopic simulation VR credentialling program resulted in a significant reduction in operative time for laparoscopic salpingectomies in our institution. Educational programs should aspire to translational, patient-focused outcomes in their design and delivery.
Operating theatre time is a costly resource, requiring efficient but safe surgical proceduralists. Training on laparoscopic simulators has potential to improve surgeons’ efficiency, but the translation of these benefits to clinical outcomes is often not realized [1]. This article describes introduction of a comprehensive laparoscopic simulation education program aiming to enhance operative efficiency in gynaecological procedures at a tertiary health service. We describe our approach to educational design, clinical integration and evaluation using an outcome logic model.
Laparoscopic simulators have been demonstrated to be effective in training and assessing the skills of trainee surgeons in a teaching and training environment [2–6]. Virtual reality (VR) simulators, which offer the ability for trainees to rehearse entire surgical procedures, may be more efficient in skill development than simulators offering rehearsal of basic skills alone [7]. The Royal Australia and New Zealand College of Obstetricians Gynaecologists (RANZCOG) has now mandated that all trainee gynaecologists have access to box trainer laparoscopic simulators at their workplace [8]. However, simulation equipment such as VR laparoscopic simulators is an expensive investment and may not be accessible to all trainees. Even when VR training opportunities are provided, there is limited evidence of impact and transferability of skills to the operating room [1]. Training institutions are left with the dilemma of how to best integrate and utilize these education simulation tools.
Numerous strategies have been attempted to incentivize trainee engagement in laparoscopic simulation programs, such as providing portable take-home simulators, offering personalized feedback on performance, issuing credentialling certificates and utilizing online modules [8]. Despite these efforts, the level of engagement in many laparoscopic simulation programs remains low [8–10]. One potential solution may lie in credentialling junior surgeons based on skill assessments conducted on laparoscopic simulators. While laparoscopic simulation programs have primarily focused on training with the expectation of subsequent performance improvement, incorporating proficiency testing into the process could yield better results. By utilizing objective skill assessments, supervising surgeons could make more informed decisions regarding the appropriateness of trainees assuming primary operating roles.
Evaluation of training programs through the lens of clinical outcomes is challenging, but critical if our simulation is to have a translational impact [11]. Direct, causal links between educational programs and clinical outcomes are hard to establish. An outcome logic model may be an alternative approach to evaluation [12]. This approach depicts the path from program creation to implementation to outcomes and allows a comparison among the resources required for a program, the activities of the program and the results or outcomes [11,13,14].
We theorized that the promise of laparoscopic VR simulation training might be realized through closer integration of educational strategies with real-world operating theatre processes, including credentialling and supervision, and with evaluation through the lens of clinical outcomes. We introduced a comprehensive laparoscopic simulation education program to our institution with the primary goal of improving operating room theatre efficiency, and evaluated the program using an outcome logic model.
Gold Coast Hospital Health Service (GCHHS) is a tertiary hospital located on the Gold Coast, Queensland, Australia. The hospital performs several thousand laparoscopic procedures annually spanning the specialities of surgery, gynaecology, urology and paediatric surgery. All laparoscopic surgical procedures are either performed by or assisted by training surgeons. In 2021, GCHHS employed 21 full-time general obstetrics and gynaecology (O&G) trainees at variable stages of training. For our data collection and analysis, we divided the trainees into three groups.
Senior trainees (STP) (n = 8): O&G trainees who had already completed the basic 4-year RANZCOG integrated training program. Competency at level 3 laparoscopic operative skill is required for STP trainees. Definitions for laparoscopic skill procedural levels have been outlined in the joint RANZCOG and Australasia Gynaecological Endoscopy and Surgery (AGES) guideline for performing gynaecological endoscopic procedures [15].
Intermediate trainees (ITP) (n = 5): O&G trainees who were currently completing the basic 4-year RANZCOG integrated training program. Surgical exposure and skill in this cohort is variable from years 1–4 with most trainees working on developing level 1–3 laparoscopic operative skill competency.
Junior trainees (PHO) (n = 8): O&G trainees who were not currently on the RANZCOG training program and largely performing a service provision role. These trainees are not rostered to elective gynaecology operative opportunities. They do, however, have variable exposure to emergency gynaecology operative cases. Most PHOs are aiming for future RANZCOG program selection. None of these registrars are competent to independently perform laparoscopic procedures.
Trainers were specialist obstetrician gynaecologist who had completed fellowship training with RANZCOG. There was no additional trainer selection criteria or training provided as part of this program; however, all trainers regularly supervise and mentor RANZCOG trainees. Traditionally, the decision for trainers to allow trainees to be primary operating is ad hoc. Factors that influence trainee primary operating include trainees’ knowledge of the procedure, demonstrated previous proficiency in theatre cases, willingness of the surgical trainer to supervise and theatre time availability. Thresholds for allowing trainees to be primary operators varied in the trainer cohort. Program lead gynaecologist has completed a simulation fellowship and works fractionally in a medical education role.
The hospital has an active simulation service with a focus on translational simulation. O&G trainees were regularly involved in simulation education including maternity emergency courses, multidisciplinary maternity simulations in the clinical environment and translational simulation interdepartmental opportunities including after-hours emergency, emergency medicine, trauma, anaesthetic and operative theatre teams.
A comprehensive laparoscopic simulation program was introduced to the GCHHS in 2021 for O&G trainees. This program included the requirement for gynaecology trainees to complete credentialling on the LAPSIM® VR before being the primary operator during laparoscopic gynaecological surgeries. Involvement in the take-home laparoscopic simulation program was offered after VR laparoscopic credentialling was completed. Paid, protected and supervised time was provided to practice laparoscopic simulation at work.
An outcome logic model was used to document the extent to which the simulation program attained its objectives [11]. An outcome logic model depicts the path from program creation to implementation to outcomes for participants and the institutions in which they learn/work [11,13,14]. The program objectives were selected by the research team as those that were predictive of desired feasibility and educational outcomes. To select the components for the model we took a pragmatic approach based on work previously presented and published on laparoscopic simulation programs. This informed our decision to incorporate components such as take-home laparoscopic simulation trainers, credentialling, approach to teaching sessions and outcome measures. A detailed outline is covered in Table 1.
| Inputs: What resources were dedicated to or consumed by the program? | |
| ● Faculty time | |
| ● Trainee time | |
| ● Facility space | |
| ● Data storage on LAPSIM® VR | |
| ● Non-consumed equipment | |
| ○ LAPSIM® VR | |
| ○ LAPSIM® VR operative software | |
| ○ Take-home laparoscopic box trainers | |
| ○ Instruments for take-home lap trainers | |
| ● Consumed equipment | |
| ○ Exercises for take-home lap trainers – EoSIM program | |
| Activities: What does the program do with the inputs to fulfill its mission? | |
| ● Development of online LAPSIM® instructional online learning module | |
| ● Delivery of weekly LAPSIM® training sessions | |
| ● Delivery of monthly surgical active learning sessions | |
| ● Design and promotion of trainee LAPSIM VR credentialling | |
| ● Design and delivery of LAPSIM® take-home lap box trainer program | |
| Outputs: What are the direct outputs of the program activities? | |
| ● Trainee use and overall time spent on the LAPSIM® VR | |
| ● Salpingectomy procedural number on the LAPSIM® VR | |
| ● Trainee attainment of LAPSIM® VR credentialling | |
| Outcomes: What are the immediate and intermediate benefits for the trainees and institution during and after the program? | |
| ● Survey of trainees and trainers | |
| ● Retrospective review of patient laparoscopic salpingectomies – theatre time | |
| ● Retrospective review of patient laparoscopic salpingectomies – primary operator rates |
(i) Data on inputs and activities within the program were descriptive, based on the program design and records of sessions undertaken.
(ii) Outputs included trainee use of the LAPSIM® VR. This was recorded under individual trainee login and included number of procedures performed, time spent on the LAPSIM® VR and detailed metrics including laparoscopic salpingectomy credentialling.
(iii) Outcomes included an anonymous survey distributed to all trainees and surgical trainers at the completion of the 2021 training year (survey instruments shown in Supplementary Material 2 and 3. Retrospective review and analysis were performed on all laparoscopic salpingectomies performed at GCHHS from 2020 to 2021.
Data were collected and analysed for laparoscopic salpingectomies 12 months before and after the initiation of the program. Individual electronic medical records and operative notes were reviewed for all cases by primary author (consultant obstetrician gynaecologist). Stata/IC v16.1 statistical software was used for statistical analysis [16]. Descriptive and summary statistics were produced. A least-squares linear regression model was used with the length of the procedure (minutes) as the outcome variable and the year of the procedure (2020 or 2021) as the exposure variable. The 2020 data were prior to the institution of the LAPSIM® VR program. The variables that were assessed for confounding included procedures classified as complicated (yes or no), surgeon level (consultant gynaecologist, STP, ITP or PHO) and the instrument used for the salpingectomy (loop PDS, Ligasure™ or bipolar and scissors). The results were then adjusted for those variables considered confounders. Sensitivity analyses were completed. The analysis was also completed excluding all complicated procedures. Assumptions were assessed using scatter, normal quantile, and residual versus fitted plots. Influential observations were evaluated.
This project was reviewed by the ethics committee at Gold Coast University hospital and an ethical waiver granted – project LNR/2020/QGC/67056.
Inputs for the program included equipment, space and instructional design.
Equipment was purchased with the support of a local health service innovation grant of $250,000 AUD. This included installation of a LAPSIM® VR machine, software to support VR operating including modules for basic training exercises, salpingectomy, hysterectomy, appendectomy, cholecystectomy and construction of 40 take-home simple laparoscopic simulation boxes. The take-home box trainers increased accessibility for trainee access to simulation equipment. The LAPSIM® VR machine was in the simulation centre and trainees were given passcode access able to access this equipment at any time.
Faculty support for the program included a dedicated simulation technician to support the equipment and operational elements of the program, and a gynaecologist to lead development, teaching and training associated with the program. This support was under the governance of the GCHHS Simulation Service.
Instructional design was guided by a mastery learning approach [17]. This methodology was utilized as there was large variation in prior surgical skill experience in the cohort. We trialled several approaches to incentivize trainees to engage in the LAPSIM® VR program. This included only allowing access to take-home lap trainers once trainees had already completed LAPSIM® VR salpingectomy credentialling. Other measures we used included supporting and supervising laparoscopic simulation education during work hours and providing access to an online module with instructional demonstration videos. During teaching sessions, we also used gamification and test-enhanced learning in efforts to increase engagement in the program.
Resources were developed to guide trainees with LAPSIM® VR use and take-home laparoscopic box trainer program. The module was developed using Rise® and is available as a live link (Supplementary Material).
Teaching sessions allowed trainees paid and dedicated time for LAPSIM® VR training as part of the mandated O&G trainee RANZCOG training teaching time. These sessions occurred each week for 30 minutes.
Active learning surgical sessions ran monthly with a focus on a core surgical procedural skill. Each of these sessions was 4 hours and included a 60-minute laparoscopic simulation component. These rostered educational hours were part of the RANZCOG-mandated minimum trainee teaching rostered hours.
The LAPSIM® VR provides real-time feedback on time, instrument path length and use, bleeding, injury to surrounding structures and use of diathermy. Trainees can also elect to video replay their performance to further encourage self-reflection. A ‘pass’ was achieved when trainees’ safety performed a right laparoscopic salpingectomy meeting proficiency criterion previously established by experts. Metrics used to define a ‘pass’ were the same as Larsen’s BMJ landmark paper [18]. LAPSIM® VR was made accessible to all gynaecology trainees on site 24/7.
Participation in this program was only accessible to trainees who had completed LAPSIM® VR salpingectomy credentialling. The aim of this program was to develop proficiency in laparoscopic suturing.
During 2021 there were 234 VR salpingectomies performed collectively by trainees on the LAPSIM® VR. This represents over three times the number of actual laparoscopic salpingectomies performed at GCHHS for ectopic pregnancy in 2021. Time spent on the LAPSIM® VR as a collective registrar group performing laparoscopic salpingectomies throughout 2021 was 537 minutes, with a mean of 21 minutes and of range 7–93 minutes.
There was great variability on the number of attempts to achieve a ‘pass’ credentialling. Many trainees stopped using the VR once as ‘pass’ had been achieved. Some, however, continued to practice beyond credentialling. ITP trainees spent the longest period of time performing VR laparoscopic salpingectomies and performed the greatest number of procedures. They were also most likely to continue to practice beyond being credentialled with a ‘pass’. Most trainees achieved a credentialled ‘pass’ with only a minimum time investment of 30 minutes or less using the VR simulator. Time spent using the VR and number of procedures performed is shown in Figure 1.
Overall, 81% of all full-time general O&G trainees employed for a full 12 months at GCHHS in 2021 achieved VR laparoscopic simulation credentialling. All eight STP and five ITP achieved credentialling on the LAPSIM® VR (100%). In the PHO group 4/8 achieved credentialling (50%). One PHO did attempt credentialling multiple times on the VR but was not able to achieve a ‘pass’. The remaining three PHOs did not use the VR at all during the 12-month period. All PHO laparoscopic salpingectomies performed in real patient cases in 2021 were done by the PHO group that had completed VR lap SIM credentialling.
Completed surveys were performed by eight trainees and eight trainers. Common perceptions are described here.
(i) Improvement in skills
Generally, the trainees thought that the practice they had performed on the LAPSIM® VR translated well to the real cases. When asked what they gained during the sessions trainees reflected on both technical skills and on team bonding. Interestingly many commented on the laparoscopic SIM sessions being an important opportunity to touch base with colleagues and support trainee team morale.
Hand eye coordination skills, camaraderie with colleagues (Trainee 1)
Very useful-particularly suturing and instrument use (Trainee 2)
(ii) Impact on supervision in the operating theatre
Only one trainee reported that trainers always asked if they had completed LAPSIM® VR credentialling. The remainder stated that they were never or rarely asked. There was variable opinion in the trainee cohort whether completion of VR salpingectomy credentialling made any impact on trainer decision to allow trainee primary operating in real salpingectomy cases.
Really well - some bosses asked if I had done the training before doing my own ectopic surgery (Trainee 4)
No correlation it seems. I never had a boss ask me what amount of simulation training I had before deciding whether or not I could be primary operator in a lap surgery (Trainee 5)
When asked whether they would allow a trainee to be primary operator if they had completed the LAPSIM® VR salpingectomy credentialling, six trainers agreed they would in principle support this.
Yes, but I probably would have anyway. If we start making this a requirement it will be a good carrot to do the lap sim (Trainer 5)
Trainers wanted to assess and appreciate themselves what the trainees were capable of rather than relying on certificates of competency from the laparoscopic simulation programs.
I talk to them about their previous laparoscopic experience and how they would aim to perform this operation (Trainer 1)
They are on the program and so there to be taught, or they are a promising PHO and deserve an opportunity (Trainer 2)
When asked if the LAPSIM® VR had any appreciable benefits or drawbacks to laparoscopic operating five trainers felt there was not yet discernible benefits or drawbacks observed. Only one trainer reported that registrars appeared to have improved dexterity after initiation of the program.
There were 149 laparoscopic salpingectomies performed for ectopic pregnancy at GCHHS in 2020–2021. Table 2 reports numbers of procedures per year and complicated versus uncomplicated cases. Uncomplicated procedures included cases limited to a unilateral salpingectomy for a tubal ectopic pregnancy only. Complex cases included procedures requiring adhesiolysis and complicated ectopic pregnancies such as cornual, ovarian or heterotopic ectopic pregnancies. There were also cases where a decision was made to perform excision of endometriosis or further surgical procedures such as bilateral salpingectomy during the emergency operative procedure.
| Total | Uncomplicated | Complicated | |
|---|---|---|---|
| 2020 | 70 | 52 | 18 |
| 2021 | 79 | 57 | 22 |
There were 109 cases of laparoscopic salpingectomy for uncomplicated ectopic pregnancy across the 2-year period. In 2020 the mean time for an uncomplicated salpingectomy for ectopic pregnancy was 52 minutes (range 12–88 minutes) versus 44 minutes in 2021 (range 16–96). For uncomplicated procedures, there was a reduction in operating time of 14 minutes between 2020 and 2021 (p < 0.001, 95% CI: 9–19 minutes) after adjusting for instrument and surgeon level. Operative time was recorded by a theatre nurse at the start and end of each surgical procedure on the electronic medical record and did not include anaesthetic induction, anaesthetic recovery or transfer in and out of the operating theatre.
In 2021 compared with 2020 all surgical groups O&G trainers, STP trainees, ITP trainees and PHO trainees had an improvement in mean operative time to perform uncomplicated and complicated salpingectomies as shown in Figure 2.
Overall LAPSIM® VR statistics based on trainee surgeon level
When all ectopic pregnancies were considered including cases with added complexity – there was improvement in times across the cohort. After adjusting for complicated procedures and surgical technique there was evidence (p = 0.005) of a reduction in operating time between 2020 and 2021 (difference of 7 minutes, 95% CI: 2–12 minutes).
Trainer gynaecologists did the majority of primary operating in both year cohorts as shown in Figure 3. Overall, across the 2-year period 81/149 (54.3%) cases had a consultant gynaecologist primary operator. In 2020 35 of the 70 cases (50%) were performed by consultants. In 2021 46 of 79 cases (58%) were performed by consultants. Surgeon level was adjusted for in the statical analysis of time for laparoscopic salpingectomies.
Surgical procedural time – uncomplicated salpingectomy per surgeon level
Overall salpingectomy number per surgical seniority
Enhancing the efficiency of operating theatres requires a comprehensive approach beyond the mere acquisition of VR simulators to train surgeons. Simply purchasing simulation equipment does not guarantee improved trainee education or patient outcomes [1,8]. Achieving translational results necessitates careful consideration of the educational frameworks that support the effective use of simulation tools in training, and the integration of those frameworks within the realities of clinical practice. We focused on a specific surgical procedure taught using a mastery learning approach and assessed as part of a credentialling process. Our integrated, multifaceted LAPSIM® VR program was associated with quicker procedure times in the operating theatre, illustrating the potential for educational programs to improve clinical outcomes.
A core tenet of our program was credentialling – requiring trainees to achieve a pre-specified level of confidence prior to being granted primary operator status in real surgical cases. This was based on the assumption that this would lead to safer, faster procedures performed either by credentialled trainees or by their supervising surgeons if the trainees were not yet at the demonstrated level. Credentialling provides trainees with a clear goal and well-defined metrics to strive for. While many programs typically view competence credentialling as an endpoint, we challenged this approach by making it the initial step and primary goal. Whilst most trainees in our program achieved credentialling, there were significant number who did not. Whether this was an issue with training methods or learner motivation is unknown. Notably, the only junior PHO trainees who were primary operators for salpingectomies were those who had completed the credentialling process. Trainees were not forbidden to perform surgical cases as primary operator if they had not completed credentialling as ultimately the discretion of this decision was left to the surgical trainer.
Trainer trust of credentialling appeared to be problematic. Despite many trainers supporting the notion of primary operating after laparoscopic credentialling in surveys this didn’t translate to increased trainee operating. These findings are not isolated to our unit and echoed in research completed by Blackhall et al. [19] Primary operating within our trainer group increased after introduction of trainee credentialling. Determination of trainee ability to be primary operator appeared to rely on preoperative discussion between the trainer and trainee on how many procedures they had previously performed, surgical experience, steps to the operation and comfort level performing the surgery. Most trainees reported no correlation between completing laparoscopic credentialling and ability to be primary operator. This raises an important question – how do we engage our trainers to trust laparoscopic simulation credentialling?
There are nuances in understanding causal relationships within any complex service or educational intervention [20]. Using an outcome logic model may be a more useful approach to comprehensively evaluate simulation educational programs than traditional approaches that seek linear causality [11–14]. Specifically, it affords examination of questions beyond effectiveness of interventions, i.e. feasibility, efficiency, return on investment and more. Our work highlights the need for a greater degree of intentionality in educational program design when it is designed to operate within, and impact upon a health service. The connection between an outcome logic model and translational simulation lies in their shared focus on understanding and improving complex systems, particularly in the realm of program planning, implementation and evaluation [21]. The outcome logic model serves as a framework for comprehending how various components of a program can contribute to the desired outcomes. While the conceptual framework is provided by the outcome logic model translational simulation offers the necessary resources and toolkit for effective implementation.
Our findings present a promising outlook for educational programs aimed at enhancing operating theatre efficiency. To fully harness the potential of this program at our institution collaboration with other surgical specialities is imperative. The opportunity to optimize operating theatre efficiency may be even more significant with procedures such as laparoscopic appendectomy or cholecystectomy given the higher surgical operative volumes of these procedures. Additionally, further research is necessary to explore the factors influencing trainers’ decisions to allow trainee primary operating, barriers to trainee engagement in laparoscopic simulation credentialling, how often credentialling should be performed and explore trainer trust of simulated credentialling.
Our cohort was relatively small, and our review only covered a 12-month period before and after the LAPSIM® VR program intervention which may limit the generalizability of our findings. Our focus was primarily on the clinical measure of time to complete laparoscopic salpingectomies, which may not fully capture the broader impact of the program on other surgical procedures or patient outcomes. We evaluated procedural time as an indicator of faster and more efficient surgical procedures; however, we acknowledge that a faster procedure is not always the safest approach. There may also have been other unexplored factors that contributed to the observed reduction in mean operative time, which were not accounted for in our study.
Introduction of a laparoscopic simulation VR credentialling program resulted in a reduction in operative time for laparoscopic salpingectomies in our institution. Educational programs should aspire to translational, patient-focused outcomes in their design and delivery. Credentialling may be a useful strategy to support clear and transparent communication in surgical goal setting, competence assessment, and directed feedback. Adopting an outcome logic model offers a promising approach to evaluate and complement translational simulation education programs more comprehensively.
BL designed the manuscript concept and contributed to data collection, writing and revising the manuscript. AW provided statistical interpretation and support and contributed to revising the manuscript. JM contributed to data collection, initial manuscript drafts and revisions. VB provided overall project guidance and support and also contributed to writing and revising the manuscript.
This article describes introduction of a comprehensive laparoscopic simulation education program to enhance operative efficiency in gynaecological procedures. Equipment was purchased with the support of a local health service innovation grant of $250,000. This included installation of a LAPSIM® VR machine, software to support VR operating including modules for basic training exercises, salpingectomy, hysterectomy, appendectomy, cholecystectomy and construction of 40 take-home simple laparoscopic simulation boxes. There was no additional funding outside of usual employment arrangements for the authors.
The data sets used and or analysed during the current study are available from the corresponding author on reasonable request.
This project was reviewed by the Ethics Committee at Bond University and an ethical waiver granted – project LNR/2020/QGC/67056.
All authors are all currently employed by GCHHS. BL and VB are also employed by Bond University. All authors have no other competing interests to declare.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
Virtual reality laparoscopic simulation for operating theatre efficiency: an outcome logic model program evaluation
Facebook
Twitter
Linkedin
Whatsapp
Tweets