International Journal of Healthcare Simulation

Introduction

Point-of-care ultrasound (POCUS) has emerged as a valuable tool for guiding the triage and management of a variety of medical situations, including life-threatening acute cardiovascular conditions in the inpatient setting [1–4]. Acute decompensated systolic heart failure, for example, frequently presents with non-specific clinical features that may be missed on physical exam, and chest x-ray (CXR) fails to detect pulmonary oedema in up to 30%–40% of cases [5,6]. In contrast, POCUS is more sensitive at detecting pulmonary oedema than CXR (88% vs. 70%) [6] in addition to being able to demonstrate new and diminished left ventricular (LV) systolic function.

Similarly, a large pericardial effusion can present with non-specific clinical features that may be missed on physical exam and CXR, which can lead to the subsequent development of cardiac tamponade and obstructive shock [7]. Fortunately, POCUS has become a valuable tool in the rapid identification of both pericardial effusion and tamponade [7–9]. First-year (post-graduate year 1, PGY1) Internal Medicine residents are sometimes confronted with patient presentations of acute decompensated systolic heart failure and large pericardial effusion, but they may not immediately recognize these conditions.

There is a need for more research in assessing the effectiveness of simulation training in POCUS. Simulation training offers participants exposure to a wide array of clinical findings. Further, simulation training occurs in a supportive teaching environment, conducted in a time-effective and lower-stakes manner that does not sacrifice patient safety or comfort [10].

The aim of the present study was to utilize a simulation-based educational intervention to improve the knowledge and comfort of PGY1 Internal Medicine residents in using cardiac POCUS to diagnose acute decompensated systolic heart failure and large pericardial effusion.

Methods

This mixed-methods research included 48 PGY1 Internal Medicine Residents receiving POCUS simulation training at an urban academic centre. We assessed participants’ knowledge and comfort with cardiac POCUS both before and immediately after their simulation training, as well as 3 months post-training in order to assess the durability of any increases in knowledge or comfort.

Results

Forty-two participants completed the pre-intervention survey (88% response rate), 48 participants completed the immediate post-intervention survey (100%) and 26 participants completed the 3-month post-intervention survey (54%). Only 22 respondents (52%) had personally performed any kind of POCUS on one of their patients prior to the intervention (Figure 1). On both the immediate post-intervention and the 3-month post-intervention surveys, 100% of the respondents found the POCUS simulation session to be either somewhat helpful (1/48 and 6/26, respectively) or very helpful (47/48 and 20/26, respectively).

Figure 1:

Frequency of respondents’ prior experiences personally performing any kind of POCUS on one of their patients prior to the intervention

Participants’ self-appraised knowledge also increased after the intervention: 31% of respondents (13/42) pre-intervention stated that they knew the general cardiac POCUS findings associated with various shock states (assuming no mixed shock state), compared to 83% (40/48) immediately post-intervention and 73% (19/26) at 3 months post-intervention. A chi-square test of independence revealed these differences to be significant (p < 0.001). Subsequent pairwise chi-square testing demonstrated that the increased self-appraisal from pre- to immediately post-intervention was significant (p < 0.001), as was the increase from pre- to 3 months post-intervention (p < 0.001). The decreased self-appraisal from immediate post- to 3 months post-intervention was not significant (p = 0.295) (Figure 2).

Figure 2:

Percentage of respondents who stated that they knew the general cardiac POCUS findings associated with various shock states (assuming no mixed shock state) at each time point

The mean number of correct responses (out of 6) on the knowledge assessment quiz was 3.9 pre-intervention, 4.9 immediate post-intervention and 4.8 at 3 months post-intervention (Figure 3). One-way ANOVA testing demonstrated a significant difference between these means (p < 0.001). Post-hoc unpaired t-testing demonstrated that the increased number of correct responses from pre- to immediately post-intervention was significant (p < 0.001), as was the increase from pre- to 3 months post-intervention (p = 0.006). The decreased number of correct responses from immediately post- to 3 months post-intervention was not significant (p = 0.724).

Figure 3:

Mean number of correct responses (out of six) for knowledge assessment at each time point (with 95% confidence interval bars)

On each of the three study surveys, participants also rated their comfort with performing four different cardiac POCUS tasks. After responses were dichotomized into ‘comfortable’ and ‘not comfortable’ as described above, chi-square testing for each question demonstrated a significant difference in the percentage of comfortable residents across the three time points (p < 0.001 for each question). Chi-square testing was then used for pairwise time-point comparisons.

For the first comfort question, the percentage of respondents who felt comfortable selecting the correct ultrasound probe and orienting it correctly when obtaining POCUS images increased significantly from pre- to immediate post-intervention (26.2%–87.5%, p < 0.001), and from pre- to 3 months post-intervention (26.2%–73.1%, p < 0.001). The decreased percentage of comfortable respondents from immediate post- to 3 months post-intervention was not significant (87.5%–73.1%, p = 0.120) (Figure 4).

Figure 4:

Percentage of respondents who felt comfortable selecting the correct ultrasound probe and orienting it correctly when obtaining POCUS images at each time point

For the second comfort question, the percentage of respondents who felt comfortable identifying major cardiac structures in the four core cardiac POCUS views increased significantly from pre- to immediate post-intervention (31.0%–93.8%, p < 0.001), and from pre- to 3 months post-intervention (31.0%–76.9%, p = 0.023). The decreased percentage of comfortable respondents from immediate post- to 3 months post-intervention was significant as well (93.8%–76.9%, p = 0.034) (Figure 5).

Figure 5:

Percentage of respondents who felt comfortable identifying major cardiac structures in the four core cardiac POCUS views at each time point

For the third comfort question, the percentage of respondents who felt comfortable identifying decompensated systolic heart failure POCUS findings increased significantly from pre- to immediately post-intervention (16.7%–91.7%, p < 0.001), and from pre- to 3 months post-intervention (16.7%–69.2%, p < 0.001). The decreased percentage of comfortable respondents from immediately post- to 3 months post-intervention was significant as well (91.7%–69.2%, p = 0.012) (Figure 6).

Figure 6:

Percentage of respondents who felt comfortable identifying decompensated systolic heart failure POCUS findings at each time point

For the fourth comfort question, the percentage of respondents who felt comfortable identifying a large pericardial effusion with POCUS increased significantly from pre- to immediately post-intervention (38.1%–97.9%, p < 0.001), and from pre- to 3 months post-intervention (38.1%–88.5%, p < 0.001). The decreased percentage of comfortable respondents from immediately post- to 3 months post-intervention was not significant (97.9%–88.5%, p = 0.090) (Figure 7).

Figure 7:

Percentage of respondents who felt comfortable identifying a large pericardial effusion with POCUS at each time point

On the 3 months post-intervention survey, 80.8% of respondents reported that they had utilized cardiac POCUS on one of their patients at least once and 34.6% had done so on five or more occasions, compared to 52.4% and 21.4% on the pre-intervention survey, respectively.

Finally, it is worth noting that both the immediate post- and 3-month post-session surveys allowed participants to offer free-text comments about their POCUS simulation experience. The feedback was overwhelmingly positive, with many asking for ‘more simulation machines and more time’ to allow for additional experience.

Discussion

Acute decompensated systolic heart failure and large pericardial effusion are common medical conditions that our Internal Medicine residents encounter during their training. These diagnoses often require prompt recognition and intervention but can be difficult to differentiate from other diagnoses without the use of ultrasonography. All residents have access to POCUS machines in our hospital, yet our study participants indicated a substantial lack of comfort with cardiac POCUS skills prior to our simulation intervention.

Studies have shown that Internal Medicine residents are interested in achieving greater POCUS proficiency during residency and that structured POCUS teaching curricula can be effective in improving image acquisition, image interpretation and incorporation of POCUS in medical decision-making [11,14–17]. Our study found that a 3-hour simulation-based educational intervention led to a significant increase in residents’ self-appraised cardiac POCUS knowledge as well as improved performance on a cardiac POCUS knowledge assessment from pre- to immediately post-intervention. Similarly, we found a significant increase in the percentage of residents reporting comfort with various aspects of cardiac POCUS pre- to immediately post-intervention. Further, compared to their pre-intervention levels, our residents’ improvements in each of these areas were significantly sustained 3 months post-intervention, demonstrating the durability of their gains in self-appraised knowledge, true knowledge acquisition and stated comfort. Though some measures of self-reported resident comfort decreased immediately post- to 3 months post-intervention, this is not inherently surprising, and even these measures remained significantly increased 3 months post-intervention compared to pre-intervention.

Our study adds to existing literature on effective ways to teach POCUS to Internal Medicine residents and other trainees in general [12,13]. Elhassan et al. [14] conducted a POCUS needs assessment among Internal Medicine residents and found that those who had received prior POCUS training possessed greater confidence in the interpretation of specific lung and cardiovascular POCUS images and scored higher on POCUS image interpretation tests compared to those who had not. Additionally, they found that most Internal Medicine residents supported POCUS training via workshop methodology (which was, in effect, our intervention’s methodology as well).

Singh et al. [18] conducted a systematic review of simulation-based POCUS curricula that had been published within MedEdPORTAL. They concluded that more research is needed to determine ways to ensure lasting POCUS proficiency. While our study did not measure clinical proficiency, we did demonstrate retention of knowledge and comfort 3 months after the training session.

This study has several limitations. First, we included Internal Medicine residents in only one training programme, so our results may not be generalizable. Second, we focused on cardiac POCUS, so our findings may not apply to other clinical domains where POCUS may be used. Although the survey-based assessments were developed by experts in our academic programme, they were not validated. In addition, there is likely a significant overlap between self-appraised knowledge and comfort. Finally, we were not able to analyse data at an individual level and perform paired t-test analysis, as participant survey responses were anonymous.

Conclusion

Simulation-based POCUS training appears to be effective at teaching Internal Medicine residents not only the fundamental skills of cardiac POCUS, but also on how to utilize the modality to diagnose acute decompensated systolic heart failure and large pericardial effusions. Although more studies are needed to identify ways to optimize knowledge and skill retention following simulation-based interventions, we found significant durability of gains in self-appraised knowledge, true knowledge acquisition, and stated comfort three months after our cardiac POCUS simulation session. Our findings have been very helpful in building a POCUS curriculum for our Internal Medicine residents.