COVID restrictions saw the migration of the entire teaching–learning process to online mode. Medical educators faced challenges in the execution of skills teaching via online platforms. This study was conducted to evaluate the process and outcome of online skills teaching compared with historical in-person training.
Participants were undergraduate medical students during clinical skills training (n = 150). Interventional group students attended online teaching of cardiac and respiratory auscultation via virtual and video demonstrations. The control group was a student cohort from the previous academic year taught face-to-face. Students’ performance was assessed by Objective Structured Practical Examination (OSPE) and compared by the Mann–Whitney U-test. Qualitative data were collected through student surveys and faculty focus groups.
OSPE scores of the interventional group were lower compared to controls (2.93 vs. 3.75 and 2.76 vs. 3.90) with statistical significance (p < 0.0001*). Positive findings were faculty expression of a sense of accomplishment and students’ satisfaction with staff preparedness, preliminary instructions and time allotment. Faculty expressed a lack of opportunity to provide hands-on training, lesser learner participation and technical issues. Students expressed a lack of confidence, dissatisfaction with interactions and inability to correlate sequences.
We could infer that outcome of online teaching was lower compared to the control reasons that were evident from subjective feedback. The control group had better avenues for interaction, error correction and repetition. Strategies to improve outcomes are small group size, hybrid teaching, faculty training in digital technology and a supportive technical team.
What the study adds
When the COVID pandemic restricted face-to-face educational activities medical educators utilized online platforms for teaching and assessment [1]. Literature shows evidence of a few virtual teaching attempts by educators even before COVID [2]. A systematic review of e-learning by Traba et al documents the improvement of knowledge in 70% of virtual educational interventions [3]. Woodham et al report of problem-based learning with recorded videos [4] and video-assisted instruction in dentistry (VACID) by Naseri et al are some more examples [5].
Though theoretical components were taught online, teaching clinical skills virtually hadn’t become the routine norm. To fill the gap educationalists adopted innovative virtual teaching methods to impart clinical skills. Since the attempts were novel, challenges were encountered during these online skills teaching sessions and remedial strategies were evolved [6]. To provide hands-on practice during the virtual session, the educator team of University College London adopted the ‘remote not distant model’. Faculty took portable manikins home and performed skills on the manikin at remote locations. Students were mailed a few consumables to practice procedures on household items following the virtual instructions (intravenous infusion in kitchen sponges, suturing in bananas, intramuscular injections in oranges). Efficient learner performance and satisfaction were reported by Wallace et al [7].
Khan used the ‘Remote Peyton’s 4-stage approach’ at St Georges Hospital, University of London. The model followed the initial three steps of Peyton’s four-stage approach: video demonstration without commentary, discussion with multiple short video clips and comprehension by student narration. The fourth phase of execution was replaced with consolidation by a Question & Answer session. The model was a success and well accepted by both faculty and students [8].
Teaching via digital platforms could be delivered as synchronous virtual demonstration and asynchronous projection of recorded videos watched by students remotely [9]. Evidence shows that each of these models had its own merits and challenges [10]. This study was designed to analyse the execution process of online skills teaching and to compare the outcome with that of face-to-face teaching in an undergraduate medical program at a Malaysian University.
An interventional case–control study was conducted at the Clinical Skills Centre of a Malaysian medical university. Institutional ethical committee clearance was obtained for the conduct of the study [Ref No: AUHEC/FOM/2021/5].
One hundred and fifty Year 1 undergraduate medical students during the academic year 2020–2021 were the subjects for the study. Three of those who missed one of the teaching or assessment sessions were excluded, leaving a subject cohort of 147 students.
A historic cohort of students belonging to the previous academic year (2018–2019) who underwent face-to-face teaching formed the control group (n = 150). Case and control groups were matched in knowledge level as all had been selected for the program through a standardized screening examination. An equal time of 2 hours was allotted for both virtual and face-to-face teaching.
The interventions were online teaching by virtual demonstration of cardiac auscultation and video demonstration of respiratory auscultation. The content of the online teaching sessions was aligned with the objectives mapped in the curriculum to ensure uniform content delivery that would match the learning of the control group. Thirty students were recruited per teaching session. They were divided into small groups of six students each by non-randomized sequential grouping according to roll numbers by the simulation centre support staff. Each group was assigned a facilitator exclusively for that group who was responsible for all the teaching/learning activities of the group during the study period. The learning objectives of cardiac and respiratory auscultation are given in Appendix I.
The entire batch of students was given an online introduction to the basic physiological principles of cardiorespiratory auscultation and protocol for the sessions. They were instructed to ensure uninterrupted internet access, log in and log out on time and keep their video on throughout the session.
A virtual demonstration of cardiovascular auscultation was delivered using the ‘Remote Peyton’s 4-stage approach’ by Khan [8].
Step I – Demonstration: Virtual demonstration of cardiac areas location and auscultation on a simulated patient performed by faculty without narrative. Normal heart sounds and cardiac murmurs were played from the audio device ‘Sound tutor’.
Step II – Deconstruction: Virtual demonstration of the steps by the faculty with narration, pause and discussion whenever necessary.
Step III – Comprehension: Virtual narration of the steps by students in turns.
Step IV – Consolidation: The Question & Answer session for contextualization replaced the step 4 execution of Peyton’s original model. Faculty promoted learner participation through focused questioning and clarifying any queries.
Video demonstration of respiratory auscultation: The facilitators taught respiratory auscultation by streaming pre-recorded videos created at the centre. An exclusive video of 20 minutes was recorded at the simulation centre on the steps of respiratory auscultation demonstrated by the lead facilitator on a simulated patient. The recorded video was played uninterrupted followed by a discussion.
An introductory large-group session was conducted with the control group on the basic physiology of cardiorespiratory auscultation and the protocol to adhere to during simulated patient encounters. Since each group was large (30 students), the Nikendei modification of Peyton’s four-step approach was used for the training [11]. The facilitator performed the steps of auscultation on a simulated patient without narrative and later with narrative. Then the facilitator performed the steps while one student narrated and others observed. Next, the first student performed the skill while the second student narrated the steps. This procedure was then followed in turn by all the students. Peer and facilitator feedback was provided to all the students after the performance.
Assessment of skill was done by Objective Structured Practical Examination (OSPE) for both groups of students [12]. Students performing the specified task within the given time (5 minutes per task) were evaluated by examiners provided with task checklists. The tasks aligned to the learners’ knowledge and competency level mapped in their curriculum. To eliminate interrater variability a model answer checklist with the score for each step performed was provided to the facilitators for assessment (Appendix IIA, B Model OSPE task checklist). The examination was conducted face-to-face as part of a summative assessment when students were back on campus. OSPE scores were analysed by the Mann–Whitney U-test. To analyse the outcome of virtual versus face-to-face teaching, the mean OSPE score obtained by the students taught virtually was compared with the score of the control group. Scores obtained through online facilitator demonstration for cardiovascular auscultation versus respiratory auscultation taught by online video demonstration were also compared.
This was conducted with six faculty who facilitated both the online and the face-to-face sessions. It was video recorded and transcripts were analysed using a thematic approach to identify the benefits and challenges of online teaching.
Student feedback on their experiences of attending online teaching was obtained by a 10-item questionnaire provided with Likert responses from one to five (strongly disagree = 1 to strongly agree = 5) shared as Google forms (Appendix III). The questionnaire was a modified version of the tool used by Elaine et al for a student survey on online teaching, which had a Cronbach’s alpha score of 0.67 and Kendall’s tau-B correlation coefficient of 0.134 to 0.39 [13]. This questionnaire was discussed by three subject experts and questions 1–5 of the survey tool were modified to match the objectives of the present study through consensus. The questionnaire collected feedback about the course content, ability to follow online demonstrations, correlate the demonstration with concepts, provision for interaction during virtual sessions and confidence level in performing the skills.
The mean OSPE score of cardiovascular auscultation obtained by a virtual demonstration was 2.93/5 and the score of the control group was 3.75/5 with statistical significance (p < 0.0001*). The mean OSPE score of respiratory auscultation obtained by video demonstration was 2.76/5 and the score of the control group was 3.90/5 with statistical significance (p < 0.0001**). When mutually compared, virtual demonstration resulted in better performance (2.93) compared to video demonstration (2.76), the difference being statistically insignificant (p = 0.141) (Figure 1).
Comparison of mean OSPE scores between test and control groups. Skill I: cardiac auscultation; Skill II: respiratory auscultation; *p < 0.0001; **p < 0.0001.
A focus group was conducted by the first author with the six facilitators involved in virtual teaching. The discussion was led with a focus on the problems faced during the conduct of the session and the benefits they perceived.
Three major themes were identified during the focus group: connectivity/technical issues; a lack of active learner participation; and satisfaction with the online teaching process despite the limitations. Faculty were either unfamiliar with the technology or had connectivity issues. For example, one facilitator said, ‘Since we were new to virtual teaching, I faced a few issues. I found it difficult to focus while switching from my PowerPoint to my demonstration on Simulated Patient’. Another said, ‘I had issues with connectivity, especially while streaming the videos’. Student active participation was felt to be difficult to achieve. For example, one of the facilitators stated, ‘I observed that the number of questions raised and students volunteering to answer were lower when I compare this session with the previous years’. They expressed that ‘the virtual teaching screen limits the number of students seen at any given point and our attention gets restricted only to those who are visible. A smaller group might enhance the visibility of learners’. The positive takeaway of the session was the feeling that learning still took place despite the online format. ‘I felt satisfied for being able to execute the task despite the cessation of face-to-face lectures. Interacting with students during a period of social isolation gave me a feeling of accomplishment’. Another faculty said, ‘Successful conduct of online session made me feel technically updated’.
Student qualitative perception of virtual teaching.
Student feedback had mixed responses. A positive response as agree/strongly agree was given by more than 50% of students to items 6, 8, 9 and 10 about preliminary instructions, staff preparedness, and time allotment [item 6 – 64%, item 8 – 55%, item 9 – 67% and item 10 – 64%]. Only 54% of the students wanted to recommend similar sessions in the future. More than 50% of students disagreed/strongly disagreed with items 2, 4, 5 and 7 pertaining to confidence level, correlating with concepts, following sequence and sufficient interaction [item 2 – 70%, item 4 – 67%, item 5 – 71% and item 7 – 66%]. Hence, the majority of learners were not confident about performing the task in future, unable to correlate the concepts with the performance, unable to follow the sequence of tasks and dissatisfied with the level of interaction (Figure 2).
The outcome of virtual demonstration (2.93/5) and video demonstration (2.76/5) were above 50% which was the eligibility criteria for qualifying as a pass. Hence we infer online teaching would enable the attainment of basic auscultatory skills as per our University standards. Among the two online teaching models, the video demonstration score was lower. The lack of interaction during video demonstration by uninterrupted streaming of recorded video could have been a reason for such an outcome. When both models of online teaching scores were compared with the respective face-to-face teaching, the control group scored higher (3.75/5 and 3.90/5, respectively) with statistical significance (p < 0.0001). This score difference between online and face-to-face teaching observed is of concern. Even though virtual teaching achieved basic competency, it was not sufficient to impart proficiency as attained by face-to-face teaching. We explored the probable reasons for better outcomes in the control group. Face-to-face teaching enables better interaction between the teacher and student including non-verbal cues which were missing in online teaching. We believe that it would have been easier for the students to interact in a physical medium than an online platform to follow steps and clear doubts. The opportunity for repetition was present in face-to-face teaching which was lacking in online teaching. The unfamiliarity of the faculty with technology might have led to poor transfer of skill learning.
We also explored the student’s feedback and thematic analysis of the faculty focus group to identify issues with online skills teaching. Eighty percent of students expressed a lack of confidence in performance and dissatisfaction with avenues for interaction, whereas faculty expressed difficulty with active learner participation. We infer that the lack of active student participation might have resulted in the subjective feeling of lack of confidence and satisfaction. A similar finding was reported by the cross-sectional survey of fourth-year medical students in the United States by Franklin et al [14]. The survey revealed that 65% of the students expressed a lack of confidence as they did not find hands-on practice possible with online teaching. It was reported that such a lack of confidence during preclinical years might produce serious implications in the clinical years [15]. To overcome the challenge of diminished learner participation during virtual teaching a strategy was adopted by the Department of Medicine, Justus Liebig University Giessen. Neurological examination was taught by video conference with a faculty–student ratio of 2:4 by Schleicher et al. A tutor demonstrated on another tutor, and the student demonstrated at a remote location on a family member. The model reported positive learning outcomes and the student satisfaction level was high [16]. We believe that the student–faculty ratio is a critical aspect of online teaching and that online sessions must be designed with a smaller group to enable better interaction.
More than half of the students expressed an inability to follow the sequence and correlate the concepts, which might have been due to the technical/connectivity issues during online teaching as expressed by faculty. Literature shows that educators had faced similar problems while attempting to teach online, with one study by Dost et al reporting that poor internet connection, lack of compatible online platforms, competent IT personnel, poor computer technical skills and absence of institutional policies were barriers to online learning [17]. Developing a strong technical support team with an advanced IT setup, good bandwidth and faculty training in digital technology should be able to solve this issue.
The faculty in the focus group felt a sense of accomplishment for being able to deliver the teaching session during a period of social restrictions. The initiative and efforts of the faculty were substantiated by student survey responses. Around 50% of students strongly agreed that staff preparedness, sufficient preliminary instructions and time allotment were adequate. This feedback revealed the commitment of the faculty and that they had devoted sufficient time and effort to online teaching. Similar feedback was reported by Shahrvini et al as 64.1% of their study population preferred virtual teaching for the flexibility of learning it offered at a remote location [18].
We correlated our experience and outcome of online teaching with the evidence from the literature to propose measures to overcome the challenges and improve outcomes.
Clinical skills can be effectively taught online, yet face-to-face teaching enables better attainment of proficiency. Probable reasons for better outcomes by face-to-face teaching are easier teacher–learner interaction, provision for hands-on practice and opportunity for repetition. Strategies such as small group size, hybrid teaching to enable hands-on training, faculty training in digital technology and a supportive technical team can improve outcomes. Though online teaching emerged as an alternative during the pandemic, with the advancement of technology it might become one of the routine teaching models in the future since it would enable learners at low resource settings to receive the benefits available in an advanced centre.
Long-term retention of skill and knowledge has to be tested. The study evaluated only the psychomotor component of teaching–learning. The effectiveness of online teaching in cognitive and affective domains of learning has to be addressed.
We acknowledge the IPSS INSPIRE (International Paediatric Simulation Society, International Network for Simulation-based Pediatric Innovation, Research, and Education) as the study is a part of IPSS INSPIRE fellowship 2020–2021 research. The study was conducted at the Clinical Skills Centre, Asian Institute of Medical Science and Technology University, Malaysia.
None declared.
None declared.
None declared.
None declared.
There are no conflicts of interest or disclosures from the authors.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
| Cardiac auscultation | Locate four cardiac areas |
| Identify normal heart sounds S1, S2 | |
| Respiratory auscultation | Locate the various lung fields |
| Auscultate lung fields from above downwards both sides | |
| Identify vesicular and bronchial breath sounds | |
| Identify abnormal breath sounds – crepitations, wheezing |
| Question: Auscultate over the lung fields and trachea of the subject and report. | |
|---|---|
| Expected steps | Scoring (maximum 5) |
| Introduces him/herself, asks the subject to relax, explains the procedure and get the consent | 1 |
| Exposes the area to be examined | 0.5 |
| Places the diaphragm of the stethoscope over the lung fields sequentially on both sides and auscultates. | 1 |
| Places the diaphragm of the chest piece over all the trachea to hear the breath sounds | 0.5 |
| Comments on the type of breath sounds heard over the lung fields as vesicular breath sound and over the trachea as bronchial breath sound. | 1 + 1 |
| Question: Auscultate the mitral area on the simulated patient and report your finding. | |
|---|---|
| Expected steps | Scoring (maximum 5) |
| Introduces him/herself, asks the subject to relax, explains the procedure and get the consent | 1 |
| Inspects the chest wall for visible pulsations | 1 |
| Locates the apex beat with palm, ulnar border of hand and points with two figures. | 1 |
| Auscultates the apex with the diaphragm of the stethoscope. | 1 |
| Reports the finding. | 1 |
Kindly choose the most appropriate option based on your personal feeling with regard to the online skills sessions you attended. The questionnaire is anonymous and is not a part of the curricular assessment.
| S. no | Statement | Strongly agree – 5 | Agree – 4 | Neutral – 3 | Disagree – 2 | Strongly disagree – 1 |
|---|---|---|---|---|---|---|
| 1 | I recommend such sessions in future. | |||||
| 2 | I am confident in performing the skill taught by online teaching. | |||||
| 3 | I could clearly visualize and follow the demonstration | |||||
| 4 | I could correlate the basic concepts with the performance of physical exam. | |||||
| 5 | I was able to follow the sequence of the demonstration. | |||||
| 6 | Preliminary instructions are useful and sufficient. | |||||
| 7 | Interaction during online session was satisfactory | |||||
| 8 | Staff have enough experience in e-learning requirements | |||||
| 9 | Educational activities got enough time during the online teaching | |||||
| 10 | E-learning compensated the suspension of face-to-face teaching due to COVID pandemic |
Effectiveness, merits and challenges of simulation-based online clinical skills teaching compared to face-to-face teaching – a case–control study
Facebook
Twitter
Linkedin
Whatsapp
Tweets