Cultural competence is one component of effective communication between patients, families and healthcare professionals. Tools to assess physicians’ clinical cultural competencies need validity evidence. This paper describes Lawshe’s method for determining the Content Validity Index (CVI) for the Clinical Cultural Competence Questionnaire (CCCQ) for North America (NA), Pakistan (PK) and an international group (IG) using physician simulation educators (PSEs) for diverse cultures.
Five simulation educators pilot-tested the CCCQ, and initial changes were made based on their feedback. A total of 10 PSEs experts from NA, 11 PSEs from PK and 10 PSEs from IG completed two rounds of validation testing using Lawshe’s CVI survey for the CCCQ. The PSEs rated each item of the CCCQ as ‘essential’, useful but not essential’ or ‘not useful’. Lawshe’s CVI was calculated for the initial CCCQ; the CCCQ was then modified for individual items, separately for each group, NA, PK and IG. The IG comprised PSEs from Bahrain, Oman, Qatar, KSA, UAE, Australia, Argentina, India and Israel.
The survey response rate was 83.33% for NA and IG and 91.6% for PK, respectively. The CVI of the CCCQ in round 1 was 0.689 for NA, 0.545 for PK and 0.691 for IG. In the second round of the CCCQ, with modified items, the CVI was 0.89 for NA, 0.802 for PK and 0.862 for IG. The major modifications suggested by the PSEs were to remove the unnecessary items, e.g. demographic information and last medical school attended, as they were deemed unnecessary or reword them for better understanding and combine related items to reduce the length of the CCCQ survey. We also evaluated the comments of PSEs from NA, PK and IG to explore the similarities and differences in their opinions regarding the CCCQ tool items.
Our research emphasizes the need to thoroughly examine questionnaire content in tools like the CCCQ to accurately capture the cultural competence knowledge, attitudes and skills crucial for healthcare providers in diverse settings.
Cultural competence is the understanding that cultural differences and similarities prevail and influence values, learning and behaviour [1,2]. Healthcare providers (HCPs) are expected to care for socio-culturally diverse patients [3,4]. Caring for multicultural patients requires awareness of a patient’s beliefs and cultural perspective [5], for which a culturally focused education is needed [6]. Cultural competency in healthcare is vital for patient care. It involves understanding and engaging with diverse cultural backgrounds, ensuring patient-centred care that respects individual needs and values. This competency improves communication, builds trust and contributes to more inclusive and equitable healthcare, ultimately enhancing patient outcomes [3,4]. A lack of cultural competence in health professions education can lead to a myriad of adverse outcomes, including health disparities for patients from diverse cultures [5]. It has become increasingly important for the acute care setting because of the short interaction time between physicians and patients, challenges in developing the patient–physician relationship and making spot decisions for life-threatening situations [7]. Like (2001) developed the initial Clinical Cultural Competence Questionnaire (CCCQ) tool, based on the four domains of Knowledge, Skills, Comfort with Encounters/Situations and Attitudes consistent with multidimensional models for cultural competence, for 15 faculty physicians participating in a cultural competency training program [8]. Later, it was used only with pharmacy and medical students in clinical settings [8].
The CCCQ tool has the following shortfalls [9,10]:
To fulfill this need, we asked physician simulation educators (PSEs) from North America (NA), Pakistan (PK) and an international group (IG) to help evaluate the tool. While exploring the literature, the CCCQ was not validated with simulation educators from diverse cultures. We decided to address any existing gaps or challenges.
Lawshe’s Content Validity Index (CVI) has been described as one method to measure the content validity of a tool [11–13]. Lawshe’s CVI has been used to evaluate and quantify content validity in various tools in diverse fields, including health care, education, organizational development, personnel psychology and market research [14]. Our research objective was to investigate simulation educators specifically. We recognize that for the effective development of curricula and the successful incorporation of cultural competency into educational practices, it is imperative that simulation educators themselves demonstrate a high level of cultural competence. This deliberate focus on educators aimed to contribute to a more comprehensive understanding of the role cultural competence plays in the educational landscape, particularly within the context of simulation-based training. Therefore, we used Lawshe’s CVI [11] process for the CCCQ tool with PSEs from NA, PK and IG.
We performed Lawshe’s CVI for our study for the CCCQ with PSEs from NA, PK and IG. During this process, we could either eliminate or reword items from the CCCQ based on the comments from PSEs to improve the understanding of the PSEs for the items in the CCCQ for future utilization in diverse settings.
This study measured the CVI of the CCCQ by using the formula suggested by Lawshe [11]. We chose Lawshe’s Content Validity Ratio (CVR) method for its simplicity and efficiency in gauging expert judgement on the relevance of items in the CCCQ for PSEs. While the modified Delphi method is another valid approach involving iterative expert feedback, Lawshe’s CVR aligned more closely with our study’s specific goals and context. PSEs from three different geographical areas were asked to use the Lawshe CVI process for the CCCQ [11–13]. Many tools are available in the literature to assess cultural awareness, including the ‘CCCQ’ [15]. The CCCQ assesses the knowledge, skills, attitudes and comfort levels in performing culturally competent healthcare for a diverse patient population [16]. The CCCQ is a self-administered assessment instrument with 77 items and response choices based on a 5-point scale (not at all, a little, somewhat, quite a bit and very) [10]. The CCCQ (Supplementary Appendix 1) was selected for the following reasons [17]: (1) the tool was compatible with our goals and objectives; (2) it was based on a theoretical foundation; (3) it could be utilized for various levels of learners with various professional backgrounds, and (4) is free to use [18–21]. We obtained permission from the original author to use and modify the CCCQ (Like RC, personal communication, 4 January 2019). For the Lawshe CVI, we asked the PSEs to rate each item of the CCCQ as ‘essential’, ‘useful but not essential’ and ‘not useful’ to determine the CVR for each item and CVI for the tool.
We analysed results separately for various countries or regions exploring the CCCQ for PSEs because it signified a commitment to recognizing and accommodating cultural diversity. Acknowledging that cultural competence is context-dependent, we aimed to capture nuanced cultural differences that may impact the perception and evaluation of cultural competence within various specific contexts of physician simulation education. This approach aligns with contemporary best practices in cross-cultural validation, ensuring the questionnaire’s robustness and applicability across diverse populations of PSEs. Furthermore, the intention was to provide a tool attuned explicitly to each region’s cultural and contextual nuances. Separate analysis offered insights into potential adaptations required for different cultural contexts. This approach also facilitated a more detailed examination of findings, allowing us to discuss the generalizability of results and identify any significant variations that merit further exploration within the realm of PSEs.
To ensure equity, diversity and inclusion, we invited 12 PSEs from three groups:
The PSEs with at least 2 years of experience in SBE were invited using purposive sampling and snowballing techniques. The inclusion criteria were chosen to ensure a baseline familiarity with simulation methodologies. However, to find subject-matter experts in cultural competence, we assessed PSEs with other criteria, such as proven experience in integrating cultural competency into simulation practices, participation in teaching and training for diverse groups of learners and a record of cultural competence-related research or educational initiatives. The response rate was 83.33% for NA (n = 10/12), 91.66% for PK (n = 11/12) and 83.33% for IG (n = 10/12).
Lawshe’s method, initially proposed in 1975 [11], has been widely used to establish and quantify content validity in diverse fields, including health care, education, organizational development, personnel psychology and market research [14]. Content validity, as per Lawshe’s CVI, is described as the extent to which components of an assessment instrument align with and accurately represent the intended construct for a specific assessment objective [22].
To determine the content validity of the CCCQ, we calculated the CVI [12], which is simply the mean of the CVR values for all the items included in the final instrument [23]. We decided to follow Tilden et al.’s [24] suggestion for the CVI and considered the value of 0.70 as the validity cut-off point for the CCCQ. We calculated the CVI of the CCCQ in two rounds, separately for each PSEs group from NA, PK and IG.
Example No. 1: For item #3 in the North America CCCQ, all the experts responded as ‘essential’, so ne is 10 as the number of panel members indicating an item ‘essential’ was 10. The N is 10, as the total number of panel members was 10.
Therefore, the CVR = 1, higher than 0.78, is evidence of good content validity [11,12].
Example No. 2: In item #1 in North America CCCQ, five experts responded as ‘essential’, so ne is 5 as the number of panel members indicating an item ‘essential’ was 5. The N is 10, as the total number of panel members was 10.
This means that the CVR = 0, which is lower than 0.78, and the item did not reach the threshold so that it could be deleted from the final tool [11,12].
We converted the CCCQ into a survey using Google Forms [25] and included the items as they appeared in the CCCQ (see Supplementary Appendix 1).
We asked the PSEs to rate each item of the CCCQ as ‘essential’, ‘useful but not essential’ and ‘not useful’ to determine the CVR for each item and CVI for the tool. They were also asked to provide suggestions to rephrase an item if they found it difficult to understand.
There were two rounds of Lawshe’s CVI survey conducted for the CCCQ. The PSEs provided detailed feedback and suggestions for the individual items.
The process map of measuring Lawshe’s CVI of CCCQ conducted in two rounds
After the modifications of the CVR calculations and based on the PSEs comments, the items with less than 0.78 were either deleted or reworded. The revision summary, with its rationale, is presented in Supplementary Appendix 3.
We incorporated the following steps to ensure the trustworthiness of the CVI process:
This study was deemed exempt (IHHN_IRB_2023_01_016) from the Indus Hospital & Health Network Institutional Review Board.
The primary outcome was the CVI for the CCCQ tool by PSEs from three groups: NA, PK and IG.
Ten PSEs from NA and IG each and 11 from PK responded to Lawshe’s CVI survey for the CCCQ. As shown in Table 1, the IG was composed of PSEs from Bahrain, Qatar, Oman, Saudi Arabia, UAE, Australia, Argentina, India and Israel with a good understanding of English. Other demographic characteristics are shown in Table 1.
| Items | North America (USA & Canada) | Pakistan (PK) | International group (IG) (beyond USA & Canada, PK) | |||
|---|---|---|---|---|---|---|
| Physician simulation educators (PSEs) | n = 10 | n = 11 | n = 10 | |||
| Current country of residence/work | Canada | 7 (70%) | Pakistan | 11 (100%) | Bahrain Oman Qatar Saudi Arabia UAE Australia Argentina India Israel |
1 (10%) 1 (10%) 1 (10%) 1 (10%) 1 (10%) 1 (10%) 1 (10%) 1 (10%) 2 (10%) |
| USA | 3 (30%) | |||||
| Years in the country of work or residence | Less than 1 year | 1 (10%) | 0 (0%) | 0 (0%) | ||
| 2–3 years | 2 (20%) | 1 (9.09%) | 0 (0%) | |||
| 4–5 years | 0 | 2 (18.8%) | 1 (10%) | |||
| More than 5 years | 7 (70%) | 8 (72.72%) | 9 (90%) | |||
| Year of experience | 2–3 years | 4 (40%) | 9 (90%) | 2 (20%) | ||
| 4–5 years | 1 (10%) | 1 (10%) | 4 (40%) | |||
| More than 5 years | 5 (50%) | 1 (10%) | 4 (40%) | |||
| Languages in which you can communicate | English Hebrew Italian French Spanish German |
10 (100%) 2 (20%) 2 (20%) 2 (20%) 1 (10%) 1 (10%) |
Urdu English Punjabi |
11 (100%) 11 (100%) 2 (18.18%) |
English Arabic Spanish Hebrew Hindi Tamil |
10 (100%) 5 (50%) 1 (10%) 2 (20%) 1 (10%) 1 (10%) |
| Training for cultural competence | Yes No |
6 (60%) 4 (40%) |
Yes No |
4 (36.36%) 7 (63.63%) |
Yes No |
1 (10%) 9 (90%) |
| Involvement in teaching and learning with individuals of diverse cultures | Yes No |
10 (100%) 0 (0) |
Yes No |
5 (45.45%) 6 (54.54%) |
Yes No |
9 (90%) 1 (10%) |
The CVI for the CCCQ tool in the first iteration for NA was 0.689, PK was 0.545 and IG was 0.691, respectively, as shown in Table 2. The CVI of the CCCQ tool was less than 0.70 for each of the three groups, indicating that the CCCQ did not demonstrate validity evidence in its current form. Therefore, the items with a CVR value of less than 0.78 were revised by deleting or rewording the items for NA, PK and IG, as shown in Supplementary Appendix 3. The items were revised based on the comments from the PSEs, as shown in Supplementary Appendix 3. Later, the modified items of the CCCQ were sent again to the PSEs from NA, PK and IG. The second round of CVI for the CCCQ was 0.89 for NA, 0.80 for PK and 0.86 for IG, respectively (Supplementary Appendices 4–6). It demonstrated that the CCCQ tool with modified items had a CVI of more than 0.70 for each of the three groups, showing improved validity evidence compared to the round QC (Table 2).
| CVI of CCCQ | North America PSE | Pakistan PSE | International PSE | |||
|---|---|---|---|---|---|---|
| CVI – first round | CVI – second round | CVI – first round | CVI – second round | CVI – first round | CVI – second round | |
| CVI of CCCQ | 0.68 | 0.89 | 0.54 | 0.80 | 0.69 | 0.86 |
CVI: content validity index of the CCCQ tool for the first and second round for North America, Pakistan and International PSEs whole scale. The items with CVI > 0.70 are bolded.
We looked at the comments of PSEs from NA, PK and IG to explore the similarities and differences in their opinions regarding the CCCQ. The PSEs from all three groups agreed that the birth date and medical school attended were not considered essential, especially from the lens of cultural competence. Some of the quotes from PSEs were: ‘Medical school attended is less important’, ‘Asking about your medical school is not as important’ and ‘I do not believe the medical school and year of graduation is essential info’. The PSEs reported that the demographic information was unnecessary and was cumbersome to fill out. The overall language of the CCCQ tool was also felt to be more geared towards NA. One PSE from the IG group said, ‘There is a decent amount of language that is quite specific to an American context. If I were to use it in Ireland or Canada, I would need to change questions specific to “residency” and specific resources mentioned’ (see Supplementary Appendix 3 for the modified CCCQ).
Although some PSEs considered the details of visited countries important to ask, other PSEs did not agree. One PSE mentioned, ‘Countries visited is a vast question, and I would need a much longer time to think of all the countries’. Another important difference was that the ‘type of residency is only useful in an American context; many international postgraduate programs have a mix of large cities, small towns, and many remote locations’. All the PSEs did agree that asking about the knowledge and skill for providing culturally sensitive care and awareness of their patient’s cultural values in their setting was important (see Supplementary Appendix 3 for round 2 CCCQ).
The feedback from PSEs in NA, PK and IG reflected a collective recognition of the importance of culture while simultaneously showcasing diverse perspectives – a manifestation of the principles of cultural pluralism and integral infusion. This acknowledgement highlighted the value placed on cultural diversity, emphasizing equal rights and opportunities for all cultural groups in our society.
Our study is the first attempt, to the best of our knowledge, to calculate Lawshe’s CVI for the CCCQ using PSEs from diverse cultural, geographical and linguistic backgrounds, including NA, PK and IG. To our knowledge, no tool in the literature has demonstrated validity evidence for PSEs [26]. We explored various layers of cultural understanding while determining the CVI for the CCCQ.
While comparing the survey results from NA, PK and IG, we could see many attributes with consensus, including school attended, demographic characteristics and years of living in the present location. The medical school attended was not considered important, especially from the lens of cultural competence. It echoed the thought that medical school may influence cultural understanding but is not the only factor influencing a physician’s cultural understanding [27]. Factors such as upbringing, life experiences, and ongoing professional development and training can also shape a physician’s ability to provide culturally sensitive care [28].
Other demographic information was perceived as unimportant and cumbersome to fill out, such as travelling to other countries, details of residency and such. Interestingly, we found that travelling to other countries was considered an enhancing factor for cultural learning preparedness and resilience [29].
PSEs from PK and the IG reported that the CCCQ tool was more geared towards a specific audience. Many elements were not translatable to other settings, e.g. for item 72, the comment was: ‘I am not sure if college meant the undergraduate degree. If yes, it depends on the population’. Before entering medical school, an undergraduate degree is implanted only in the NA educational system [30]. Many schools in NA have moved away from requiring a specific degree for admission to medical school. As a result, the individual experience varies [31].
Item 22, ‘Ethnopharmacology’ was considered irrelevant information by the PSE from NA. Ethnopharmacology is the scientific study of materials used by ethnic and cultural groups as ‘medicines’ [32]. As per the comment of one of the PSEs: ‘Knowledge of this item might be very subjective for people targeted by this survey’. Ethnopharmacology has been perceived as closely connected to various cultural entities in a specific cultural sphere [33].
Another critical area of conflict was using a medical interpreter, as this was a very commonly utilized and well-established service in NA; globally, it is not very readily available. The words of a PSE, ‘This question is not rational as not all facilities have a certified medical translator’, are echoed by Jaeger et al. [34], who researched the significance of medical interpreters’ needs, barriers and solutions.
The concept of integral infusion is embedding a multicultural perspective throughout the content of a given course and academic curricula [35]. It is essential for cultural competence because it recognizes that cultural competency is about acquiring knowledge and technical skills related to different cultures and developing a deeper understanding of cultural differences and their impact on healthcare delivery [5,36]. An integral infusion approach to cultural competency training recognizes its multifaceted nature. It seeks to integrate different dimensions of learning, including cognitive, affective, social and kinesthetic dimensions, into cultural competency training [36]. By incorporating these elements of integral infusion, healthcare professionals may provide patient-centred care that is respectful, effective and culturally responsive [37,38]. The notion of integral infusion was expressed by one of the PSEs, who stated: ‘I got the opportunity to work, learn & teach people from diverse cultures during my residency, fellowship & work as a staff physician. This continuing experience enhanced my effective communication, got me to understand others better, minimized stereotyping, stimulated my mind, helped me understand diverse ways of doing things, and prepared me to cope with various challenges’.
Cultural pluralism refers to a society or group that includes multiple cultures with distinct norms, values and traditions and where these cultures are respected and coexist peacefully [39]. Cultural pluralism emphasizes the importance of maintaining cultural diversity and recognizing the contributions of all cultures. Unlike a melting pot, which seeks to assimilate individuals into a dominant culture, cultural pluralism encourages individuals to maintain their cultural identity while participating in and contributing to society [40]. If we look at the comments from PSEs from NA, PK and IG, it is evident that everyone recognized the importance of culture but held their unique perspective, which echoed the concept of cultural pluralism. Cultural pluralism also emphasizes the importance of equal rights and opportunities for all cultural groups in a society [41]. It recognizes that diversity can be a source of strength and innovation and that a society that values and embraces cultural diversity is more likely to be successful, creative and prosperous [41].
Moreover, the information gleaned from the CCCQ will serve as a benchmark for understanding baseline cultural competence and guide the refinement of simulation education curricula. Educators and curriculum developers can tailor training programs to address specific needs and foster a more inclusive and culturally aware learning environment by finding areas where cultural competency can be improved.
Our study had the following limitations:
In our future research, we aim to extend the investigation of the CCCQ for PSEs beyond content validity. We plan to explore added sources of validity evidence, including internal structure and relationships with other variables, using the Messick framework. This expanded focus will contribute to a more thorough questionnaire evaluation, enhancing its effectiveness and applicability across diverse contexts.
Our research demonstrated a methodology for determining the CVI of a tool like the CCCQ for use with a particular population of educators. Modifications were needed for each of the three groups in this study to make the tool valid. This validation of instruments for use in cross-cultural simulation is rarely done. The study serves as a model for others who conduct cross-cultural research and may use tools that are not culturally valid from region to region.
Supplementary data are available at The International Journal of Healthcare Simulation online.
JF, AK, KL, MB, IG and SK-E participated in the conceptualization, planning, and design of this paper. All authors contributed to data collection. AK and JF did a quantitative analysis. All authors contributed to the writing of the manuscript and followed the author’s instructions; they have read and approved the manuscript.
No funding was available for this paper.
None.
None declared.
The authors declare no conflict of interest.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
Exploring the content validity of Clinical Cultural Competence Questionnaire in diverse cultures
Facebook
Twitter
Linkedin
Whatsapp
Tweets