Identifying the core components and items to measure health workers' cultural competence in the Ethiopian context

Health service provision in the Ethiopian context

Ethiopia has a short history of modern health services and upholds its roots during the Emperor Regime simultaneously with the 1948 Ministry of Health establishment. ³⁸ The Emperor Regime and Ethiopian subsequent governments commonly have preventative, curative, and rehabilitative health goals. Ethiopia evidenced high population growth rates with 120 million people in 2022, estimated 250 million in 2050, ³⁹ urban (21%) -rural (79%) population variations, ⁴⁰ is the home of more than 80 ethnic groups, ⁴¹ and people communicate using 90 languages of Semitic, Cushitic, Omotic, and Nilo-Saharan families. ⁴² These cultural factors conglomerated with others have made Ethiopia a home of paramount cultural diversity that will call for the need for CC assessment in crucial service sectors.

So far, heart-breaking conflict, violence, displacement, and war have continued as curses in recent and past Ethiopia. This might be due to inadequacy to address CC in the policies and practices. These devolved situations have supported Huntington’s 1993 assertion; human conflict and violence in the new millennium will not be primarily generated on economic or ideological grounds but rather from the limitations of designing culturally responsive systems in crucial service sectors. ⁴³

The health sector, as key service sector, is not only limited in incorporating CC but the health policies and practices overlooked it. ⁴⁴ Or else, past and current governments of Ethiopia commonly worked massive health facility expansion and increasing number of health workers to meet the increasing health demands of rapidly growing population. ^{45 – 47} However, Ethiopia is still known for very low densities of health workers at 0.96 per 1000 population far below the African density of health workers of 2.2/1000 population and five times less than World Health Organization’s (WHO) minimum threshold of 4.45 per 1000 population to meet the SDG health targets. ^{37 , 48} According to Haileamlak’s study finding Ethiopia has less than 100,000 health workforce and the health workforce in 2022 is estimated to be less than 120,000. ³⁷ Upon this, Ethiopia should require more than 200,000 health workers to reach the density of African health workforce and 500,000 to fulfill Universal Health Care (UHC) minimal requirements. This means that in order to achieve UHC by 2030, Ethiopia expected to train more than 65,000 health workers (medical doctors, health officers, nurses, and midwives) per year for the next eight years.

However, the work to increase Ethiopian Health workers has increased variations on cultural factors such as language, ethnicity, food, clothing, religion, etc., both within their colleagues and patients. ⁴⁹ Alternatively, to minimize health gaps, scholars suggested that the Ethiopian government revise its health education curricula, training modality, trainee recruitment, and quality of teachers to go in line with CC international standards, parallel to health workers increasing tasks. ³⁷ Whereas, increasing diversity, conglomerated with limited numbers of health workers, unavailability of CC tools to assess health practices, limited CC attention in health policy, and education curricula have hampered delivering quality healthcare, obstructed closing health disparity, and prevented achieving national and global health goals. ⁵⁰

Despite this, the international health education curriculum overwhelmingly incorporates CC issues and conducts a comprehensive assessment of health workers as prime priorities of health research, policy, education, and practice. ⁵¹ Because, designing mechanisms to improve in-service training to equip health-workers with knowledge, attitude and skill of CC is helpful to deliver quality healthcare by reducing societal dissatisfaction. However, CC as one of mitigating strategies to Ethiopian health challenges, particularly gaps resulted from increasing variations of cultural factors, has gained little attention from both the past and current Ethiopian governments.

In order to bring high-quality health outcomes to satisfy health demands of culturally diverse patients, Ethiopia health system should design strategies to enhance health workers or providers CC, and conduct comprehensive CC assessments to understand existing health gaps as essential components of health practices have emerged as suggestions. ^{34 , 52} According to studies, Ethiopia should emphasize CC education, on-the-job training, interpreter service, and comprehensive assessment of health workers to deliver culturally appropriate care. ⁵³

However, beyond CC limitations evidenced worldwide; the Ethiopian health system has demonstrated additional gaps. For instance, nurses reported as if they do not hear the term cultural competence both in their schooling and practices, rather they accustomed to clinical competence. ⁵³ Another study evidenced a considerable numbers of Ethiopian people believed that modern health services showed unacceptable practice. ⁵⁴ A growing numbers of studies overseas recommend the need for CC assessment of health workers to address patients’ health demand. ^{4 – 6 , 18 , 55} Not surprisingly, improving the CC of health professionals by formulating CC policy and conducting comprehensive CC assessment has become the most recurring theme in Ethiopian health. According to Bonsa (2018), it is critical to conduct nationally representative surveys of CC from patient, provider, and professional respondents. The cultural difference of western countries as low context culture and non-western countries as high context culture have limited the universal and consistence use of CC instruments to measure CC of health workers. ^{56 , 57} For instance, the cultural belief of health or illness acceptable in one culture may not be acceptable in another culture. ^{58 – 62} A study on the health or illness belief of Ethiopians evidenced that spiritual healing is an important treatment for illness, prefer injections to tablets, and prefer translators from the same ethnic group. ⁶³

Additionally, Ethiopia’s proven existence of a high number of cultural factors, limited attention to CC in health practices, ambivalent or complex nature of CC, and inconsistencies in the rationales of CC tool construction overseas might make using a specific CC tool to assess the CC of Ethiopian health workers unsound. ^{34 , 64} Therefore, the CC tool for use in Ethiopia should be comprehensive in nature, let alone other cultural factors, and it should address the existing marginalization resulted by prejudice or stereotypes within same culture. Marginalization of individuals by their perceived dominant counterparts within same ethnic group; body contacts with them are not allowed, and touching should be believed as bedraggled that heightens patients’ dissatisfaction on the healthcare service.

The proven existence of health disparity of Ethiopia resulted from such and other cultural factors have made imperative to conduct such types of study prior to developing, adapting and validating CC tools. Therefore, conducting this type of study is becoming mandatory not only to achieve the objectives but also crucial to use as a steppingstone for the development of Ethiopian Health Workers CC scale. The next sections will discuss the conceptualization and theoretical frameworks of CC.

Conceptualization and theoretical frameworks of CC

There is no single agreed-upon definition of CC, continues as complex, ambivalent, and ambiguous to conceptualize it. ⁴⁴ Scholars prefer to denote it interchangeably using various terms; ‘culturally congruent’, ‘interculturality’, ^{13 , 65} ‘transculturality’, ^{66 , 67} and ‘crossculturality’. ²² The major defining features and concepts deduced from these terms might not contradict with each other. ⁶⁸ For example, the terms commonly share the necessity to address the needs of clients who are essentially different from the one who provide services to them by one or more cultural factors. ⁶⁸ The aforementioned terms were used in this study to represent the constructs and concepts addressed in CC.

In spite of its lack of consensus, CC is conceptualized as a set of congruent behaviors, attitudes, and policies that come together to enable a system or among consumer providers and professionals to work effectively in cross-cultural situations. ⁶⁵ Again, Padopoulos et al., defines CC as the study of cultural differences and similarities in health and illness, underpinning societal and organizational structures to understand current health care practice and contribute to future development in a culturally responsive way. ⁶⁷ The ambiguity of the concept is manifested in the definition stated as mechanisms of giving attention to patients’ prior healthcare experiences in their country and abroad, to possible migration backgrounds and experience therein. ⁶⁸

The differences in defining features may become sources of contradiction to use a single model or theory on various health cultural contexts. Therefore it is mandatory to identify most widely used definitions, and the Campinha-Bacote’s definition of CC, and its sub-constructs are found to be appropriate to use as operational definitions of this study. ⁹

Beyond the differences, the definitions commonly strive to achieve three core components of global health goals; health as human rights, attention to the entire world’s perspectives, and use of an interdisciplinary approach. ⁶⁹ Regardless of the continued ambiguity, scholars continued to develop, adapt, and evaluate CC measuring tools in the context of health using theories, models and conceptualizations.

Among the classic theories of culture and culturally appropriate health, the most notable one is Leininger’s theory; she worked as a clinical nurse specialist with worried children in the USA and noticed recurrent behavioral variations among children. ^{66 , 70} It motivated her to formulate a hypothesis stating whether such variations have a cultural basis or not. ^{59 , 71 , 72} To test it, she studied and conducted intensive research to understand the existing nature of culture outside the Western context. ⁶⁶ She came to Africa, Papua New Guinea, and studied the culture of the country, and she later synthesized her grand theory, called the Cultural Care Theory (CCT).

CCT aims to equip health professionals with the capacity to care for culturally diverse people as crucial elements of the healthcare field. During the study, she gave emphasis to the transcultural capabilities of nurses in delivering culturally appropriate care to clients. She devoted considerable time not only developing the CCT but also worried about making use of it in most cultural contexts. Leininger’s philosophical roots of CCT include: extensive and diverse nursing experiences; anthropological insights; life experiences and values; and reflection upon spiritual insights and beliefs. ^{58 , 66} Leininger represents her theory using sunrise model, symbolizes rising of the sun to represent care. Her CCT is depicted in Figure 1. ⁷³

Figure 1. Leininger’s Sunrise Enabler to discover culture care.

This figure has been reproduced with permission from Leininger, 2008. ⁷¹

The figure depicts the four foci of the Sunrise Enabler as a cognitive map of the culture care theory; within the foci in the upper portion of the model are components of social structure and worldview factors that influence care and health. The figure depicts how health as a service made available to diverse clients should consider cultural variations manifested on individuals, families, groups, and communities when providing services, delivering health education, and designing curriculum. ^{58 , 62 , 74}

Leininger’s theory primarily emphasizes culturally based factors such as technological, religious, and philosophical; kinship and social; and cultural values, beliefs, and life-way’s in the second foci. ^{58 , 59} Leininger argues that delivering culturally congruent health care is not gained by any single factor, but rather by a complex interaction of social, organizational, and internal and external dimensions and factors of health providers and professionals. ⁷⁵

The other model of CC is Giger and Davidhizar established Model of Transcultural Assessment and Intervention (MTAI) first published in 1988. ⁷⁶ The MTAI explores variations that exist in caregivers’ responses and perspectives relative to cultural diversity presented in a specific country. This theory can be used to examine the evaluation, decision-making, and implementation approaches of health professionals regarding CC in health. ⁷⁶

Some improvement is observed in 2000 and 2005 in Purnell’s model of CC in recapping the limitations of MTAI and clarifying CCT’s three activities and decisions. The factors in Leininger were identified as 12 domains in Purnell’s model that include overview/heritage, communication, family roles and organizations, workforce issues, bio-cultural ecology, high-risk behaviors, nutrition, pregnancy and childbearing practices, spirituality, healthcare practices, and healthcare practitioners. ⁷⁷ The most widely used CC model is developed by Campinha-Bacote; making many improvements to her initial model that aimed to recapture the ambiguous nature of CC. Her model is called the Process of CC in the Delivery of Health Care Services. ^{10 , 17 , 78} Her 1991 model incorporated a limited number of constructs such as cultural awareness (CA), cultural knowledge (CK), cultural skill (CSK), and cultural encounters (CE) without expressing their interdependence at the same time. She then decided to revise her model, adding a fifth construct called cultural sensitivity (CS) and changing the pictorial representation from linear to a set of circles. ^{18 , 78}

The five CC constructs in the model share more common features. Campinha-Bacote’s model assumptions include: CC is a process, not an event; there is more variation within ethnic groups than across ethnic groups (intra-ethnic variation); levels of competence of health workers are related to their ability to provide culturally responsive health care services; and it consists of the aforementioned five sub-constructs or factors. ⁹

For this study, the concept of CC and sub-constructs defined by Campinha-Bacote’s 2003 work were used as operational definitions. ⁶⁹ This is because the validation, adaptation, and development of most CC instruments were highly informed by the sub-constructs of CC identified by Campinha-Bacote’s model. ^{30 , 79 , 80}

Cultural competence (CC) is health workers’ ability to deliver culturally appropriate care to culturally diverse patients or the ability to synthesize and apply previously acquired cultural awareness (CA), cultural knowledge (CK), cultural skill (CSK), cultural encounter (CE), and cultural sensitivity (CS) to provide culturally appropriate patient care’.

Cultural encounter (CE) is an ongoing process of interacting with patients from diverse cultural backgrounds. It aims to validate, refine, or modify existing values, beliefs, and practices of a cultural group.

Cultural desire (CD) is the motivation of the healthcare professional to ‘want to’ engage in the process of becoming culturally competent, not to ‘have to’.

Cultural awareness (CA) is deliberate self-examination and in-depth exploration of one’s biases, stereotypes, prejudices, assumptions, and ‘-isms’ that one holds about individuals and groups who are different from them.

Cultural knowledge (CK) is the process of acquiring a solid educational foundation about culturally and ethnically diverse groups.

Cultural skill (CSK) is the ability to collect culturally relevant data regarding the patient’s presenting problem as well as accurately perform a culturally-based physical assessment in a culturally sensitive manner.

Ethical statement

Hawassa University, College of Natural and Computational Sciences, Research Ethics Review Committee (RERC) has approved the implementation of the research on September 11, 2022, with reference number RERC 10010/22 (Letter of Research Ethics Review Committee). The Committee confirmed that the research met the criteria, which included minimizing subject risk, selecting subjects equitably, determining respect for person-informed consent, documenting it appropriately, and maintaining privacy and confidentiality. Written informed consent was obtained from all participants.

Sources of information and search strategy

We searched the term ‘review of CC measuring tools in health settings’ in the Google Scholar database for publications from November 2021 up to August 2022. The search was narrowed down again by searching the statement ‘health workers’ Cultural Competence Measuring Instruments: a systematic review’ within the first search. Then, 347 articles or documents published in English were downloaded for further evaluation to select relevant materials. Making screenings based on COSMIN checklist manual, ⁸¹ three articles such as “measures of cultural competence in nurses: an integrative review”, ⁵¹ “psychometric properties of instruments used to measure the cultural competence of nurses: a systematic review”, ²⁴ and “cultural competence: analyzing the construct” ²⁵ were chosen for intensive review to further narrow down the selection of articles.

Based on the insights gained, full-text screening of studies related to CC in health settings and factor analysis of sub-constructs, models, and theories with target groups of health professionals, health workers, nurses, nursing students, physicians, and health providers were conducted. Articles on CC assessment, scale, tools, instruments, and questionnaires were also considered for inclusion in this study.

Finally, 103 articles or documents of two types in the databases of Academic Medicine, Sage Pub, Academia, Course Hero, Science Direct, Transcultural Care Net, Quiz Let, Semantic Scholar, Simply Psychology, Springer, and Research Gate which were available on Google Scholar and published between 1972 to 2022 were identified and reviewed.

The first type of papers, papers from overseas outside of Ethiopia, including: methodological materials; review of culture and CC; developing, adapting, and validating CC tools; and CC concepts, theories, and frameworks. The second type of paper, papers based in Ethiopia or about Ethiopian topics, including: sociolinguistics, population, and human resources; health policy, strategy, goals, diversity for outcome disparities, and standards; and sociocultural health beliefs, languages, and ethnicity.

Procedure

Eight carefully selected experts participated in this study. The experts were selected based on their relevant contributions to the issue under study through in-depth discussions held with college deans, department heads, and researchers. There were three experts in the first group; one physician with an MD degree, teaching health-related courses for health students at Dilla University and delivering health services at the university’s referral hospital for more than five years; one sociologist (PhD) with more than ten years of teaching experience at the same University; and one clinical nurse working at the university’s referral hospital for more than a decade. The two health workers of this group were selected as participants upon the recommendations of the health college dean of Dilla University. The third expert in this group is a sociologist, whom the authors chose after carefully evaluating the profiles of the teachers in the departments of social anthropology and sociology. Experts in this group served as judges’ in the selection and inclusion of CC tools, sub-constructs, and respondents’ personal information. Besides, these experts have had a chance to independently select participants from health workers to participate as experts in the second group of the study, i.e., the screening of items from their agreed-upon CC tools

Upon the recommendations of the first group of experts, the other three experts in the second group included a physician working at Black Lion Hospital; a clinical nurse holding a master’s degree and who was working at Jemmo Health Post in Addis Ababa, and a lecturer in public health working at Jimma University. These experts have more than seven years of proven experience in the health setting of Ethiopia and participated in the screening and selection of items for further evaluation of CC instruments initially decided for inclusion by the first group of experts.

Following this, the third group were set up for the final screening. To select experts who participated in the third group, the authors have had in-depth discussions. They arrive at a consensus to select these experts from psychology department. Because courses related to item development, adaptation, and validation have been delivered to undergrad and postgrad students at psychology department. The group constituted two experts with MA degrees in psychology who specialize in measurement and evaluation that aimed to equip learners with the necessary knowledge, skill, and value for valid and reliable assessment instrument construction, appraisal, and administration with proven experience in teaching test construction and related courses. Both experts in the third group were lecturers at the Department of Psychology, College of Education, Dilla University.

For the purpose of obtaining their willingness to participate in the study, experts in the first and third groups were communicated with through both face-to-face contacts and telephone calls. These experts in both groups were communicated with for data gathering at Dilla University, Odaya Campus, in the lecture hall of the college of education and behavioral sciences at different time points. While experts in the second group were contacted via phone calls, data was gathered at the Office of Excellence Consultancy and Training Center (ECTC) in Addis Ababa. The willingness expressed during our contacts has been taken as their initial consent to participate in this study. Beyond the discussions held at different time points, the experts in the entire three groups have independently engaged in activities related to the tasks assigned in the respective groups of the study. Finally, the written consent of experts in the entire three groups has been obtained during the face-to-face contact made during data gathering. The data collections were held in Dilla and Addis Ababa starting from September 2022 up to November 2022. Figure 2 depicts the procedure followed to guide this study.

Figure 2. Procedure of the study.

To guide the subsequent stages, rating criteria have designed based on review of prominent scholarly works on principles of scale development and item construction ^{82 – 85} ; review of articles on CC instruments ^{24 , 25 , 51} ; and the COSMIN checklist manual. ⁸¹ The rating criteria were synthesized and guided by the prominent work of McAlister et al., ⁸⁶ on qualitative coding to assess Inter-Raters Reliability (IRR) through experts discussion in iterative processes. The rating criteria for raters to rate have prepared by reviewing the technical manuals of CC instruments, and relevant articles.

The rating criteria used to select CC instruments for inclusion in the study include the adequacy of the number of sub-constructs; the use of a clear framework; the appropriateness of the target group; an adequate sample size; methodological flows; originality; and reports of validity and internal consistency. The personal characteristics were selected based on raters identified from CC measuring tools, literature reviews, CC frameworks, and practical experience on personal, professional, and job-related cultural factors believed to have significance in delivering culturally congruent care to culturally diverse Ethiopian patients. The sub-constructs were decided from the number of factors identified in the selected CC tools, literature review, CC frameworks, and factor analysis results of related studies. The prepared rating criteria to guide this study are indicated in Table 1. ⁷³

The experts at each stage have prepared codes to use as rating criteria guided by the prominent work of McAlister et al., ⁸⁶ on qualitative coding to assess Inter-Raters Reliability (IRR) through experts discussion in iterative processes. As well, the preparation of rating codes were also supported by test construction materials, three articles on systematic reviews of CC instruments, and COSMIN checklist manual. ⁸¹ Three experts in the first group designed rating codes mainly formulated from the COSMIN checklist and test construction material. According to the checklist, in such studies, health measuring instruments should be included if they meet the following criteria: internal consistency or reliability, content validity, structural validity, and cross-cultural validity. As well, they further enrich their rating codes with test construction materials and the three reviewed articles. They agreed upon the number of sub-constructs in the cultural competence instrument, appropriateness of the target group, comprehensiveness in addressing prominent cultural diversity factors, attention to the use of a clear study framework during instrument development, or validation or adaptation as their rating criteria to guide their instrument selection decision.

Out of the selected CC tools, the first group of experts decided on 20 relevant measuring tools to pass to the next stages of this study. Next, IRR rating codes were further enriched and agreed on adequacy of number of sub-constructs, clear theoretical frameworks, appropriateness of the target group, sample size adequacy, internal consistence report of the full scale and sub-scales, indicating relevance validity report, methodological flows, and originality for their decision by rating “1” to agree with the inclusion of the CC instrument or “0” to disagree with the inclusion of the CC instrument. The prepared rating criteria to guide this study are indicated in Table 1.

In general, the three health experts in the first group have participated to decide inclusion of number of CC tools, identify sub-constructs and determine relevance personal information for the subsequent parts of this study, i.e., pilot testing, validating and factor analysis. As well, they have decided the number of CC instruments to pass to item selection, and evaluation subsequent stages.

To select items from agreed CC instruments, three experts in the second group have further designed IRR codes based on McAlister et al., work. They prepared the codes after attaining orientation on good personality item writing, and independent readings. After synthesizing the principles to select items from COSMIN ⁸¹ and test construction materials. ^{82 – 84} Expert of this group have agreed to these items incorporated from relevance sub-constructs, appropriateness to the target population, and adherence to principles of item construction as rating codes for their decision of inclusion. For instance, the codes to guide experts of both in the second and third group to for item selection and evaluation respectively were used test construction materials of Kline (2015), DeVellis (2012), and Nunnally and Bernstein (1994). ^{82 – 84}

According to DeVellis, the criteria to select or evaluate the utility of personality items, including: don’t make stuff in the past tense; make items with only one idea; keep away from double negatives; items with a simple and direct sentence structure are preferred; avoid employing absolute words as only, just, always, and none; avoid items with many interpretations; avoid items that are likely to be endorsed by everyone; use straightforward language; and keep items to a maximum of 20 words. ⁸² As well, rating criteria such as the acquiescence response set, social desirability response set, a preference for responding to things in the center or ‘uncertain category’, and a subject’s preference for responding to the extreme response categories of Kline were considered as codes of rating in both the selection and evaluation of items. ⁸⁴

Finally, the following item construction principles suggested by Nunnally and Bernstein were also taken into consideration during both item selection and evaluation. ⁸³ Their suggestion include: items must be clear; use straightforward wording; all respondents will be able to offer information reliably when asked for an item; each respondent must be able to provide an answer to each question; and write specified short items. ⁸³ Therefore, experts in the second group and third group were taken into consideration in selecting and evaluating items by using the aforementioned item writing suggestions to guide their ratings and use as coding criteria. Table 1 depicts the coding criteria to be used for inclusion, and exclusion ratings. ⁷³

According to McAlister et al., ⁸⁶ preparation of criteria is an initial step towards establishing IRR between raters within a team of experts. IRR was computed to determine the percentage of agreement between raters. IRR indicated that each rater might not have to rate either for inclusion or exclusion of same number of items, tools, sub-constructs, and demographic variables.

The IRR was calculated as the number of agreed ratings over the total number of ratings on the issue under study as per required. The computed IRR, which gives the fractional percent of the number of rated materials that agree, should be compared to the desired 80%-90% agreement. The overall IRR was calculated the number of ratings that all three raters agreed on divided by the number of total ratings. In addition, IRR between each individual pair of raters computed the total number of rated materials or sections divided by the number of ratings that both raters agreed on. Each set of two raters had two IRR values: (1) the number of times rater one agreed with rater two divided by the total number of ratings used by rater one, and (2) the number of times rater two agreed with rater one divided by the total number of ratings used by rater two.

Cohen’s Kappa was computed to know the IRR of two raters that evaluate the quality of the final items before being included. Cohen’s, symbolized by the lowercase Greek letter, is a robust statistic useful for either inter-rater or intra-rater reliability testing. Similar to correlation coefficients, it can range from -1 to +1, where 0 represents the amount of agreement that can be expected from random chance, and 1 represents perfect agreement between the raters. ⁸⁷

The computed reliability for the inclusion of instruments, CC sub-constructs, item selection, evaluation, and demographic variable identification were used as IRR coefficients ⁸⁶ and inter and intra-rater reliability testing standards. ⁸⁷ For computed IRR, the fractional percent of the number of rated materials that agree should be compared to the desired 80%-90% agreement.

Two raters were screened and evaluate items in this study. The IRR is computed using Cohen’s Kappa. The probability of number of items both raters of experts in the third group agree on inclusion (a) and exclusion (b) were calculated. Then, for the items IRR Cohen’s Kappa was computed. The IRR result should be evaluated against indicated reliability index indicated. The two experts again discussed on contextualizing items further for inclusion, then an increase on the computed Cohen’s Kappa is expected to attain. The final number of screened items was given for two experts for further evaluation.

Finally, the computed reliability has been compared with McHugh’s ⁸⁷ standard of reliability coefficients and level of agreement, it is supposed to be: none for 0-.20; minimal for .21-.39; weak for .40-.59; moderate for .60-.79; strong for .80-.90; and excellent for above .90.

Review of CC measuring tools

Before presenting findings of the study, the reviewed results of CC measuring tools were presented in Table 2. ⁷³

These 20 instruments include: Nurse CC Scale (NCCS) of Perng and Watson ²⁷ ; Cultural Self-Efficacy Scale (CSES) of Bernal and Froman ⁹³ ; CC Clinical Evaluation Tool of Jeffreys and Dogan ⁹⁵ ; Cultural Competency Instrument (CCI) of Kosoko-Lasaki et al. ⁹¹ ; Inventory for Assessing the Process of Cultural Competency (IAPCC and IAPCC-R) of Campinha-Bacote ²⁷ ; Cultural Knowledge Scale (CKS) of Brathwaite and Majumdar ⁹⁰ ; CC Assessment Tool (CCAT) of Vasiliou and Kouta ¹⁹ ; Transcultural Self-Efficacy Tool (TSET) of Jeffreys and Smodlaka ⁸⁷ ; Cultural Diversity Questionnaire for Nurse Educators— CDQNE of Sealey ⁸⁹ ; Ethnic Competency Skills Assessment Inventory (ECSAI) of Napholz ⁸⁶ ; CC Assessment Instrument University of Illinois at Chicago (CCAI-UIC) of Suarez-Balcazar et al. ²⁹ ; CA Scale (CAS) of Rew et al. ⁹³ ; CC Assessment (CCA) of Schim et al. ⁸⁸ ; Cross-Cultural Evaluation Tool (CCET) of Hughes and Hood ⁹² ; Cultural Sensitivity (CS) of Chang, Yang, and Kuo ⁹⁴ ; Korean CC Scale for Clinical Nurses (K-CCS-CNs) of Chae and Lee ²³ ; Italian Nurses CC Questionnaire (INCCQ) of Gol, Dorosti, and Montazer ⁹⁸ ; Critical CC Scale (CCCS) of Almutairi and Dahinten ⁹⁶ ; Cross Cultural Evaluation Tool (CCET) of Terrance ⁸⁵ ; and CC Inventory for Nurses in China (CCINC) of Cai et al. ⁹⁷

The 20 CC instruments were developed, adapted, and validated between 1987 and 2019. 11 of these were for use in the United States; two for Taiwan; and seven for use in Cyprus, Canada, Korea, Saudi Arabia, China, Italy, and Louisiana. The CC tools consist of 20 to 83 items.

Four out of the 20 CC tools consist of 20 items each. Three tools were developed for use in the USA, including the CCI developed for clinical researchers and research nurses ⁸⁸ ; the CCET validated for nursing students, ⁹⁰ and the CCET developed for nurses. ¹⁰⁰ One CC tool, the INCCQ, was adapted for Italian nurses. ⁹¹

ECSAI developed by Napholz ⁸⁹ for nursing students of the USA consists of 23 items. Two tools consist of 24 items each, including the CCAI-UIC developed for occupational therapy practitioners in the United States, ²⁸ and the CSK, adapted for public health nurses in Canada. ⁹²

There are three tools for assessing CC, each with 25 items. The IAPCC-R was developed for Registered Nurses in the USA, ¹⁰² the CCA developed for Hospice nurses and caregivers in the United States ⁹⁹ ; and the CS validated for Community Health Nurses in Pingtung Country of Taiwan. ⁹⁶

Eight CC tools consist of 26 to 72 items, including the 26 items of the CCCS developed by Almutairi and Dahinten ⁷⁹ for healthcare workers of Saudi Arabia; the 29 items of the CCINC adapted by Cai et al., ⁹⁷ for nurses and nursing students of China; the 30 items K-CCS-CNs adapted by Chae and Lee ²³ for nurses of Korea; the 36 items of CAS developed by Rew et al., ⁹⁵ for undergraduate and graduate nursing students of the USA; the 40 items of the CCAT developed by Vasiliou and Kouta ¹⁹ for community nurses of Cyprus; the 41 items NCCS validated by Perng and Watson ²⁷ for on-the-job nursing students of Taiwan, and the 72 items of CDQNE developed by Sealey ⁸⁰ for nursing faculty members of Louisiana.

Two of the 20 CC tools with 83 items were developed for use in the USA to the target group of students and teachers are the CCCET of Jefferys and Dogan, ⁹⁴ and TSET of Jeffreys and Smodlaka. ⁹⁸ The sections below raise conceptual frameworks, response options, and reliability of CC instruments.

The conceptual framework used by eight CC tools, such as IAPCC-R, CKS, CCAT, and CCAI-UIC, is based only on the Process of CC in the Delivery of Healthcare Services by Campinha-Bacote. The remaining four CC tools were highly informed by this model and with one or more additional frameworks. These include NCCS, which was based on Jeffreys’ Confidence Model and Literature Review; CCA was based on the CC Model of Schim and Miller; K-CCS-CNs was based on Padopoulos, Tilki and Taylor ⁶⁷ CC model, and CCCS was based on the Critical CC (CCC) model of Almutairi and Dahinten. ⁷⁹

CSES was based on the Consistent with Bandura Self Efficacy (Social Learning) Theory and Leininger’s Cultural Care Theory; the CCCET was based on Jeffery’s CC and Confidence Model and it is an assessment tool adapted from TSET; CCAT was based on the Papadopoulos, Tilki, and Taylor Model for Developing CC; TSET was based on Albert Bandura’s Social-Cognitive Theory; concepts of transcultural nursing; CAS was based on the Pathways Model and consistent with the Purnell Model of CC. TSET was based on Albert Bandura’s Social-Cognitive Theory and concepts of transcultural nursing. ^{94 , 98} CSES, CCCET, and CAS were based on the Purnell Model of CC. The theoretical basis of ECSAI, CS, and INCCQ was not reported.

Two instruments, such as IAPCC-R and CCAI-UIC, have four-point Likert-Type rating response options. Nine CC tools such as NCCS, CSES, CKS, CDQNE, ECSAI, CCET, K-CCS-CNs, INCCQ, and CCINC have a survey of a five-point Likert-Type rating scale. The CAS and CCET have seven-point Likert-Scale response options. One exceptional CC tool, CS, has response options in the interquartile range of 0-70. CCCET has a survey with a 10-point Likert-type scale with indifferent rating scales, and TSET has a survey with a 10-point Likert-type scale, for all items with 1—not confident to 10—totally confident.

CSES, CKS, TSET, CCA, CCET, and CCINC full-scale reliability ranged from 0.83 to 0.98, and sub-scales reliability ranged from 0.73 to 0.98. NCCS, CCAI-UIC, K-CCS-CNs, and CCCS sub-scales reliability ranged from 0.64 to 0.96, but the full scales reliability was not reported. On the contrary, the full-scale reliability of ECSAI, CAS, CS, and INCCQ ranged from .82 to .94, but the reliability of the sub-scales was not reported. However, both the full-scale and sub-scale reliability of CCCET, CCI, and CDQNE were not reported.

In subsequent sections, study findings on IRR for inclusion and exclusion of CC measuring tools, socio-demographic characteristics, CC sub-constructs, item screening, and evaluation of items were presented.

IRR of study constructs

The 20 carefully selected CC tools were further evaluated to select the final instruments, determine sub-constructs incorporated, identify major socio-demographic variables, select relevance items, and evaluate items. Table 3 depicts the aforementioned issues, and the IRR of each themes of the study. ⁷³

Selection of CC measuring tools

For the selection of CC measuring tools, the IRR of three experts’ ratings were computed. The results show that IRR was found to be between 75-87.5%. In other words, the raters’ disagreement ranged between 12.5%-25% owing to raters differing interpretation in evaluating tools that might be originated from lack of prior experience in executing similar tasks.

Eight CC tools, including; NCCS, IAPCC-R, TSET, CCAI-UIC, CCA, K-CCS-CNs, CCCS, and CCET have gained 100% agreement of the three raters. Whereas, four instruments, including; CCI, ECSAI, CCET, and INCCQ have gained 100% disagreement of inclusion by the three raters. As well, six instruments including CSES, CCCET, CCAT, CAS, CS, and CCINC have attained 67.7% agreement of two raters, and disagreement of one rater. Finally, two instruments, the CKS and CDQNE have gained 33.33% of agreement by one rater, and disagreement by two raters.

Identifying sub-constructs

The computation of IRR here is conducted for the major six factors incorporated in the remaining 16 CC measuring tools based on the agreement of inclusion by one or more raters. The IRR of CS is 38%, CD is 44%, CE is 50%, both CK and CA is 75%, and CSK is 81%.

Beyond raters’ agreement, IRR was computed with the exclusion of CAS and CCET due to unrelated sub-scales and limited number of sub-constructs respectively approximately reached 69%. Likewise, the percentage of each of the sub-constructs incorporation in these tools approximately ranged from 43% for CS to 93% for CSK. The CA and CK sub-constructs have equally incorporated by 86% in the aforementioned instruments. As well, CS, and CE were incorporated by 50% and 57% in the Reviewed instruments.

Socio-demographic variables

Regarding socio-demographic variables, the IRR was found to be 50-86%. From this, gender, age, religion, type of organization, department, language, professional category, education, language use, language skill, professional category experience, cultural diversity encounter, cultural training, and levels of cultural competence, ethnic self-identification, and performance evaluation result were decided to be incorporated to measure the socio-demographic characteristics of health workers in the cultural setting of Ethiopia.

Item selection

The IRR for the selection of items were computed similarly and found to be 59% for rater three, 62% for rater one, and 67% for rater two. The first three experts were agreed to keep TSET and IAPCC from the selected instrument to use as the framework of the subsequent study. Then, six CC measuring tools, including; the 41 items of NCCS, the 24 items of CCAI-UIC, the 26 items of CCCS (excluding the 17 items of critical empowerment), the 20 items of CCE, the 25 items of CCA, and the 33 items of K-CCS-CNs were included for the purpose of item selection. Therefore, the raters were expected to select relevance items from the total of 169 items compiled from the aforementioned six CC instruments. Based on this, the raters were agreed and selected 65 items to pass to the next section of evaluation and item screening (See extended data - appendix A). ⁷³

Item screening and evaluation: Rational of exclusion and inclusion of items

The two raters both agree on the inclusion of 46 items and exclusion of 13 items. Then, the computed IRR by Cohen’s Kappa was found to be 0.75. The IRR result should be agreed to be moderate reliability. The two experts again discussed on contextualizing items for inclusion and both agreed to incorporate 10 additional items. Following this, the screened 65 items have further evaluated to decide the final included and excluded number of items.

The IRR was computed using the quotient on the number of items violated test construction principles that already identified in Table 2 of this paper (n=12) and the number each rater identified. Based on this, IRR of rater one is found to be 50% and rater two is 67%. Based on agreed appraising criterion indicated in Table 2, both raters have arrived in a consensus to discarded 13 items out of 65 items. The 65 selected items from the six CC tools, items evaluated by experts, the final 56 included items, and the 11 excluded items are indicated in Appendices A, B, C, and D, respectively. ⁷³

The items such as ‘health care knowledge and the patient’s comprehension of interpretation of health/illness are usually different systems’, ‘even if a patient’s use or adoption of a treatment method differs from my professional knowledge’, ‘I usually don’t prohibit it’, and ‘knowledge of cultural differences about the decision maker in the family’ were automatically discarded due to violation of one or more item construction principles. ⁷³

Items such as ‘I can explain the possible relationships between the health/illness beliefs and culture of the patients’, ‘I do not feel that I have the skills to provide services to ethnic minority clients’, and ‘there are no cultural variations within a cultural group of people’ were rejected, because they are endorsed by every respondent. Items such as ‘I openly discuss with others’ issues I may have in developing multicultural awareness’, and ‘Awareness of culture’s impact on perception on health and illness’ were rejected because they are prone to multiple interpretations.

Whereas the item ‘cultural and linguistic differences could compromise healthcare provider’s well-being’ was rejected due to violations of both double barring and brevity principles. The item, ‘it is not important to assess a patient’s preferences in terms of healthcare services if I am knowledgeable about their culture’ was excluded on violation of item writing principles such as the item should be short or comprehensible and brevity.

The items ‘thinking what it would be like if I were a foreign patient’, ‘trying to say simple phrases in the foreign patient’s language when I care for them’, and ‘knowledge of cultural differences about sensitivity to pain’ were rejected due to lack of clarity, in appropriateness of the target group, and lack of both self-referent and clarity respectively.

As well, both experts of the third group agreed to accept 11 items with some minor modifications as per the nature of the items. Some of the items both experts agreed for inclusion with some modification by mainly incorporating self-referent phrases include: ‘I understand a patient’s cultural background can promote the quality of Health care’, ‘I can compare the health or illness beliefs among patients with diverse cultural background’, I have the Knowledge of cultural/religious belief that limits caring depending on gender’, ‘I use a variety of sources to learn about the cultural heritage of other people’, and ‘I have knowledge of cultural differences about touch or Space’.

Therefore, 13 items were excluded, 11 items modified for inclusion, and remaining 52 items passed for further evaluation.

Item wording to contextualize to the cultural setting of Ethiopia

The words or phrases in the items selected from six CC tools by replacing by contextual words or phrases to make items more appropriate for use in the health cultural context of Ethiopia. These words or phrases, including: ‘goals’, ‘most people’s’, ‘it is difficult replaced’, ‘race’, ‘a service provider’, ‘provider’s’, ‘compromise’, ‘people’, ‘all’, ‘members’, ‘AND’, ‘identify’, ‘school’, ‘relate’, and ‘foreign patients (FP)’ replaced or changed by ‘activities’, ‘culturally diverse patients’, ‘I face difficulties’, ‘health professional’, ‘health professional’s’, ‘hinder’,’patients’, ‘different’, ‘patients’, ‘OR’, ‘encounter’, ‘place’, ‘treat culturally diverse patients or patients whose ethnic is different from mine’ respectively. The noun ‘nursing’ has been replaced by ‘clinical’, and ‘health care’ depending on context. The word ‘client’ has been replaced by the ‘culturally diverse patient’, and ‘patients’.

The experts’ reasons to replace or remove words/phrases have included: to address lack of targets; to avoid items likely to be endorsed by everyone; to avoid items with multiple interpretations; to improve the clarity of items; to make items independent as possible; to avoid jargon words of items; to articulate the items intended factors to measure precisely; to restate items to keep proper arrangements, and to improve the manifestation of patterns of associations. As well, words like ‘only’, ‘usually’, and ‘completely’ that denote absoluteness of the items have been also removed. Furthermore, those items violated one or more principles of item constructions including double barring, lacks target group, might be open for endorsement, prone to multiple interpretations, caused brevity, included absoluteness words, stated as long sentences or with more than 20 words, lacks simplicity, stated to direct with idiomatic expressions, written too short, and proven lack of comprehensibility has been rewritten to improve the quality of the items.

These double barring items were rewritten either into two independent statements or stated as a single item based on the significance of the items. For instance, the item which stated ‘Can have appropriate verbal and nonverbal communications with foreign patients’ was rewritten as two statements; ‘I can have appropriate verbal communications with patients whose ethnicity or culture is different from mine; and ‘I can have appropriate nonverbal communications with patients whose ethnicity or culture is different from mine’ after the necessary modifications were made by experts. Another double-barring item, ‘I find ways to adapt my services to individual and group cultural preferences’ has been restated as a single item and written as ‘I find ways to adapt my services to individual cultural preferences.’

Lastly, the phrase ‘ethnic minority clients’ has been replaced by phrases such as ‘my culturally diverse patients’, and ‘patients whose ethnicity is different from mine’. On the other hand, the words or phrases like ‘behavior’, ‘ways’, ‘usually’, ‘completely’, ‘will’, ‘disability’, ‘being flexible, empathetic, and non-judgmental,’ and ‘group’ have been decided and removed.

Finally, 56 items were selected for further validation of the items to be included in the final version of the newly developed CC Scale to use in the context of Ethiopian health care settings.