Development and validation of a Knowledge, Attitudes and Practices (KAP) questionnaire for healthcare professionals on environmental sustainability in healthcare in Southern Africa

Study design and setting

The study made use of a descriptive research design to develop and validate a KAP questionnaire as an instrument to collect quantitative and qualitative data on the KAP of healthcare professionals regarding environmental sustainability in healthcare in Southern Africa.

Stages of the study and development of the questionnaire

The development and validation of the KAP questionnaire consisted of a six-stage process:

Study population and sampling

While developing and validating the instrument, the researchers recruited experts to participate in stage two (n=9) and stage six (n=7) of the study. These numbers were recommended by multiple comparable studies as a requirement for the expert panel to establish content validity.^22–24 Non-probability purposive- and snowball sampling were used in the selection to identify and select experts appropriately based on the study’s inclusion criteria.^25,26 The experts were recruited through social networking, using LinkedIn or ResearchGate, where both were identified as professional or academic social media platforms. To be considered as an expert, participants had to demonstrate expertise in their respective fields where sufficient expertise was defined through one or more of the following: holding academic qualifications at master, doctoral or higher level; being a researcher in a relevant subject field; having relevant publications and/or clinical experience. Here, relevant fields included: KAP studies/instrument development/validation studies, environmental sustainability, the Southern African context and healthcare from either of the following professions: physiotherapy, occupational therapy, speech-language pathology, audiology, human nutrition and dietetics, or nursing. Unlike the mixed-methods study of 2021,²¹ the profession of human nutrition and dietetics was included in this study as this is a prevalent profession in delivering healthcare.

In stage four, stratified and snowball sampling were used to select the pilot-study participants.²⁷ The pilot-study participants had to be healthcare professionals in Southern Africa, working in any speciality of their profession, including physiotherapy, occupational therapy, speech-language pathology, audiology, human nutrition and dietetics, and nursing; practising for at least six months or working in academia. Emphasis was placed on recruiting participants from all the above-mentioned professions as well as all Southern African countries were represented among the included pilot-study participants.

The research team initially aimed to source 18 healthcare professionals as pilot-study participants, specifically two from each Southern African country. After distributing 34 ‘request to participate’ emails and sending numerous ‘follow-up’ emails, the research team received responses from 21 healthcare professionals who were willing to participate. The pilot-study participants were requested to complete the questionnaire and provide feedback. Despite sending reminders via email, only 13 pilot-study participants followed through with this.

Developing questionnaire and item clarity

During the initial expert evaluation (stage two) and the pilot study (stage four), participants were requested to evaluate the entire instrument in terms of layout, instructions, length and/or time taken to complete the questionnaire. The experts were also requested to indicate whether they used a cellphone/mobile phone or laptop/computer to complete the questionnaire such that the research team could identify problems in completing the questionnaire on different common devices. The pilot study participants were divided in half, whereby seven of the pilot-study participants were requested to complete the questionnaire on their laptop/computer and the remaining six were requested to complete the questionnaire on their cell phone/mobile phone. This was done to ensure the accessibility of the questionnaire through various devices. Additionally, the experts also had the opportunity to provide any additional input regarding the content and the format. The pilot study participants were requested to provide overall feedback on the clarity and formulation of the questions.

During stages two and six, the experts followed a four-point satisfactory scale in terms of how satisfactory they considered each questionnaire item to be in measuring the construct and/or purpose of the research study: 1) unsatisfactory; 2) somewhat satisfactory, 3) quite satisfactory and 4) highly satisfactory. Additionally, they were requested to provide input as to how each item could be improved or rephrased if they provided a rating of one to three. This feedback was, however, only incorporated during stage three of the study and not in stage six. This data is still available for further questionnaire development.

The questionnaire was uploaded onto Qualtrics, a web-based tool, for data collection that allows for the creation of questionnaires and the analysis of their results.²⁸ This provided a link sent to participants to enable them to complete the questionnaire.

To receive feedback from the participants in the different stages, the researchers used Microsoft Word and Google Forms. During stage two, the information was electronically completed in Microsoft Word and tabulated into a Microsoft Excel document in stage three to ensure that the data could be easily accessed and compared. During stages four and six, feedback was requested on Google Forms, which automatically allowed the results to be analysed in Microsoft Excel.

Evaluating content validity

The pilot study participants provided feedback regarding the clarity and formulation of the questions and the overall content of the questionnaire. The questionnaire was rated by the experts in both stage two and stage six. The item content validity index (I-CVI) refers to the average expert ratings of each questionnaire item. I-CVI ranges from zero to one and expresses a portion of agreement on the relevancy of each item.²⁹ The scale content validity index (S-CVI) was calculated using the average S/CVI (S-CVI/Ave) method, where the sum of all I-CVI scores is divided by the total number of questionnaire items.

In stage six, the final questionnaire was returned to the experts for a second and final round of feedback. Unfortunately, only seven out of the nine experts were available. In stage seven, following the calculation of both I-CVI and S-CVI/Ave, five questionnaire items were removed as they were rated with an I-CVI of lower than 0,79. One questionnaire item which received a rating of 0,714 was retained following researcher discussions, due to its relevance. The remainder of the questionnaire items all received an I-CVI greater than 0.79 (> 0.79). The final S-SCI/Ave was greater than 0.9 and ensured excellent content validity.²⁹

Data analysis

Data analysis occurred in stages three, five and six of the questionnaire development. In stage three, the first round of expert feedback relating to each questionnaire item and the questionnaire as a whole was carefully analysed and discussed by the research team before making changes. In stage five, a similar approach was used in incorporating the feedback from the pilot study. In stage six, the experts rated the questionnaire items according to a four-point satisfactory scale; these ratings were used to determine the questionnaire’s content validity.

Stage one: Item generation and questionnaire design

In this stage, the research team evaluated and reviewed the results of the South African mixed-methods study.²¹ The KAP questionnaire used in this study consisted of 14 close-ended and 11 open-ended questionnaire items. A total of 203 participants started the questionnaire, but only 100 participants completed the questionnaire, indicating a somewhat poor response rate.²¹ In hindsight, this was considered as being a potential result of overly difficult knowledge-based questions and having too many open-ended questions. Next to sourcing additional and updated literature, the research team used the questionnaire items from the previous study as a basis for developing their pool of questionnaire items. The qualitative data was used to identify prevalent themes and categories. After this process, 109 potential questionnaire items were availed. To reduce this number and ensure relevance, the research team evaluated each questionnaire item individually. This iterative evaluation process was continued until agreement was reached on the final 29 questionnaire items, excluding the demographic section at the beginning of the questionnaire. Section one consisted of 13 knowledge-based questionnaire items, section two consisted of five attitudinal questionnaire items, and section three included six questionnaire items pertaining to environmental sustainability practices. The barrier and educational sections contained three and two questionnaire items, respectively.

Stage two: Evaluation by an expert panel

The nine experts recruited fulfilled various combinations of the expert inclusion criteria. For instance, one of the experts demonstrated expertise in environmental sustainability, the Southern African context and as a healthcare professional. Of the nine experts, seven were healthcare professionals, two demonstrated expertise in KAP study/instrument development/validation studies, four demonstrated expertise in environmental sustainability, and two demonstrated expertise in the Southern African context. Regarding the demographics of the experts, six were female, and three were male, all ranging between the ages of 33 and 61.

The research team interpreted and analysed the expert ratings and the feedback that was provided. This process was methodical in that each questionnaire item, along with the expert feedback, was discussed individually. Before making the changes to the questionnaire items, the research team reached a consensus for amendments, removal or other measure to be taken for each questionnaire item.

Stage three: Validation and incorporation of changes from the expert panel

During this stage, the I-CVI and S-CVI were calculated on the first draft of the questionnaire to determine a baseline for later comparison. As seen in Table 1, unfortunately, two questions were not evaluated by one of the experts. The S-CVI for section one was calculated at 0.749; for section two at 0.978 for section three at 0.924, section four at 0.963 and section five at 0.944. The S-CVI/ave for the whole questionnaire was 0.860.

Following this, the feedback was analysed and then used to change the questionnaire. Regarding the formulation of the instructions, one expert suggested that the phrase ‘based on your own knowledge’ be added to the instructions of the true or false questionnaire items in the knowledge section so that the participants understand that they are not expected to research their responses from a reference source, but to rather use their current knowledge.

Multiple experts noted that questionnaire items 1.1b, 1.1c, 1.1f and 2.1a contained too many theoretical constructs or different terminology, which may conflict with each other and therefore argued that this could confuse participants regarding what aspect of the questionnaire item they should refer to in answering it. For example, questionnaire item 1.1b originally read: ‘The impacts and risks of climate change on mental health are rapidly accelerating’. The expert argued that ‘impacts and risks (are) not necessarily the same thing’. In response to this, the research team confined the questionnaire items listed above to one construct as far as possible, with for this question, the final item reading ‘Climate change has an impact on mental health’.

One expert indicated that it could not be assumed that all participants completing the questionnaire believe that climate change has man-made origins with catastrophic effects. At the same time, participants can have certain beliefs whilst still being advocates of environmental sustainability. Therefore, the research team added an option to select ‘non-applicable’ in response to certain questionnaire items. Experts suggested that the option of ‘other’ be included for the questionnaire item ‘Climate change has a direct negative impact on human health, through which of the following?’ because some experts recognised additional negative human health impacts of climate change. Another expert noticed that two questionnaire items in the practice section, namely Q3.2 ‘Regardless of whether your place of work has a policy or not, which of the following strategies to improve environmental sustainability in healthcare have been implemented at your place of work?’ and Q3.5 ‘Which of the following environmentally sustainable measures for your place of work would you support?’ each listed different answering options for selection. The expert suggested that the same list of options be provided for both questionnaire items as this would create more flow. Therefore, the options were combined and edited so that they were applicable to both questionnaire items.

For the questionnaire item ‘What would be your preferred method of being educated on this topic as a practising professional?’ it was suggested to provide the option of ‘Continuous professional development (CPD) activities’ as this could be a means of gaining knowledge on the topic.

The expert panel further suggested that certain questionnaire items should be rephrased for greater clarity. For instance, it was suggested that the statement ‘environmental sustainability practices should be incorporated into healthcare’ was not specific enough, so the researchers amended this questionnaire item by specifying ‘into healthcare services’. It was also suggested by multiple experts that ‘in your practice’ should be changed to ‘place of work’ to avoid ambiguity of the term ‘practice’, which is often related to ‘private practice’ and thus not applicable for those working in academia or a hospital. Some experts also suggested that a section should be created at the beginning of the questionnaire where definitions for key terms such as climate change and environmental sustainability are provided. The research team decided against this as it would indirectly disclose the answers for the knowledge section in advance and, consequently, participants’ knowledge would be inaccurately represented.

An expert also noticed that one could not backtrack between the questionnaire items. This expert suggested that for easier navigation and to allow participants to return to previous questionnaire items to provide the option to change their answers, backtracking should be allowed. The research team agreed with this suggestion and activated the backtracking function. A different expert recommended using larger text boxes where participants are allowed to type, such as in the case of selecting the option of ‘other’ for open-ended additions. The questionnaire was adapted accordingly. Although more than one expert proposed the use of Likert scales in more than just the attitudinal section, the research team decided against this recommendation in favour of the aim to derive rich information from participants, which would be limited by using a Likert scale. Minor spelling, numbering and punctuation errors were also highlighted and amended.

Several experts also suggested that the questionnaire items be more specific to ensure that participants understand what is being asked. Therefore, additional information was provided in brackets after certain questionnaire items and/or options. As requested, all acronyms were discarded and replaced by full terms.

Finally, all overall feedback was discussed and used to adapt the instrument and make necessary changes to individual questionnaire items. After the changes were incorporated, the questionnaire consisted of 11 knowledge, five attitudes, eight practice, three barriers and two education questionnaire items with a total of 29 questionnaire items.

Stage four: Pilot study of the target population

Literature pertaining to the design and conduction of KAP questionnaires indicates that a small number of pilot-study participants (five to ten) are recommended as volunteers from the target population.²⁶ In this study, thirteen pilot-study participants provided feedback. They consisted of two participants from Zimbabwe and Lesotho, respectively; three participants from South Africa and Namibia, respectively; and one participant from Angola, Eswatini (Swaziland) and Mozambique, respectively. The pilot-study participants consisted of healthcare professionals in various fields, namely five nurses, four occupational therapists, one physiotherapist, one speech-language pathologist, one audiologist and one professional from human nutrition and dietetics. Six males and seven females between the ages of 24 and 66 participated in the study.

Stage five: Incorporating feedback from the pilot study

In the feedback, all of the pilot-study participants stated that the questionnaire was easy to understand and clearly formulated and that they understood what was expected of them when answering the questionnaire items. Of the 13 pilot-study participants, 11 indicated that it did not take them longer than the approximate time stated, that being 15 to 20 minutes, to complete the questionnaire. Therefore, no changes were made to reduce the length of the questionnaire. Four pilot-study participants raised concerns with regard to the terminology used and suggested that simpler words be used to ensure understanding for those less proficient in the English language. Therefore, the research team examined the entire questionnaire and made necessary adjustments to ensure comprehension.

At this stage, the research team realised that participants from countries that do not have English as an official language, such as Angola and Mozambique, which both have Portuguese as their official language, would have difficulty understanding the questionnaire. Therefore, questions were included in the demographic section, instructing the participants to indicate their first (home) language as well as to rate their proficiency in English according to the provided options of poor, fair, good or excellent. No changes were made to the number of questionnaire items in this stage.

Stage six: Final review by the expert panel

Final validation and review of the questionnaire was performed by the expert panel. Unfortunately, two experts from the first round of feedback were unavailable for the final round of feedback due to other commitments. Therefore, the final expert panel consisted of seven experts (n=7).

Stage seven: Validation, item removal and final validation of the questionnaire

A statistician used the final expert ratings to calculate the Item-Content Validity Index (I-CVI) and the Scale-Content Validity Index (S-CVI). After the final evaluation by the expert panel, four questionnaire items from the knowledge section and one from the practice section, with ratings below 0.79, were removed from the questionnaire. An additional questionnaire item from the educational section also received a rating below 0.79. This questionnaire item required participants to select all methods that they consider applicable: ‘What would be your preferred method of being educated on this topic as a practising professional?’ The research team analysed this questionnaire item, and decided to retain it based on the shared conclusion that it would provide important information which could be used for intervention strategies in future studies.

As illustrated in Table two below, after calculating the S-CVI/Ave, the developed questionnaire has reached excellent content validity with a rating of 0.922. The S-CVI/Ave for each section was also calculated; the only section with a rating lower than 0.900 was the educational section with a rating of 0.857.

Notably, in the final expert feedback, some questionnaire items received lower S-CVI ratings than in the first round of feedback. Despite this, it was determined by comparing the S-CVI/Ave ratings from the first round of expert feedback (Table 1) to those from the final round of expert feedback (Table 2) that the validity of the questionnaire improved; with an initial S-CVI rating of 0.86 and a final rating of 0.922.

Limitations

Despite extensive recruitment, the research team could not obtain any pilot study participants for Botswana and Zambia. Therefore, the questionnaire is not validated for these two Southern African countries. Additionally, two of the nine first round of experts (stage 2) were unavailable to assist in completing the validation of the questionnaire during the second round of expert feedback (stage 6). The research team also considered that some of the pilot-study participants did not have English as their first language and that the comprehension and interpretation of answers could have been misconstrued due to this language barrier.

Recommendations

It is recommended that the questionnaire be validated in other regions across the world, commencing with the remainder of the countries in Sub-Saharan Africa, to determine its applicability across a wider geographical area. Together with this, it will be important to translate the questionnaire into the respective languages of each target population.

It is hoped that the questionnaire will be used by researchers from the respective countries in Southern Africa to determine a baseline of information from which various responses aimed at improving environmental sustainability in healthcare can be developed. Additionally, the employment of this can be a means of developing partnerships across Southern African countries - to share ideas, experiences, knowledge, and policies and implement changes.