A mixed-methods evaluation of capacity strengthening within an international conservation agriculture research consortium

Ethical statement

Ethical approval for this study was obtained from Liverpool School of Tropical Medicine’s Research Ethics Committee (REC), UK under approval number (LSTM REC 18-038). All interview participants provided written informed consent. No incentives were provided for participation.

Study design

A mixed methods study consisting of an online survey and semi-structured interviews (SSI), both conducted with CEPHaS consortium members from all partner countries in the final six months of the programme.

Study setting

The CEPHaS consortium sought to strengthen research capacity among a network of African and UK researchers, and their respective institutions, to fill knowledge gaps on the impacts of CA practices on the water cycle in cultivated soils. The consortium brought together a cross-national, multidisciplinary network of researchers who worked together at sites where African partners have long-established CA trials. These included the University of Zambia’s Liempe Farm, Chitedze Research Station in Malawi and Domboshava, Zimbabwe. Through collaborative planning, installing, monitoring and data interpreting at each site, consortium members were expected to develop 1) their understanding of cross-discipline contributions to problem-solving; 2) their research skills in cutting-edge methodologies; and 3) their generic research skills (e.g., in project design or academic writing). CEPHaS delivered a wide range of training across diverse subject areas to researchers and research support staff across the CEPHaS network and supplied specialist field and laboratory equipment to consortium partners. A learning-centred structure of the demonstration studies was utilised to enable participants to take the lead in establishing similar studies or experiments both during and beyond the CEPHaS project lifetime. Further information about the CEPHaS consortium can be found at: https://www2.bgs.ac.uk/CEPHaS/.

Sampling

The sampling frame for the survey included anyone who had been invited to attend and/or delivered a CEPHaS training between January 2018 and June 2021 and for whom an email address was held by CEPHaS project management (N=60). Survey participants were asked to indicate whether they would also be willing to participate in an SSI and, if yes, to provide their name and email address. Interview participants were purposively selected from this sample, with the objective of achieving representation across the final interview sample in terms of CEPHaS partner country, career stage, gender and position. An additional 12 CEPHaS co-investigators and staff at partner institutions, who had not been included in the survey sample, were also invited to participate in the SSIs, with the objective of ensuring representation from across CEPHaS leadership and from across a range of positions including scientific, managerial and technical. These individuals were nominated by CEPHaS project management and email addresses supplied.

Procedure

The survey was developed by the research team and piloted with African early career researchers not belonging to the sample frame to test face and content validity of the stated questions. Minor changes to wording and survey lay out were subsequently made. The revised survey was administered via the ‘Online Surveys’ platform (https://www.onlinesurveys.ac.uk/) and took approximately 10-15 minutes to complete. Participation was both anonymous and voluntary. Participants provided consent by selecting the ‘I consent to the survey’ option in order to access the rest of the survey. An information sheet was included with the initial survey invitation, which was sent via email with a link to the survey form. Two ‘reminder’ messages were sent, also via email. The survey remained ‘live’ online for a six-week period between June-August, 2021. Participants were able to complete the survey at any time during this period. The online survey consisted of four sections: 1) demographic and professional information; 2) investigation of uptake, utilisation and transfer of any training provided by CEPHaS as well as potential barriers and enablers to training utilisation and transfer; 3) examination of the use of any resources provided by CEPHaS as well as enablers and barriers to the use of these resources; and 4) broader exploration of participant experiences of CEPHaS participation. Survey structure consisted primarily of closed response questions with one or multiple answer options.

Prospective interviewees were invited to participate by email and, if they accepted, a suitable date and time for an interview was agreed. All SSIs were completed remotely, via a recorded Zoom call, between September 2021 and February 2022. Participation was voluntary in all cases, with each interview taking between 30-80 minutes to complete. Semi-structured interviews were informed by a topic guide which covered the same four sections as included in the survey, albeit in an open-ended format. Both the survey and interview guide were designed to elicit information pertaining to both the individual participant as well as their respective university or research institution. The two methods were considered complementary as they afforded breadth (survey) and depth (SSI) to the subsequent data analysis and allowed for data triangulation to improve reliability. All design and evaluation activities of this study were conducted by the lead author (KAD) and supported by a senior colleague (JP), based in the UK at LSTM, and both were completely independent of project activities. Survey design was reviewed by CEPHaS members (MM, AD) to inform contextual understanding and inclusion of key project areas for evaluation.

Data analysis

Survey data were exported from online surveys into Stata/SE V.14.1 (https://www.stata.com/) for analysis. RStudio, (https://posit.co/products/open-source/rstudio/) a freely accessible alternative, is capable of the same analysis used in this study. Univariate analysis was performed to describe characteristics of the sample and for calculating frequencies and percentages across key focal areas of training, including knowledge/skill gain and transfer, resource use, and CEPHaS activities experienced and perceived value. All interviews were audio recorded and transcribed in full. Transcripts were entered into NVIVO 12 (https://lumivero.com/products/nvivo/) for coding and thematically analysed using a framework synthesis approach,¹¹ informed by the interview guide. Taguette (https://www.taguette.org/), a freely accessible alternative, is capable of the same analysis used in this study. Key themes were identified and systematically reviewed. Key themes within and across consortium dynamics were examined in a series of discussions by two investigators (KAD, JP) at key intervals within the coding cycle. Preliminary findings were presented at the CEPHaS Conference in December 2021 to help interpret the findings in the context of the programme. Survey findings have been aggregated to further protect participant anonymity. Interview excerpts presented below have been labelled with relevant participant characteristics, including profession/career stage, location and gender, to aid interpretation yet maintain anonymity. Unique participant codes, ranging from P1-P19, are also presented against interview excerpts to indicate where the same participant has been quoted more than once.

Participant characteristics

A total of 40 respondents completed the online survey (response rate of 67%) and 19 participants completed a semi-structured interview. As shown in Table 1, survey and interview participants were predominantly male (77.5% and 78.9%, respectively), aged between 25-44 years (65% and 68%, respectively) and graduate or early career researchers (60% and 63%, respectively). Participants were relatively evenly spread between the four CEPHaS partner countries.

Survey findings

Of the survey participants, 72.5% (29/40) reported attending two or more training events provided by CEPHaS, 12.5% (5/40) attended one training, and 15% (6/40) did not attend a training event (i.e., they were invited, but did not attend). The 34 participants who reported attending at least one CEPHaS training were asked to report the frequency with which they used this training and how they had applied the training. Participants who attended more than one training were asked to respond based on the training they used the most. As shown in Table 2, 50% of these 34 participants reported using their training at least monthly and 29% at least weekly. The most frequent applications of the training received were in support of the participants’ own research (68%), supporting others research (50%) or teaching within their respective institutions (38%). ‘Other’ responses included consultancy work (n=1) or application within CEPHaS (n=2).

When asked ‘apart from teaching, have you transferred this training in any other way?’, 24 out of these 34 participants responded ‘yes’. Table 3 presents the various ways in which these 24 participants reported transferring knowledge/skills gained from a CEPHaS training. As shown, knowledge/skills transfer primarily took place in the context of student supervision or supporting/supervising institutional colleagues.

All participants who reported attending or transferring a training were asked to identify, from a list of specified response options, enablers or barriers to utilising or transferring CEPHaS training. As shown in Table 4, the most frequently reported enablers to utilising and transferring training were ‘training was applicable to my work’, (74% and 75%, respectively), ‘having the applicable knowledge/skillset’ (62% and 71%, respectively) and ‘sufficient training’ and ‘sufficient mentorship/support’ (both 53% and 54%, respectively). The most frequently reported barriers included ‘insufficient time’ (30% and 24%, respectively) and ‘insufficient access to equipment’ (15% and 9%, respectively). However, over 50% of respondents reported ‘no barriers’ to either the use or transfer of training.

All survey participants were presented with a list of resources supplied by CEPHaS and were asked to identify those to which: A) they had access to; B) had used; and C) had used most often (when more than one resource had been used). As shown in Table 5, most participants had access to, and had used, training materials, equipment and software supplied by CEPHaS; however, the resource used most often by 60% of participants was supplied equipment.

Participants were asked how frequently they used the resources supplied by CEPHaS and for what purpose. For participants who reported using more than one resource, they were asked to respond based on the resource that they utilised the most. As shown in Table 2, almost all participants used the stated resource at least once a month or more (92.5%) most often in support of their own research (77.5%) or teaching within their own institution (47.5%). ‘Other’ responses included for practice (n=1) or were undefined (n=2).

All participants were asked to identify, from a list of specified response options, enablers or barriers to utilising CEPHaS provided resources. As shown in Table 4, the most frequently reported enablers to resource use were ‘resource was applicable to my work’ (65%), and ‘having the applicable knowledge/skillset’ (57.5%). The most frequently reported barriers included ‘insufficient time’ (20%) and ‘insufficient training’ (10%). However, 67.5% of respondents reported ‘no barriers’ to resource use.

All survey participants were presented with a list of research-related activities and asked to identify which, if any, they had experienced during CEPHaS participation; they had experienced for the first time during CEPHaS participation; and had been most useful to them and their respective institution. As shown in Table 6, ‘data analysis’ (85%) and ‘field work’ (82.5%) were the most widely reported activities, with the former also the activity most often experienced for the first time (20%) and the activity considered most useful for the individual (27.5%) and their respective institution (25%).

Interview findings

Participants described a wide range of benefits from CEPHaS membership at both an individual and institutional level. Table 7 presents a summary of these stated benefits as attributed to training provided by CEPHaS, resources provided by CEPHaS or the broader experience of CEPHaS membership.

Interview participants reported a range of challenges that to some extent impacted on their, or their institution’s, ability to make optimal use of (some) opportunities made available through CEPHaS membership. Challenges identified included Covid pandemic travel limitations within and between countries and the subsequent loss of face-to-face interactions; misalignment of procedures amongst institutes, and between institutes and the funders, resulting in delays in project activities; and differing partner working and communication dynamics across institutes and time-zones.

“I think I know there is always a difference between virtual meetings versus physical meetings. Because that physical relationship normally at times is important in establishing networks, that’s how I look at it … You meet, you discuss, and then you are sat somewhere on a bench, you are talking and physically you can touch this person. Yeah, it’s different from a virtual, yeah, that’s…even when you are doing all sorts of training. I think sometimes that physical one adds a bit of value than a virtual one.”

Four primary themes that positively influenced the consortium experience for consortia members emerged from the interview data, including: resource provision, strategic integration, the partnership network, and inclusivity. We discuss each theme in turn below alongside selected illustrative quotes in support of some points.

Resource provision

The CEPHaS project provided a multitude of resources to consortium partner institutes, including funding, equipment, software, training materials, and procedural guidelines. African institutes were the primary beneficiaries of the equipment which they could retain indefinitely. Institutes received both field and laboratory geophysics and soil science equipment that were used to rebuild and in one instance, establish a new soil science laboratory unit. The receipt of this equipment was seen as both individually and institutionally beneficial across consortium partners. Individually, members benefitted from equipment that reduced the work and time required to complete research tasks. The new equipment mechanically completed tasks that previously would have been meticulously done by hand, easing the workload of researchers and providing time to complete additional tasks.

“I used to see most of the time they had a lot of samples in the lab that needed drying then they would just remain somewhere on the benches. But now this is not the case. If you want to dry something they have bigger capacities so they can be able to do that, and much more efficiently. So, I would say generally it has really made work easier. And for most of us it has also given us that opportunity to be able to do research which maybe in most cases would be a little bit difficult.”

Consortium members had utilized the equipment not just to conduct research for CEPHaS but research areas outside of the scope of CEPHaS, and where this had not already begun, members felt that this new equipment would enable them to expand the scope of their research and explore new areas. Likewise, having the equipment to explore new research areas was seen as a long-term benefit to the member institutes as it had the potential to expand its research portfolio. Northern consortia members, who did not receive this equipment, also saw this as beneficial to their personal research and research institutes as it meant that there were now institutes in other geographical locations that have well equipped laboratories for future collaborations and new research.

The receiving institutes were also utilizing the new equipment to support student research in individual research projects as well as existing formal courses. Consortia members at receiving institutes felt that the new equipment increased the institutes’ prestige by increasing student satisfaction, by attracting prospective students and by making them appealing to other institutes and funders as a potential research partner. Students could now have hands on experience applying the theories they learned in class, and where conducting research, students would no longer need to pay for samples to be processed at other institutions.

“Our students are also going to benefit a lot in terms of doing their practicals, to test some things in the lab … Prior to the project, our labs were not functioning well. For instance, I can mention the pressure plates. The pressure plates we use to measure the capacity of the soil to retain water were no longer functional. Thanks to the project, we managed to get those equipment that are very, very important. Especially for those students who are doing soil science and those who are doing agronomy. Because at some point, they would want to understand how the soil would be able to retain water after some amendments to it … so it’s very important for us as an institute.”

In some instances, this was also seen as an opportunity that could provide additional revenue streams through the provision of for-profit laboratory services which in turn would support local organizations and farmers that may benefit from these types of tests.

Strategic integration

The CEPHaS project was designed to capacitate institutes and individuals within the consortium and consciously incorporated complementary, integrated components into program delivery to facilitate this. Resources were provided with accompanying formal and informal training. Training started at the basics, facilitating a common scientific language across a multidisciplinary team and ensuring inclusion across demographics. Resource provision was intentionally multi-purpose. Equipment and software were provided not just to complete CEPHaS research activities, but also to establish laboratory facilities and field sites that could function as key CA research facilities in the region and provide institutes with equipment that could support revenue building services. The project framework was responsive to the context of the environment and the individuals and institutes. For example, in the initial phase of the project, it became apparent that the consortium partner institutes had different research management processes. Having aligned practices was required for the smooth transfer of funds and meeting funder requirements. The project was responsive to these needs, allocating time, resources and staff to support the training of institute staff not directly employed in CEPHaS to support project activities.

“But I think in terms of preparing the financial reports, I think people in our accounts system have benefitted. They are able to generate reports in accordance with the department or the funder. I think that experience is helpful for them when they implement other activities within the university but also even for any future or present projects in the institute.”

Staff included on the project were both permanent institute employees and employees hired specifically for CEPHaS, with the rationale to provide for the present needs of the project and capacitate sustainability beyond the end of the project life. Including permanent institute employees meant that the skills and knowledge obtained by project staff would be retained at the university and that teaching staff at teaching institutes would be able to facilitate knowledge transfer within the existing institute framework beyond the project life. Permanent institute employees also have other institute responsibilities and have time allocated across non-CEPHaS activities which could restrict availability. Employing staff outside of the institute allowed the CEPHaS project to have full time staff, while also capacitating recent institute graduates from the departments engaged in CEPHaS and other local organisations, allowing the project’s activities to support not just the consortium partners but other individuals and institutes within the local areas.

“Looking at our lab technicians, some of my colleagues where they’re academics people, they have been trained. That is a long-lasting knowledge that will be used for uplifting some of the issues that we do for our institution. For me also being trained in terms of the geophysics, it means I’m capable to say if somebody else brings in the issue of geophysics, I should be able to address and the institution will benefit or maybe they can be able to collaborate in another project.”

The project design included partners from the global north and the global south with the view to allow mutual learning across cultures and geographical context. The partners included multidisciplinary staff within the field of CA to allow for knowledge transfer across sub disciplines and allow specialists to see potential connections and develop a holistic understanding of the field. The partnerships included early-, mid- and senior career researchers. Within the project framework, staff were organized into distinct units called ‘Working Groups’ that allowed the incorporation of individuals from each of these demographics to support the inclusion of a multitude of perspectives and facilitate the creation of networks across institutes.

Partnership network

The partnership network created by the CEPHaS project was noted across participant members. The inclusivity, structure and duration of the project were key factors in supporting the network development. In addition to the diversity within the working groups, regular project workshops and trainings throughout the duration of the project and a range of communication mediums allowed for continuous formal and informal engagement between partners regardless of institute or geographic location. Members felt comfortable reaching out to other members through email with queries or requests for support and WhatsApp groups that allowed for both informal and formal communication. Members felt the network that was created through CEPHaS was something beneficial during the project and that it would continue beyond the life of the project, allowing for further development and research opportunities. Members felt comfortable that if queries or opportunities arose after the project, the CEPHaS members who supported them during the project would be willing to continue that engagement. These networks were seen as particularly important to early career researchers (ECRs) who could utilize this new network at a pivotal point in their career.

“And some of the contacts which I have, I think they are really important for the development of my career, which is crucial at this point as a junior researcher, I can say. I think I have long-term benefits participating in the CEPHaS … I can say some of the guys, I don’t hesitate to talk to them when I need any assistance or anything … But the network and the interaction include the whole CEPHaS team, which at times if I need anything, I can communicate with anyone without hesitation, of course.”

Members attributed this to trust that was developed throughout the project due to the continuous engagement between members, the length of the CEPHaS project, and the inclusivity that was generated through the project facilitation and leadership style (see below). Being a part of a project as long as CEPHaS was seen as a rare opportunity, and this allowed members to get to know working and communication styles as well as areas of expertise of other members. Additionally, although the CEPHaS project did not include PhD studentships, in at least two instances CEPHaS network membership led indirectly to scholarship opportunities for early career researchers.

“But I’d say even the scientists have learnt a lot from each other. Because as I say, the soil scientist is now appreciating the hydrogeologist, or the hydrogeologist and the shallow geophysicist are working together. They’ve instrumented a borehole and they’re both getting results from that borehole.”

Networks were facilitated across countries and supported mutual learning opportunities to develop cross-cultural professionalism and a cultural exchange of ideas to understand CA in different geographical environments.

Inclusivity

Inclusivity was a key feature of the CEPHaS project for consortia members. CEPHaS included both African and European institutes and individuals from a range of specialties and career levels. At the inception of the project, a workshop was held that was specifically designed to engage each member in the formation of the project plan and identify needs. Highly engaging project brainstorming activities around project deliverables were utilized. Perspectives and participation were encouraged by the project leads, facilitating a sense of value and respect amongst members regardless of position. Having this type of respect and seat at the table, from the beginning and throughout, was valuable to members and fueled their continued engagement.

“From the beginning, every member of the project was involved. We didn’t have a top-down thing. Rather, it was like, together, let’s achieve this. That was a great experience for me, how to arrange a large project of this nature, interdisciplinary or maybe cross-cultural, many countries, how to arrange it so that we could be able to work together and deliver.”

The sense of inclusion that was generated throughout the project was attributed in part to the project leadership. The leader of the project, working groups, and the administrative staff provided verbal encouragement and support to allow members of the consortium to feel empowered, regardless of age, position or nationality. This leadership style further inspired members to want to develop a similar leadership style and incorporate elements of this approach within their own project management and teaching.

“I think it was not like a forcing, it was an encouraging. It was flexible, motivating, allowing for people to think freely, like your ideas are welcome. That was important. Even when you bring an idea, you see your idea is being debated further and sometimes even taken up. Even trainings, people were allowed, can you suggest which are the trainings you would like to attend. So, people suggest a number of trainings.”

Another aspect that facilitated inclusivity and collaboration between members was the way members were grouped within the consortium. Small interactive working groups that reported within a defined consortia structure allowed everyone to participate. Division of tasks across demographics within working groups facilitated ongoing communication and meant all members were included and participating in the research and outputs.

This inclusive environment afforded opportunities to travel and engage in activities and discussions that were considered rare for ECRs.