Rapid mixed-methods evaluation of the Maternity and Neonatal Independent Senior Advocate (MNISA) role pilot: A study protocol

Ethical approval

This study is a collaboration between researchers from the NIHR RSET,⁵⁷ Rapid Service Evaluation Team (RSET) public contributors, and Sands. Sands⁵⁸ is a third-sector organisation that works to support those affected by pregnancy loss or the death of a baby and to prevent future deaths. Sands will collaborate on this study and will be closely involved in the design of the study, data collection, analysis (together with the research team), and writing.

This study was reviewed and given a favourable opinion by the NHS East of England – Cambridgeshire and Hertfordshire Research Ethics Committee [REC reference: 24/EE/0205] and received Health Research Authority approval. The protocol was peer reviewed by two reviewers external to UCL.

To ensure the appropriate support and safeguarding of participants in this study, we used safeguarding and distress protocols developed specifically for this study (see Figure 2 for a summary of safeguarding and distress protocols).

Data management

Data was managed in line with legal and regulatory requirements, including the General Data Protection Regulation (GDPR), the Data Protection Act (2018), and necessary research approvals.

Study oversight

The research team met every two weeks to discuss the project progress. The project leads (HW/NC/JS) also reported regularly on progress to the wider RSET⁵⁷ Executive Management group.

We set up an independent project advisory group to provide independent advice, governance, and feedback on the project at all stages. This group includes a wide range of stakeholders, including those within maternity and neonatal organisations (system level, regional level), regulatory bodies, third-sector organisations, professional bodies, clinicians, and academics. We met with this group 2-3 times during the project.

Patient and public involvement and engagement (PPIE)

Our project team includes three PPIE contributors (RM, SF, and CM), who have actively been engaged in weekly, and more recently, bi-weekly, project team meetings, and wider meetings to discuss and develop the study and provide written input intermittently. They will continue to attend meetings and contribute to all aspects of the study (including interpretation, write-up, and dissemination of findings). We also sought feedback from NHSE service user representatives on the study design and development.

Additionally, we held a wider PPIE workshop with five members of the public who had experienced adverse outcomes within their maternity or neonatal journey and/or investigation and complaint procedures. This workshop focused on the study of data collection tools and safeguarding processes. We held wider PPIE workshops to inform the key stages of the project, such as findings and dissemination.

Feedback from our PPIE contributors had informed many changes to our study including ‘thank you’ vouchers for family participants, expanding the type of interviewees included in the study, data sources, and considerations relating to equity, diversity and inclusion. PPIE contributors provided lived experience insight into the process’ navigated by families after an adverse event and the identification of relevant stakeholders. PPIE input has provided a more diverse recruitment strategy for interviewees and has enhanced the tailoring of support for those families who seek to engage in the study, offering more flexible and inclusive interview and submission options for families (including the option to input via verbal interview, writing, or a combination of both). The PPIE feedback and outcomes were recorded using a PPIE study log. PPIE contributors were supported throughout the study by PPIE leads (PLN, RM) and project leads (NC, HW, JS) to ensure that any needs and/or preferences were considered.

Equity, diversity and inclusion (EDI)

We applied the NIHR Rapid Service Evaluation Team (RSET) EDI project-specific assessment tool at three stages during this project: (i) during the development of the project evaluation, (ii) during data collection and analysis, and iii) following data collection and analysis. The assessment tool covered EDI considerations throughout the entire project, including when building the initial team, drawing on published EDI frameworks to consider EDI aspects relevant to the evaluation during the discovery and scoping phases, protocol development, stakeholder engagement, recruitment, data collection, data analysis, and dissemination. The study team met every six weeks to discuss and review EDI progress.

To ensure that our project thoroughly considered issues of EDI, we held several broad scoping conversations with stakeholders (such as policymakers, third sector organisations, clinicians, professional and regulatory bodies, and academics) to identify potential areas of inequity that may exist within this role in relation to the service. These conversations highlight that some groups of parents were more likely to experience adverse outcomes due to ethnicity and deprivation-related characteristics. This highlights the need to ensure that we considered within this evaluation whether the service reached those groups, potential barriers, and how they experienced the service.

We considered research questions related to EDI (see Table 2). We took steps to encourage diverse public voices to participate as PPIE contributors throughout this study. We also discussed with stakeholders how to best develop a recruitment strategy, data collection materials, data collection, and data analysis processes that will enable a wide range of families to participate in this evaluation.

Design

This was a rapid mixed-methods study combining qualitative (workstreams 1-3), quantitative and health economic approaches (workstream 4) to evaluate the implementation of MNISAs in England. An integration workstream (5) brings together the qualitative and quantitative findings (see Figure 1 for a summary).

Figure 1. Summary of evaluation workstreams.

Qualitative workstreams (1-3): Implementation and experience

Aims

Within these qualitative workstreams, we aimed to evaluate: i) family experience of the MNISA service (including those who have received and those who were eligible to receive but have not received support), ii) service development and implementation, and iii) staff experience of the MNISA role (including MNISAs and wider staff ).

Design

Qualitative design, comprising semi-structured interviews, documentary analysis and workshops.

Sample

Site sample and recruitment.

The MNISA role was piloted in 16 Integrated Care Boards (ICBs) in England. All 16 ICBs were invited to participate in this study. All 16 ICBs were included in the study (family and MNISA interviews). Additionally, 10 of the 16 ICBs were selected as in-depth case study sites (family, MNISA, and wider staff interviews). We purposively sampled these case study ICBs to ensure their representation with a range of characteristics, such as the model of MNISA employment used in that area (e.g., ICB employed or external organisation employed), geographical area, single/multi-trust ICB, time since implementation, and level of oversight/engagement from the ICB.

Individual sample

Please see Table 3 for a summary of our sample, and eligibility criteria.

For the family interviews, we aimed to conduct interviews with 15-35 parents and/or guardians who had received support from a MNISA and up to five interviews with parents and/or guardians who would have been eligible for support from a MNISA but had not received or accessed the service. While we aimed to recruit 40 families with diverse characteristics, recruitment was dependent on several factors, including the number of families and the diversity of families who had accessed the service (due to the infancy of the service) and the number of families who were willing to be interviewed. We aimed to purposively sample families across a range of characteristics, including the range of adverse outcomes experienced, family role (e.g., mother, father, or other guardian), age, geographical area (to cover the range of ICBs included in the pilot), ethnicity, length of time engaging with the MNISA, stage in their journey following the outcome, disability, and sexuality (see Table 3 for details of eligibility criteria).

For the staff interviews, we aimed to conduct: i) interviews with 3-8 national stakeholders (e.g., those working in policy roles, commissioning roles (including at ICB’s who initially signed up for the pilot but did not continue to implement the role), programme leads, regulatory roles (e.g., CQC), public inquiry and review teams, professional organisations, third sector organisations, and training organisations), ii) individual interviews and a workshop with up to 20 MNISAs from the 16 ICBs (currently or previously in post), and iii) interviews with up to 30 staff who work with MNISAs from the 10 case study ICBs (up to three staff per ICB, spanning network/regional level, ICB level, and trust level staff ). For the wider staff interviews, flexibility in the sampling approach was informed by scoping work, which indicated that those best placed to discuss the MNISA’s role will vary across ICBs. Network or regional level staff roles included operational delivery network staff, local maternity and neonatal system staff, maternity and neonatal safety investigation staff, and child death overview panel members. ICB level staff included line managers and maternity and neonatal voice partnerships (MNVPs). Trust level staff included those involved in developing local policy, midwives, nurses, bereavement teams, medical examiners, family liaison coordinators, those dealing with maternity and neonatal complaints, or those involved with incident reporting processes.

For the documentary analysis, we sampled relevant national and local documents pertaining to the role and implementation (e.g. operational guidance, communications toolkit, relevant reports relating to the set-up/design of the role, and training materials).

Recruitment

See Table 3 for a summary of our recruitment processes.

For the family interviews, participants were identified using a range of different recruitment processes, including identification through the participating 16 ICBs, and recruitment via social media and third sector or advocacy organizations.

Families (individuals or dyads, e.g., both parents) who were interested in participating in the evaluation were asked to contact the researchers via email or telephone. After this initial point of contact, the researchers (NC/HW/RL/NH) asked some basic eligibility questions (whether they had received support from a MNISA, which ICB they were in, and whether they were over the age of 18) and provided an information sheet and consent forms. We offered the option for information sheets and consent forms to be translated if needed. Potential interviewees were informed that their participation is completely voluntary. If interviewees were interested in participating, they were asked to provide electronic, written, or audio-recorded verbal informed consent prior to participating in the interview. Prior to the interview, the participants were asked to complete a short socio-demographic survey.

Staff interviews (MNISAs, wider staff, national stakeholders):

For national stakeholder interviews, researchers (NC/HW/RL/NH) identified relevant organisations or job types that may have a view of the MNISA role. Researchers emailed these organisations to ask for expressions of interest from their organisation/team in taking part in an interview.

For MNISA and wider staff interviews, researchers (NC/HW/RL/NH) identified relevant organisations, teams, and job roles and emailed a point of contact for these organisations, teams, and job roles to ask them to share details of the study recruitment with relevant members of their teams. Additionally, staff may cascade the details of the study with other staff members who they think would be eligible/interested in. Interested individuals were asked to contact the researchers to express their interest in participating.

For all staff interviews, researchers sent information sheets and consent forms to those who express interest. Potential interviewees were informed that their participation is voluntary. If interviewees were interested in participating, they were asked to provide electronic, written, or audio-recorded verbal informed consent prior to participating in the interview.

MNISA workshop:

For the workshop (MNISAs), researchers emailed all MNISAs to check whether they would be happy to participate in a workshop for the study. MNISAs were sent an information sheet and consent form in advance and asked to provide electronic or written consent before the workshop.

Measures

To ensure sensitivity, appropriateness, and comprehensiveness, semi-structured interviews and workshop topic guides were developed for this study, following conversations with relevant stakeholders during scoping, input from our patients and public involvement members, and input from Sands. (see Table 4 for a summary of topics covered.)

Prior to the interviews, all participants were asked to complete a sociodemographic questionnaire. For families, this included gender, age, sexual orientation, area of the UK in which they live, ethnicity, first language, relationship status, and category of adverse outcomes. For MNISAs, this included their employer, previous experience/background (e.g., clinical/non-clinical), training received, number of families working with, length of time in the role, type of contract (e.g., fixed term or zero hours), length of time working with families, whether they think the role is meeting NHS England’s objectives, and whether they had seen change at individual or organisational levels (and to provide examples). For wider staff and national stakeholders, this included the role and length of time in the role.

Data collection

Family interviews:

All participants were offered the option to participate in the interview on their own or with a family member (either in a joint or separate interview). Participants were offered the option to participate in the interview online (via MS Teams), over the telephone, or via a written response via a secure electronic survey link (REDCap). The written response option is consistent with similar projects that have offered alternative methods of interview participation.⁵⁹ Interviews were scheduled at a time of day which was agreed upon by the participants. If English was not a participant’s first language, we offered interpretation services so that they could participate in the interview. Each interview was jointly facilitated by a researcher from Sands (JH) and a researcher from RSET (NC/HW/RL/NH). Participants were informed of the interview arrangements prior to the interview, and the arrangements were flexed according to preferences. We offered participants the option to have an informal call with the researchers prior to the formal interview to provide an opportunity for them to share their experiences and events leading to the adverse outcome (should they wish to). This lasted between 20-30 minutes and was not part of the formal data collection, and not recorded. The formal interview, which focused on the experience of the MNISA role, lasted between 45-60 minutes and was recorded. However, if participants chose not to have two separate conversations, we explained to them that interviews would take between 30-90 minutes depending on how much they would like to say. Any interview responses via an electronic survey were directly returned to the study team for analysis electronically through REDCap. Participants were informed that they are free to withdraw for up to two weeks after the interview date. If families would like to provide further information following an interview, we offered them a short follow-up interview or gave them the option of adding any additional information via a secure electronic survey link.

As a thank you for participating in the interview, the families were given a one-off £25 voucher. This was deemed appropriate following the feedback from our PPIE contributors.

As the interviews involved family members potentially discussing distressing and/or traumatic events and adverse outcomes, we implemented several data collection protocols to support and safeguard families (see Figure 2 for a summary of our safeguarding and distress protocols).

Figure 2. Summary of safeguarding protocols for families, staff and researchers.

Staff interviews (MNISAs, wider staff, national stakeholders):

The interviews were conducted either online or over the telephone. Each interview lasted between 60-90 minutes. Interviews were scheduled to take place during regular working hours, as staff were not being compensated for their study participation.

All interviews:

If interviewees (family, MNISA staff, or wider stakeholders) wanted to provide further information following the interview, they could send further feedback in writing, or researchers would arrange a short follow-up interview. Participants were informed that they were free to withdraw for up to two weeks after the interview date.

Data collection protocols were implemented to support the staff participating in the study (see Figure 2).

MNISA workshop:

The in-person workshop lasted between 90 minutes and 2 hours. Researchers took detailed notes during the workshop to capture the key findings. Participants were informed that while they can withdraw from the discussion, any data provided until that point would be retained, as it would not be possible to remove individual data from a group discussion.

All interviews and workshops were audio-recorded on an encrypted Dictaphone (subject to consent), transcribed verbatim by a professional transcription service, anonymised, and kept in compliance with the General Data Protection Regulation (GDPR, 2018) and Data Protection Act (2018). The data are stored securely on the UCL Data Safe Haven. Only named researchers have access to participants’ personal data.

Data analysis

We used a medium Q thematic analysis approach,⁶⁰ combining inductive thematic analysis and the use of a coding framework.⁶¹ Data collection and analysis was carried out in parallel using Rapid Assessment Procedure (RAP) sheets.⁶² Qualitative data was be analysed by named researchers in the Rapid Service Evaluation Team (HW/RL/NC/NH) and Sands (JH/BW). Researchers conducting interviews would take real-time notes and input them into Rapid Assessment Procedure sheets following each interview. The categories used in the RAP sheet were based on interview topic guides. There was flexibility in adding categories during the research process. Once notes have been input into the RAP sheets, researchers will use inductive thematic analysis⁶¹ to inductively code these notes and develop initial themes and sub-themes. This rapid analysis was used to share interim findings with key stakeholders throughout the study. Following this rapid analysis, an in-depth analysis was conducted. Researchers used the initial themes and sub-themes developed during rapid analysis to develop a coding framework. This coding framework was applied to interview transcripts. Coding will be used to develop the final themes and sub-themes relating to the research questions.

We aimed to undertake cross-case comparisons across different ICBs and different characteristics (e.g., to explore barriers/inequities relating to access, use, and experience). Interpretation of findings and write-up will be discussed and agreed with the study’s PPIE group, advisory group, and project team prior to finalising.

Theoretical frameworks for the implementation of healthcare services/innovations were considered to align with the workstream research questions and objectives prior to data collection and analysis to guide the evaluation. Examples of frameworks that were considered include RE-AIM⁶³ (a framework that includes five key outcomes: reach, effectiveness, adoption, implementation, and maintenance) and CFIR⁶⁴ (Consolidated Framework for Implementation Research).

Quantitative workstreams (4): Impact and cost

Aims

This workstream aimed to explore the potential impacts and cost of the MNISA role and make recommendations about data capture and analysis for future evaluation (see Table 2 for the research questions).

Design

An observational study of MNISA caseload data, national electronic health records, other secondary data sources, and findings from qualitative interviews and workshops (see workstreams 1-3). The health economic study evaluated the financial impact of the existing pilot programme for MNISA support of families who have experienced eligible adverse outcomes from maternity and neonatal health services, and to explore the cost implications of a wider roll-out of the programme, changes in the eligibility criteria for MNISA support, and scenarios on the impact of MNISA support on patient safety.

Sample

All cases were supported by MNISA’s that are reported in the bespoke NHSE data collection compared to cases reported in hospital records (Hospital Episode Statistics) or other relevant datasets.

Measures

Integration of qualitative and quantitative methods (Workstream 5)

All the workstreams (1-4) inform each other. The collected data and findings were triangulated.⁶⁵ The mixed-methods approach will be beneficial as it will ensure rigor and allowed the team to consider research question findings from multiple perspectives in depth. For example, qualitative findings related to potential strategies to overcome inequities or gaps in service delivery in terms of inequities were triangulated with quantitative findings regarding access to the service and potential inequities. Additionally, findings on the potential benefits and risks of the role identified through qualitative workstreams could be used to inform quantitative and cost exploration. As part of the integration, we aimed to develop a logic model of the MNISA service (which outlines the implementation, potential mechanisms of change, and potential outcomes and costs). We also developed recommendations for improving and evaluating this service in the future. We worked together through regular project team meetings and the regular sharing of findings from each workstream.

Timeline

Study development (including ethical approval) took place between April and October 2024. Data collection and analysis took place between October 2024-June 2025. The study’s final report and dissemination outputs were submitted from June 2025 onwards.

Dissemination plan

We will disseminate findings in several ways, including:

Findings from the study will be used to develop recommendations for the future implementation of this role and/or similar models that may be implemented in the future in England or other countries.

Summary

This protocol outlines our rapid mixed-methods approach, which aims to evaluate the pilot implementation of the Maternity and Neonatal Independent Senior advocate role in England. This research extends previous research (e.g., international and UK research evaluating strategies to improve maternity services generally^{17,25,27,34–37} and workforce strategies^28–33) by providing evidence on the implementation, family and staff experience, and the impact and cost of this role in England.

Strengths/limitations

The strengths of this research include the rapid mixed-methods design, which will enable real-time evidence generation related to implementation, experience, impact, and cost during the pilot of the MNISA role. This will enable triangulation of different aspects of the study to provide meaningful lessons that can be used to optimise service delivery and inform decision-making around the wider applicability of the role in the future.

A further strength of this research is our collaboration with sand and the breadth of stakeholder engagement, which has been used to inform the development of the study design. Through PPIE input, wider stakeholder engagement, and scoping conversations, this has enabled the design of a study that will hopefully be feasible to achieve within the rapid timeframe, but that will also address important questions relating to the evaluation of the pilot.

However, this study has some limitations. Given the rapid timeframe, while we will make every effort to recruit as many diverse views as possible within our study, the success of recruitment will depend on some factors out of the study team’s control (e.g., the service will not have been up and running very long, and some adverse outcomes may be more frequent in certain populations). These examples, together with other factors, may have impacted various aspects of the study. For qualitative workstreams, these factors may impact the number and diversity of families that have received support and/or may wish to take part. Furthermore, we only aimed to interview up to five families that had not received support. For the quantitative workstream, sample sizes within the caseload data may not be sufficiently large to identify many aspects related to inequities. Finally, we acknowledge that the available evidence arising from this evaluation will not include thorough effectiveness and cost findings owing to the timing of the evaluation.

Implications

This rapid evaluation of the pilot of the MNISA role in England provides key information regarding the impact and cost implications of the role to date (while acknowledging the short implementation period to date), enabling the development of recommendations for optimising the implementation of this role in the future and enhancing family and staff experience of the service.

Future research

If this role is implemented more widely in the future, it is necessary for future research to capture the long-term impact (effectiveness) and cost-effectiveness of the MNISA role. Additionally, further research on implementation, training and development needs, and staff and family experience would help to explore variations in implementation and experiences over time (particularly in relation to inequalities in access to and experiences of the role for families).

Reporting guidelines

Not applicable.

Ethics and consent

This study was reviewed and given a favourable opinion by the NHS East of England – Cambridgeshire and Hertfordshire Research Ethics Committee [REC reference: 24/EE/0205] and received Health Research Authority approval.

Informed consent

Informed consent was obtained from all participants before taking part in the study, with different options for consent (written, verbal) provided to ensure that participation was accessible and inclusive. Further details are presented in the Qualitative workstream section of this article.