Evaluation of retention of knowledge, skill and competency of health workers one year after completion of the Helping Babies Breathe training program in South Sudan

Christopher Vunni Draiko; Khemika Yamarat; Alessio Panza; Judith Draleru

doi:10.12688/f1000research.17560.2

Home Browse Evaluation of retention of knowledge, skill and competency of health...

ALL Metrics

-

Views

-

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Revised

Evaluation of retention of knowledge, skill and competency of health workers one year after completion of the Helping Babies Breathe training program in South Sudan

[version 2; peer review: 2 approved with reservations]

Christopher Vunni Draiko ¹, Khemika Yamarat¹, Alessio Panza¹, Judith Draleru²

PUBLISHED 10 Apr 2019

Author details Author details

¹ College of Public Health Sciences (CPHS), Chulalongkorn University, Bangkok, Thailand
² Department of Gynecology, Juba Teaching Hospital, Juba, Central Equatoria, Sudan

Christopher Vunni Draiko
Roles: Conceptualization, Data Curation, Investigation, Methodology, Project Administration, Resources, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Khemika Yamarat
Roles: Conceptualization, Formal Analysis, Methodology, Project Administration, Supervision, Validation, Writing – Original Draft Preparation, Writing – Review & Editing

Alessio Panza
Roles: Formal Analysis, Investigation, Methodology, Project Administration, Resources, Writing – Original Draft Preparation, Writing – Review & Editing

Judith Draleru
Roles: Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Supervision, Validation, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Introduction: The aim of the study is to evaluate the long-term retention of knowledge, skill and competency of health workers who completed Helping Babies Breathe (HBB) training and effect on newborn mortality.
Methods: The study employed pre-post-interventions study and participants were selected based on their previous training on HBB protocols. Health workers were assessed for knowledge, skill and competency pre, post training in March, and 3 months in June 2017 and 1-year post implementation in September 2018. Health workers were scored on knowledge, skill and competency. The mean score was obtained and repeated ANOVA, chi-squared test and Pearson’s test was used to compare pre- and post-intervention and one-year implementation. Retention of health worker’s knowledge, skill and competency was assessed using the HBB questionnaires, checklist, practical skill and drills, and were scored on knowledge, skill and competency. The scores were computed into percentages, mean scores and mean differences, and compared against scores at 3 months and 1 year. Impact on management of newborn asphyxia was conducted using a review of delivery registry at maternity and children ward scores were group into percentages, averages means, computed using the Chi-square test.
Results: Helping Babies Breathe has shown a significant increase in knowledge, skill and competency post-test and three months. Despite the improvement in knowledge, skills and competency of the health within the three months of training, there was marked decline over 1-year period.
Conclusions: Immediate evaluation of the health workers after Helping Babies Breathe training resulted into significant increase in knowledge, skills and competency in neonatal resuscitation although this declined over period of one year. The training also resulted into decrease in neonatal mortality.

Keywords

Knowledge, skill, competency, retention, neonatal resuscitation, Education

Corresponding author: Christopher Vunni Draiko

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2019 Vunni Draiko C et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Vunni Draiko C, Yamarat K, Panza A and Draleru J. Evaluation of retention of knowledge, skill and competency of health workers one year after completion of the Helping Babies Breathe training program in South Sudan [version 2; peer review: 2 approved with reservations]. F1000Research 2019, 8:167 (https://doi.org/10.12688/f1000research.17560.2) First published: 07 Feb 2019, 8:167 (https://doi.org/10.12688/f1000research.17560.1) Latest published: 10 Apr 2019, 8:167 (https://doi.org/10.12688/f1000research.17560.2)

Revised Amendments from Version 1

We have made some changes to the following contents:

Abstract: revised the result and conclusions
Introduction: included new literature
Methodology: included notes on bag and mask ventilation drill conducted during intervention and information concerning exclusion of still birth and premature newborn
Result: revised the flow chart for health workers and newborn registry (Figures 1 and 2), uploaded new Table 1 containing demographic characteristics and included some text on retention of knowledge
Discussion: revised most of the discussion and added new items relating it to the comments
Conclusion: new discussion was included based on the new discussion
References: arranged the references as cited

To read any peer review reports and author responses for this article, follow the "read" links in the Open Peer Review table.

Introduction

Of the 200,000 children born annually in South Sudan, an estimated 40% die in the first month of life¹. Newborn mortality contributes to 39% of all deaths of under-fives in South Sudan. The decline of newborn mortality rate in South Sudan has remained slower than the global average rate for the last decade². An estimated 136 million infants are born each year, and this figure is expected to rise globally to nearly 137 million births yearly by 2016. The highest risk to newborn life is within the first day of life. Death within this period accounts for almost 5% of newborn deaths^3,4. Globally and in South Sudan, the main direct causes of newborn deaths are usually infection-related complications (26%), intrapartum complications (24%) including birth asphyxia, preterm delivery (34%), and congenital abnormalities (9%). In South Sudan, 20% of newborn deaths are associated with birth asphyxia^5,6.

The transition from intrauterine to extra uterine life requires initiation of breathing which is critical physiological changes required for newborn survival. Research indicates that the majority of newborns initiate breathing within 30 minutes and an estimated 10% breathe when they receive drying and stimulations from health workers, 3% of newborns require positive pressure (PPV), while another 2% need ventilation and intubation offered by knowledgeable and skilled health workers^7,8. Training of health of health workers on newborn resuscitation and provision of effective and timely resuscitation could prevent a good number of newborn with asphyxia and subsequently improve their survival rate.

To address the lack of basic resuscitation training in health facilities in low-resource areas and improve neonatal outcomes, a newborn care training pro- gram targeting Health Care Professional called Helping Babies Breathe (HBB) was developed in 2010 by the American Academy of Pediatrics and their partners⁹.

HBB curriculum is designed to train birth attendants in low resources countries in the skills of neonatal resuscitation. It is Evidence based specifically geared towards reducing global neonatal mortality. HBB focuses on the essential steps of resuscitation including preparation for birth evaluation of the infant, stimulation to breathe and ventilation by critical first golden minutes after birth¹⁰ HBB curriculum promotes and advances active learning using hands on skills practice with the new born stimulators, paired learning, self-reflections, group discussions and feedback after completing a task. It also focuses on paired learning which are tested with four formative assessments. HBB learner’s competence and performance can also be determine by systematic methods using multiple choice questionnaire, a bag and mask ventilation and objective structural clinical examination consisting of section A and B (OSCEA&B).

Review of publication by Singhal et al on descriptions of formative educational evaluation of HBB assessment in Kenya and Pakistan contributed to the modification of HBB curriculum assessment before its global launch¹¹ Similarly, more studies on HBB curriculum assessment both in Africa and Asia using pre and post test scores demonstrated a significant gain in resuscitation knowledge and skills immediately after HBB training^11,12. However, the studies had limitation of not examining various learner’s qualifications characteristics of learner’s performance. Understanding learner’s performance and competence could contribute to some adjustment or refinement of the HBB curriculum and development of the best educational methods to promote retention of neonatal resuscitation knowledge and skills that is tailored to the specific learner.

Various studies on the effectiveness of newborn resuscitation training programs such as HBB showed increased knowledge, skill and competency after training which was sometimes retained for 1 year. Evaluation of newborn resuscitation training has shown an immediate increased in knowledge, practical skills and competency with improved newborn outcome¹³. Although training has been carried out, there has been limited rigorous evaluation of retention of knowledge, skill, competency and newborn outcome in low-resource and post-conflict settings like South Sudan.

The outcome of training depends on the extent to which knowledge, skill and competency of health workers is retained, alongside the ability to use the learned skills, knowledge and application at appropriate time. However, the retention of knowledge and skills, and their application to improve newborn outcomes depends on several factors, such as conditions at the clinical environment, regular supervision, settings, opportunity to practice, availability of supplies and appropriate equipment¹⁴.

Training of health workers appropriately, provision of refresher training and support and ensuring good training environment facilitates the long-term retention of knowledge, skill and competency among health workers in low resource settings¹⁴. In South Sudan, nurses and midwives manage normal delivery and birth asphyxia is not recognized early enough. Medical doctors are often involved in the late stages of the management of birth asphyxia due to critical shortages, even in a major hospital setting. The objective of study is to evaluate retention of knowledge, skill and competency of the health workers and the impacts on newborn mortality after one year of implementation.

Methods

Trial background

The trial has been registered with Pan African Clinical Registry with registration number PACTR201708002469225¹⁵. The study TREND statement is available as supporting document. The original aim of the study was intended to measure only effectiveness of improving health workers, knowledge, skill and competency. The measurement of newborn mortality was conceived later leading to late registration.

Sample size

Calculation of sample size was based on the ability to detect a 20% increase in knowledge, practical skill and competency and 20% reduction in newborn mortality with an error of 0.05, 20% and dropout rate of 50%. The study used this value as references for the sample size calculation. Using G*Power version 3.1, we determined a sample size of 74 participants in each arm to account for losses, but due to the ongoing conflict the actual participants for both arms were less than estimated sample size

Selection criteria

The following selection criteria were applied. Medical officers/doctors, nurses, midwives, maternal child health officers, community health workers and clinical officers working and practicing in maternity, operating theater and children ward; health providers self-reporting that they provide routine care services at delivery and neonatal unit or departments; and Health workers willing to be available for data collection and during the period of study.

Recruitment setting

The health workers were identified and recruited from maternity wards, newborn operating theater and children ward of Juba Training Hospital (intervention site) and Wau Teaching Hospital (control). After the completion of the recruitment process, an invitation was sent to those health workers who met the inclusion criteria to participate in the study. Written informed consent was obtained from the health workers. All newborn delivered in maternity, newborn unit and operating theatre who met the inclusion criteria were included in the study.

Location of data collection

Data on the pre and post training on knowledge, practical skills of the health workers and records of newborn asphyxia and deaths both in intervention and control group was collected from the area of practice, maternity ward, operating theatre and newborn unit.

Period of recruitment

Health worker’s recruitment commenced on February 1^st and ended in 25^th February 2017. Training intervention was 27th–28th February 2017. The intervention commenced in March and follow-up started on June 1^st and ended on 30^th June 2017. The 1-year evaluation was conducted in August 2018.

Specific objectives and hypotheses

Our study objectives were as follows. To assess change in health worker knowledge, psychomotor skills, competency of the health workers regarding managing neonates with birth asphyxia after training intervention, and the ratio of perinatal mortality due to asphyxia of hospital admission within 24 hours of birth after training interventions. Meanwhile we hypothesized that HBB training would result in 20% increase in knowledge, skill and subsequent reduction of early newborn mortality among newborn born with asphyxia.

Primary and secondary outcome measures

The main primary outcome was any improved knowledge, skill and competency of the health workers on HBB protocol. The secondary outcome was the ratio of early newborn reduction with asphyxia within 24 hours.

Data collection method and instruments

Data was collected by use of a questionnaire with 17 multiple choice questions to assess the knowledge of health workers on the HBB protocol, a seven-step checklist for bag-and-mask skill assessment, a five-step checklist for preparation at birth skill checks, a 13-step checklist with simulation for the first Objective Structural Clinical Examination (OSCE), a self-observation questionnaires consisting of a 25-step checklist maternity and newborn register for recording delivery, and neonatal a registration form¹⁶.

Validity of HBB instruments

The validity of the HBB instruments was determined by expert reviews and opinions from Chulalongkorn University, college of Public Health Sciences. The study used HBB instruments for assessing the knowledge, practical skill and competency of the health workers was a standard HBB instrument tool which has been validated and used in other low setting countries where HBB and skills evaluation have been conducted¹⁷.

Retention evaluation

The retention evaluation of skill was conducted following, pre- and post-test study design conducted in Juba and Wau Teaching hospitals in Republic of South Sudan. The pre- and post-training assessment was completed in March 2017 and the 3-month assessment was done on knowledge, skill competency and newborn mortality in June 2017. Assessment at 1-year post-intervention to determine retention of knowledge skill, competency and review of neonatal mortality was conducted in August 2018. During evaluation, health workers who were available and received previous HBB training in both hospitals were approached and interviewed according to the set exclusion and inclusion criteria. The Study was approved by South Sudan Ethical Committee, Ministry of Health in Juba, South Sudan and reviewed by Chulalongkorn University, College of Public Health Sciences Bangkok, and Ethics Review committees of Juba Teaching Hospital and Wau Teaching Hospital. Informed consent, both written and verbal, was obtained from the health workers before the training intervention. Informed consent was sought from each of the health workers available at the time of evaluation. Additionally, verbal approval was sought from the mothers with newborn asphyxia by the health worker.

Training of trainers of trainers

In December 2016, four midwives went through the HBB trainers of trainer’s (TOT) workshop and become research assistants and facilitators. Training of midwives as TOTs covered teaching methodology, neonatal resuscitation using HBB model, evaluation skills and practical skill training. The second part of TOT involved conducting HBB simulation drills, preparation of births, newborn routine care, the golden minute and ventilation of newborn. The last component of TOT program was facilitation and coaching skills using American Pediatric Association model for HBB.

Selection and training of health workers

Completion of TOT course for facilitators was followed by recruitment of health workers in the Juba Teaching Hospital (intervention) and Wau Teaching hospitals (Control) to participate in the HBB study. The health workers in the control hospital were not trained on HBB protocol during the implementation period. However, plans are underway to provide training after final evaluation. The two hospitals were located 300 km apart and in different regions. Both are national teaching hospitals that train health workers. Health workers were selected from maternity and newborn wards and the antenatal clinic (ANC). All health workers working in maternity, neonatal ward and the ANC were approached to participate in the study. Informed consent written in Arabic and English were read and explained to the health workers, who signed a copy before their full participations.

Pre-training assessment was conducted before the commencement of training on demographics, previous knowledge and exposure to HBB training, knowledge, skill and competency, and also review the birth registry for number of births, neonatal asphyxia and mortality for the past year. The survey used a multiple-choice questionnaire and simulated environment using Bag and Mask checklist for psychomotor skill and OSCEA&B for competency at pre-intervention, immediately after intervention, 3 months post-intervention and 1 year post-intervention.

The initial training program covered preparation for birth, routine care, the golden minute and ventilation of newborn using bag and mask techniques. Individual training involved conducting simulation drills on mannequins (mama and baby NeoNatalie). Immediate feedback on performance of the health workers and lectures by the facilitators. The health workers were allowed to practice a number of drills, demonstration and return demonstration using mannequins. The demonstration was accompanied with an explanation on how to perform tasks to help newborn breath at birth during the study, individual health workers were assessed on their knowledge, skills and competency on neonatal resuscitation using HBB protocol, pre-training, immediately after training, at 3 months and at 1 year to determine short- and long-term retention of knowledge, skill and competency. A 20-minute written multiple-choice test was administered to the participants before and after the training to evaluate knowledge. Skills and competency assessment for the health workers were evaluated through clinical simulation using the NeoNatalie newborn simulator. The trainees were directed and observed as they perform the task on the mannequin. OSCE A&B were administered to test the performance and competency of the trainee health workers on HBB. OSCE A consists of 13 items while OSCE B consist of 18 items indicating the key component of what the trainee must learned and practice. Each item in objective OSCE A and B were scored 1 when correctly done and score 0 for not correctly done. A passing score of 80% was required to complete the course successfully. In each training session, health workers were asked to evaluate the training using a questionnaire with a five-point Likert scale¹⁶.

The HBB training included trainers and trainees reviewing training modules and practical sessions on newborn asphyxia, routine care and ventilation. Health workers practiced on the mannequin and were provided with equipment, and each of the practice sessions on the mannequin was rated by the facilitators, who were senior midwives and trained as master trainers. The mannequin which presented mother and newborn was placed on the resuscitation table and each situation presenting the condition of the baby and mother is controlled with the health workers present caring for the mother and newborn. The facilitator (rater) was responsible to ensure participants completed the skill-based checklist for assessing the performance of the health workers during the practical drill. The participant performed the drills in either Arabic or English, communication channels commonly used in South Sudan. The task performed by the health workers were explained and verbalized to the midwife rater as the health workers conducted the newborn resuscitation drill.

Content of training

Each topic was introduced by the main facilitators followed by demonstration and return demonstration by the participants. This was followed by self-directed and self-assessment exercise using the learner work book. The training comprised of six classroom hours and two hours for practical. A series of procedures surrounding births were reviewed through practical exercises under the supervision of the trainers. Scenarios reproducing routine care and neonatal resuscitation at birth were performed on a NeoNatalie new born simulator. During the training, one simulator, a stethoscope, resuscitator and suction device were made available for every two trainees. The intervention was delivered to a group 20 health workers each. Training was provided to the intervention group in Juba Teaching hospital. The HBB training intervention course was facilitated by two senior midwives who were trained as research assistants (master trainer) and assisted by the principal investigator and the research assistants. The newborn resuscitation was conducted by the health workers in various unit selected. The intervention was delivered at the main hospital training hall and the observation was at maternity, newborn unit and operating theatre. Two training session was conducted to allow adequate time for facilitators and participants to learn and practice. The ratio of one facilitator and six participants was maintained to support participants to learn in pairs as standard protocol indicated in the training. The duration lasted for 2 days. The training intervention was delivered in 6 hours each for two days and practical seasons was 20 minutes. Observation of health workers practicing resuscitation was observed throughout the 3-month period by the research assistants. Bag and mask drill was delivered every morning 5 days a week for 3 months. Health workers were provided breakfast and lunch during the two days training interventions and support supervision during morning drill of mask and bags.

Health worker assessment

Completing tasks such as preparation for ventilation, assessing the newborn, conducting ventilation using bag and mask, conducting stimulations and referral of the newborn was directly performed on the mannequin and scored by the experienced rater. Raters scored health workers one on skill and competency when the task was correctly completed and zero when not correctly completed. Health workers must have obtained 80% or above to be considered competent in all the three aspect of knowledge, skill and competency to help newborns breathe.

Post-training and intervention assessment was done immediately, and at 3 months using the simulated HBB knowledge-, skill- and competency-based checklist. Assessment were categorized into knowledge, practical skills and competency. The sum of the scores were calculated by determining the degree of completion of each task in knowledge, skill and competency, the total score for each task was 80% for knowledge, skill and competency respectively.

All health workers who completed the initial HBB training and were present 1 year later were asked to participate in HBB knowledge, skill drills and competency assessment to determine long term retention of their knowledge, skill and competency. The retention of knowledge, skill and competency assessment used the same check list for assessment and scoring at the initial training. During 1-year evaluation, each individual health workers was assigned knowledge, skill and competency score presented as percentage of correctly performed items at 1-year evaluation. Health workers were asked to complete the same five-point Likert scale survey as well their confidence in HBB practice.

Assessment for simple newborn resuscitation

During the interventions, health workers were assessed for simple resuscitation using Objective Structural Clinical Examination (OSCE A) which was made up of 13 observation steps consisting of scripted information on preparation for birth, drying the baby thoroughly, ability to recognize the baby is not breathing, positioning of the head and clearing the airway, evaluation of the breathing, clamping or tying and cutting the cord, position skin to skin and communication with mother. The successful completion requires a total score of 10 correct of 13 steps and the health worker being observed must include the Dries thoroughly, Recognizes Baby is not crying and positioning and clearing the airway. OSCE A is a performance assessment of preparation for birth and routine newborn care, and a learner must perform = 80% (10 of 13 steps) correctly to pass, including three essential steps. OSCE A is a performance assessment of preparation for birth and routine newborn care, and a learner must perform = 80% (10 of 13 steps) correctly to pass, including three essential steps.

Assessment for complex newborn resuscitation

We conducted assessment of health workers for complex neonatal resuscitation using Objective Structural Clinical Examination (OSCE B). This was made up of 18 scripted scenario on preparation for birth, drying of the baby, recognize the baby is not breathing, ventilation at 40 breathes per minutes (30–50) acceptable, looking for chest movement, evaluate breathing, call for help, improve ventilation thorough, repositioning the head, reapplication of mask, clear secretion, open mouth slightly and squeezing the bad hardly. The 18 items reflect the key components of the training course for newborn survival. Each item was scored 1 if carried out correctly and any partial or incorrect action was scored zero. Similar to OSCE A, OSCE B was a performance assessment of a complex resuscitation scenario that requires bag-mask ventilation, and a learner must perform 14 of the 18 steps correctly to be evaluated to have the competency to help newborn with asphyxia to breathe.

Daily bag and mask drills Assessment evaluation

During the entire intervention, the health workers performed equipment checks and run drills of HBB on NeoNatalie using the checklist. Where the health worker encounter neonates with birth asphyxia, the research team and assigned supervisor conduct group evaluation using the written check list and at the same time, the health workers evaluate their performance using the self-evaluation checklist. Each health workers who conducted resuscitation of neonates with birth asphyxia signs in the delivery registers and on the designed form for the research against post resuscitation procedure column. In the existing delivery registers, there was no column for indicating the procedure and this was solved by designing different form to capture the health worker’s actions.

The self-evaluation checklist

This check list was made up of 25 questions for the immediate care of the neonates and neonatal resuscitation as per the HBB protocol with check boxes. After completing care of each of the neonates, the health worker completes the self-evaluation checklist based on the steps taken according to the HBB protocols.

Hospital registry and forms

Data on neonatal mortality was collected from the hospital register at the delivery room, operating theater and neonatal unit admission book pre- and post-implementation for June 2017–June 2018. In the pre-implementation phase, hospital monthly record was used to collect the number of deliveries, neonates with breathing problems; neonates resuscitated using HBB protocols and the perinatal mortality due to asphyxia outcome immediately post-training and end of intervention. For easy follow-up, a simple form was designed by the research team to track deliveries, use of HBB protocol for newborn with birth asphyxia and the outcome of the resuscitation within 24 hours. The mean ratio of the perinatal death was used to determine the outcome/changes of the intervention within the period at 3 months intervention and 1 year evaluation. Cases of pre maturity were handled carefully and determination of death due to asphyxia was only recorded after reviewing the case note with the doctor or experienced midwives in charge handling pre term babies in the neonatal unit as the cause of death was asphyxia rather than pre -maturity and its complications. The review of hospital registry and form was approved by the South Sudan ethical review board and the hospital ethics committee.

Statistical analysis

The mean score and the significant level within and between groups were tested using repeated ANOVA and Chi squared test in the three performance areas of knowledge, skill and competency. Newborn mortality was determined and tested with Pearson chi squared test. The statistical analyses were performed using SPSS version 20 and p<_0.05 was considered statically significant.

Results

A total of 70 health care workers enrolled in this study of which 40 received HBB training and completed (100%) the pre- and post-test course assessment. Approximately 30 participants in the control group took the post-test simultaneously with those of the intervention group. After one year, only 53 of the 70 health workers who initially participated in the study were available for assessment, as shown in Figure 1.

Figure 1. Flow chart for recruitment and allocation health workers.

Characteristics of health workers

Majority of the health workers were aged between 25 years to 35 years. Nurses and midwives were the majority in intervention and control group and were predominantly female (82.5%) in intervention and 80%) providing newborn care. Most of the health workers were working in maternity labor room) 23 (57.5%) and 19(63.3%), children ward 16 (40%) and 7(23.3%) and operating theater 1(2.5%) and 4(13.3%) respectively. Majority of the health workers self-reported to be registered nurses and midwives with tertiary and college education (77.5% of the intervention versus 73.3% in control). The level of income in the middle income bracket of 1001- 2,000 SSP varies between intervention and control group with control group slightly receiving higher income compared to intervention. The difference however is not significant. The duration of practice among the health workers ranged from less than one year to over five years with most having practiced over five years (32.5%) intervention and versus (36.7%) in control group and there was insignificant difference between intervention and control group (Table 1). Similarly, at one year 58.6% of the health workers in intervention and 66.7% in the control spent their full time working in maternity (delivery unit). The health worker in average attends 115 deliveries and managed 7.7 asphyxia cases in 1 year. The Table 1 shows demographic characteristics of the health workers after 1 year of implementation.

Table 1. Demographic characteristics.

		Baseline							One year
		Intervention		Control					Intervention		Control
Demographic Characteristics		Freq. (N=40)	(%)	Freq. (N=30)	(%)		p - value	Statistical test	Freq. (N=29)	(%)	Freq. (N=24)	(%)
Age in years	25–35	25	62.5	20	66.7	0.130	0.719	Chi-square	17	58.6	15	62.5
Age in years	36 above	15	37.5	10	33.3				12	41.4	9	37.5
	Total	40		30					29		24
Gender	Male	7	17.5	6	20.0	0.071	0.790	Chi-square	4	13.8	3	12.5
Gender	Female	33	82.5	24	80.0				25	86.2	21	87.5
	Total	40		30					29		24
Education level	Primary eight	5	15.0	4	13.3	0.748	0.781	Fisher Exact	4	13.3	3	12.5
	Secondary	2	7.5	4	13.3				3	10	4	16.7
	College/tertiary	31	77.5	22	73.3				19	63.3	17	70.8
	Diploma in Midwifery	1	2.5	0	0				1	3.3	0	0
	Community H.W Training	1	2.5	0	0				2	7.0	0	0
	Total	40		30					29		24
Professional qualification	Nurse	12	30.0	10	33.3	5.690	0.623	Fisher Exact	9	31.0	10	41.7
	Midwives	17	42.5	11	36.7				10	34.5	9	37.5
	Maternal Child health officer	3	7.5	1	3.3				3	10.3	0	0.0
	Nurse practitioner	1	2.5	0	0				1	3.4	0	1.0
	Clinical officer	2	5.0	2	6.7			Fisher Exact	2	6.9	2	8.3
	Community Health workers	4	10.0	2	6.7				4	13.8	2	8.3
	Skilled birth attendants	1	2.5	3	10.0				0	0.0	0	0.0
	Intern Doctor			1	3.3				0	0.0	1	4.2
	Total	40		30					29		24
Primary area	Newborn care	11	27.5	8	26.7	5.987	0.097		8	27.5	11	45.8
	Sick children ward	4	10.0	1	3.3				3	10.3	2	8.3
	Maternal and newborn care	25	62.5	17	56.7				18	62.1	10	41.6
	Obstetrics/Obstetrician	0	0	4	13.3				0	0	1	4.1
	Total	40		30					29		24
Current place of work	Maternity ward	23	57.5	19	63.3	4.135	0.129	Fisher Exact	17	58.6	16	66.7
	Children ward	16	40.0	7	23.3				11	37.9	4	16.7
	Operating theater (OT)	1	2.5	4	13.3				1	3.4	4	16.7
	Total	40		30					29		24
Monthly income	300–1000 SSP	18	45.0	8	26.7	2.896	0.235	Chi- square	10	34.5	7	29.2
	1001–2,000 SSP	14	35.0	16	53.3			Chi- square	11	37.9	12	50.0
	2,001 SSP and above	8	20.0	6	20.0				8	28	5	21
	Total	40		30					29		24
Duration of practice	≤ 1 year	12	30.0	6	20.0	0.980	0.806		7	24.1	4	16.7
	Two – three years	10	25.0	8	26.7				7	24.1	7	29.2
	Four – five years	5	12.5	5	16.7				5	17.2	4	16.7
	Over five years	13	32.5	11	36.7				10	34.5	9	37.5
	Total	40		30					29		24

*Significant level at 0.05. The 25–35 years in case of age is based on the fact that it represents the youthful age group. Fishers Exact test have been used for cell counts less than 5.

The health workers were assessed for prior knowledge on resuscitation skills that could influence their helping Breathing performance. Those found to have received similar training on neonatal resuscitation and simulation within the last 3 months were excluded from the training. Health workers were also asked to rate the training and performance and bag and mask using Likert scale 1-5 and 5 being the strongly agree

Status of retention of knowledge

The mean score of the health workers in the interventions at pretest, posttest, 3 months and one year tested by repeated measure ANOVA show that health workers had significant increase in terms of knowledge between the pretest and immediate post intervention (mean difference increase of 55.2 (50.959.6) p<0.05) and this decreased slightly between the immediate post intervention and three months follow up with mean difference of 13.3(-17.7-8.87), p<0.05). At one year, the health worker had further shown a decreased mean score from 84.5% at 3 months to 69.4%, with a mean difference of 69.4±18.8(−15.0 (−22.7–7.4)) in the 17 domains that was assessed after 3 months and at one year after training. The changes in scores of knowledge measure at 1 year was statistically significant (p<0.05) (Table 2).

Table 2. Retention of HBB-trained health worker’s knowledge, psychomotor skills and competency.

	Intervention		P value^a	Control		P-Value
Variables	Mean	Mean df. (CI)		Mean	Mean df. (CI)
HBB Knowledge
Pre-test	42.5±17.3			48.0±13.9
Post-test	97.8±3.4	55.2 (50.9-59.6)	<0.001	51.2±11.2	3.1 (−3.0–9.4)	0.9
3 months follow-up	84.5±7.4	−13.3 (−17.7–8.87)	<0.001	50.6±16.9	−0.3 (−0.1–6.0	0.9
1 year follow-up	69.4±18.8	−15.0 (−22.7–7.4)	<0.001	61.2±20.3	10.5 (0.54-20.6)	0.04
HBB Psychomotor skills (Bag and Mask)
Pre Test	26.1±19.9			-*	-	-
Post test	94.4±8.5	69.2 (62.8-75.7)	<0.001	43.8±16.7
3 months follow-up	94.5±8.2	0.1 (−0.3–0.8)	<0.001	40.3±20.5	3.4 (11.0–4.10)	0.37
1 year follow-up	77.0±21.8	−17.5 (−27.2–−7.8)	0.001	56.5±25.5	16.2 (1.2–31.2)	0.04
HBB Competency for simple neonatal resuscitation (OSECE A)
Pre test	26.9±14.6			-*	-	-
Post test	88.8±8.5	61.2 (57.0–66.7)	<0.001	38.9±8.5	-	-
3 months follow-up	88.6±8.6	03.1 (−4.6–532)	0.9	38.0±9.1	2.65 (−8.42–3.1)	0.36
1 year follow-up	76.4±13.6	−12.2 (−18.3–−6.1)	<0.001	53.9±11.8	15.1 (8.7–21.4)	<0.001
HBB competency for complex neonatal resuscitation (OSECE B)
Pre test	17.5±8.9			**	-	-
Post test	90.9±7.1	73.47 (69.5–77.36)	<0.001	36.5±13.0	-	-
3 months follow-up	88.3±10.8	−274 (6.71–1.22	0.17	33.1±8.7	−3.63 (−8.2–0.93)	0.12
1 year follow-up	76.9±11.6	−13.6 (−19.8–−7.4	<0.001	53.4±21.7	21.0 (10.9–31.0)	<0.001

Scores expressed as mean difference.*Significant level at 0.005 at 3 months and one year after training. Pa-Value within intervention group tested by repeated ANOVA ,0.001 and 0.001 between immediate post intervention and 3 months follow up **No baseline conducted for control group for bag and mask, OSCEA&B.

Figure 2. Flow chart for birth registry pre- and post-implementation.

Status of retention of skill among the health workers

Skills retention among the health workers were assessed using the seven domains of determining skill applied at pre-training, immediately post-training and at 3 months post-training assessment for both groups. The 1-year mean skill scores declined from 94.5% to 77% from 3 months’ assessment. The decline is visible across all the seven domains: i) Check equipment; ii) select the correct mask; iii) apply the mask to make a firm seal; iv) ventilate at 40 breaths; v) Look at chest movement; vi) improve ventilation if the chest does not move; vii) reapply mask and reposition, clear secretions and squeeze the bag. Mean differences of 77.0±21.8 (−17.5(−27.2–−7.8) were found (Table 2).

Status of health worker’s competency for simple resuscitation

The competency of the health workers declined among the intervention (Juba Teaching Hospital) and greatly increased in the control (Wau Teaching Hospital) after one-year evaluation when compared with the 3 months’ post implementation. There was a significant decrease in competency between the 3 months and 1 year of training (88.6±8.6 to 76.4±13.6) (mean difference decline of −12.2 (−18.3–−6.1 p<0.001). The mean decreased between 3-months and 1-year after training evaluation for the retention of was significant. The status of retention of competency among the health workers in control showed significant increase from 38.0±9.1 to 53.9±11.8 (mean difference 15.1 (8.7–21.4 p< 0.001)) (Table 2).

Status of health worker’s competency for complex resuscitation

Evaluation of retention status of health worker’s competency in both groups after one year of post implementation assessment has shown that there was marked decreased among the intervention. In the intervention, the mean competency score decreased from 90.4±8.6 to 76.9±11.6 (mean difference −13.6 (−19.8–−7.4; p<0.001).

Status of early newborn mortality

A review of records in the hospital registry and forms from June 2017 to June 2018 found out that 6,072 live births were registered in the maternity and newborn wards and ANC. A total of 4,887 of the total live birth were recorded in the intervention hospital and 1,210 in the control hospital. All births were reported to be assisted by health workers trained in the HBB.

A 1-year post-training evaluation found that there was a significant increase in the number of newborns with asphyxia being resuscitated in both hospitals. More newborns in intervention hospital received neonatal support (resuscitation). 98.4% of the newborn received support compared to 1.6% in control group at 3 months. Meanwhile (86.7%) compared to the control (48.4%) newborn received neonatal resuscitation support from health workers at assessment after 1 year. The support given to newborn by health workers was statically significant (p<0.050; Table 3). Review of data on early mortality due to asphyxia shows that mortality decreased from 30.7% at 3 months to 17.9% at 1 year in the intervention hospital which is significant (p<0.05). There was also reduction of early newborn mortality in the control hospital. Mortality reduced from 69.2% at 3 months to 49.4% at 1 year after implementation (p<0.05). The result showed that more newborn deaths occurred within 24 hours of birth in both hospitals in comparison to deaths that occurred after 24 hours up to 1 year (Table 3).

Table 3. Early newborn mortality.

Variable		Before intervention		After intervention (3 months )			1 year after Intervention
	Intervention/ Control	Freq	Percent (%)	Freq	Percent (%)	P. Value	Freq.	Percent (%)	P. Value
Total live births	Intervention	1116	52.40	1112	53.9	-	4,887	80.2	0.000
Total live births	Control	1011	47.50	950	46	-	1,210	19.8	0.000
Newborn birth asphyxia	Intervention	88	55.7	125	57.1	0.18	312	75.9	<0.001
Newborn birth asphyxia	Control	70	44.3	94	42.9	-	99	24.1	<0.001
Newborn resuscitated using HBB	Intervention	0	00	124	98.4	0.001*	270	86.7	0.001
Newborn resuscitated using HBB	Control	0	00	2	1.6	0.114	48	48.4	0.001
Asphyxia deaths	Intervention	26	50.9	4	30.7	0.001*	57	17.9	0.001
Asphyxia deaths	Control	25	49	9	69.2	0.110	49	49.4	0.001
Death within 24 hours	Intervention	14	51.9	4	23.5	0.001*	41	71.9	0.001
Death within 24 hours	Control	13	48.1	13	48.1	0.110	33	67.3	0.001
Death after 24 hours	Intervention	12	50.0	9	33.3	0.000*	16	28.0	0.001
Death after 24 hours	Control	12	50.0	18	66.7	0.112	20	32.7	0.001

Tested by Pearson Chi square test 2×2 sided significance for birth asphyxia, newborn death within and after 24 hours before and after implementation.

Discussion

This study attempted to evaluate the retention of knowledge, skills, competency and impact of modified HBB at Public Tertiary Hospital in South Sudan. and to the best of our knowledge, it is first rigorous study of HBB study conducted in republic of South Sudan The status of knowledge, skill competency among the health workers was evaluated after 1 year and compared with the status at 3 months.

HBB Knowledge of the health workers

The overall mean knowledge pass rate for the intervention at baseline using HBB Multiple Choice Questions was 42.5 % and control 48%. There was no difference between the intervention and control group during the baseline assessment although control group had score slightly higher than the intervention group. Of recent there was no known formal psychometrics performed beyond field test on HBB multiple Choice questions. Despite the difference in prior education and professional characteristics seen in our study, Helping Babies Breathe was beneficial and appropriate regardless of profession, as demonstrated by statistically significant gains in post-test scores for the health workers.

Most significant in our study was that, health workers who received training were able to achieve high score in simulated environment despite lack of previous training on Helping Babies Breathe. Furthermore, result of the study shows that helping babies is an effective ways of addressing lack of knowledge among health workers who are often the first to act in resuscitating newborn with asphyxia. Evaluation study on Helping Babies Breathe implementation conducted in Honduras had knowledge mean score at baseline of 46% which is close to the baseline obtained in this study for both the intervention (42.5%) versus control group (48%)¹⁷. Similarly, the same study found out those health workers who had prior training scored higher at 69%. This is not similar to the result obtained in our study among control group where majority of the participants were exposed to various resuscitation education training but had low pass score. In India, health workers attending HBB training for the first time did not score higher than those who attended HBB training program before. The low baseline knowledge mean score among the first timer reveled in this study is similar to the study but raises concern on the future retention of the HBB resuscitation knowledge¹⁷.

However, the level of knowledge attained at post training declined at the end of three (3) months evaluation and further decreased at one year. Similar to the study finding, a formative educational evaluation of Helping Babies Breathe assessments, in Kenya among the trainers and learners (Nurses and doctors) has shown that the pass rate of knowledge MCQ based test increased from 75% to 95% after similar training intervention. In the same training, the pass rate for the simulated practical skill and competency was 20% for the health workers (learners)^11,14.

Using simulated based environment for teaching and learning had greatly improved the HBB knowledge of the health workers in neonatal resuscitation but this knowledge was not retained at end of one year. Authors of Helping Babies Breathe noted that maintaining the level of the knowledge acquired for long period of time was a challenge. Most of these studies found a rapid deterioration of acquired skills and, to a lesser extent, knowledge, in the months following the training¹⁸.

HBB Psychomotor skill and competency of the health workers

According to the author’s literature review, most of the study conducted on HBB training and evaluation, BMV baseline scores was not obtained¹⁹. In this study, we established our baseline line level psychomotor skill and competency for the intervention before embarking on training the health workers in South Sudan on the simplified HBB protocol. During our study, we administered, BMV checklist, Observational Structural Clinical Examination (OSCE) A and B to all our intervention group baseline, immediate intervention (post) and 3 months follow up while in control group, it was administered at immediate intervention, posttest 0 and 3 months and one year follow up. Most time, OSCE A and B was considered too difficult to be administered to the participants during pre-training period. Based on the pre assessment of the practical skill and competency of health workers in the intervention group, we tailored support to each of the participant’s ability and understanding the training and that facilitated the good result at post training.

Our study demonstrated improvement in practical skills and competency of the health workers after training, despite a significant decrease of the required skills within the period of one year. At three months, the practical skills remained persistently higher but this declined deistically at one year of the evaluation. Despite decreased in the required score among the health workers, the skill was significant to cause impact on neonatal mortality reduction. It is surprising to find out that health workers retained skill and competency at three (3) months follow up. Similar major studies about retention of practical skills and competency conducted in Rwanda and Kenya indicated that it was most time difficult to retain skill and competency at 3 to 6 months after intervention.

Furthermore, another pre and post study, evaluating the development and testing of a performance checklist to assess neonatal resuscitation mega code skill., indicated that despite significant improvements in overall test scores and pass rates following training, scores and improvement were disproportionate for those assessments that involved demonstration of skills (bag-mask ventilation, OSCE A, OSCE B). George miller indicated that, acquisition of knowledge occurs earlier and more easily than acquisition of skills. Miller’s framework of progressively more complex and higher -level thinking moves from knowledge (‘knows’, multiple-choice questionnaire), to demonstration of skills (‘knows how’, bag-mask ventilation), to performance assessments (‘shows how’, OSCE). In our study, participants demonstrated mastery of neonatal resuscitation knowledge, or the ‘knows’ level of Miller’s pyramid, as evidenced by high post-test scores and pass rates for the multiple-choice questionnaire. Health workers struggled more with assessments that required skills and higher-level performance, as evidenced by low score at baseline in both groups. However, at 1 year, practical scores among the health workers followed George Miller of knowledge acquisition. The high score obtained by the health workers at 3 months was quite different from other study result on retention of skill and competency.

An evaluation of staff retention of advanced cardiac life support on basic life skill among the registered nurses in United states using repeated quasi experimental study design infer that practical psychomotor skill and competency tend to diminish within three months earliest after training. However, in contrast, our study finding indicated that health worker retained their practical skills past 3 months’ period. This confirms that HBB is practical course that requires actions with periodic reinforcement of the skill through review, problem solving and self-assessment to ensure higher retention of knowledge and skill learned.

Similarly, a study assessing educational impact of a hospital-based modified neonatal resuscitation program in Ghana among midwives focusing mainly on the retention of midwives’ knowledge, evidence-based neonatal resuscitation practices, competency and short- and long-term educational effects of teaching a neonatal resuscitation program in a hospital setting after training indicated that knowledge and skill remained stable within the period of 9 - 12 months’ post training. This concur mostly with the retention of practical skill and competency found in our study among the health workers. Many of the researchers who conducted similar study advocated for means of retention of knowledge, skill and competencies among the trainee health workers and recommended refresher training courses between post training and implementation.

Early neonatal mortality within 24 hours (ENBM)

The training intervention reveals potential benefit of not only improving the knowledge, skills and competency of the health workers alone but has also impacted on the newborn outcomes. Pre - intervention information indicated that, most of the newborns never received resuscitation according to Helping Babies Breathe protocol. Health workers were not practicing the correct steps in resuscitating newborn with asphyxia including those who require stimulation, bag and mask ventilation for neonates with breathing problems. Additionally, prior to the implementation of our study, health workers were having challenges with proper identification of newborn with breathing problems and initiation of resuscitation within the golden minute. Much of the skills required for neonatal resuscitation was lacking before the implementation of the HBB. When the HBB training introduced and implemented, health workers competent and skill in resuscitation increased by two folds and this skill was retained at three months and 1 year follow up.

The study also documented the trend toward the overall reduction in mortality within24 hours in live births after three months and 1 year of implementation. Our result at intervention site demonstrated decreased ratio of early new born mortality by half within 24 hours of life after receiving resuscitation by trained health workers. Review of neonatal mortality has shown to have decreased at the control site. However, this could not be attributed directly to training of health workers regarding asphyxia, as they did not receive any formal training on HBB. Although, there was notably decreased early new born deaths at intervention site, the short period for baseline and implementation was not enough to make the interpretation of the result conclusive and generalized. We have noted there is need for longer period for baseline and implementation because of variation in the new born data in registers especially at the time political turmoil. Despite the differences outlined above, the study actually demonstrated that training of health workers has effect on the knowledge and practical skill of the health worker in addition to new born mortality reduction.

On the Global context, few studies have demonstrated the long term of effect HBB training of health workers in on early neonatal outcomes. Similar to our study finding, a large before and after design study conducted in Tanzania to determine the early newborn mortality and fresh still births after helping babies Breathe among the birth attendants in eight²⁰ hospital using pre and post study design shows that training and targeted implementation of helping Babies Breathe was associated with significant reduction in primary outcome of early neonatal mortality (within 24 hours) and the rate of fresh stillbirths and early perinatal mortality. The study indicated that early newborn mortality significantly reduced from 13.4% to 7.1 per 1000 live- born deliveries and the reduction in ENM was significant for both normal and low birth weight as well as term and preterm infants. The only differences with our study was that, we excluded stillbirth and look at neonates identified as having breathing problems at birth. The strength of the study is that the majority of the health workers were nurses and midwives who had training on newborns and spent their full time in the maternity and children ward; therefore, they represented a population that typically encounters and manages birth asphyxia as they occur in hospital setting. The documentation of knowledge, skill, competency and birth asphyxia during the pre-training, immediate post-training and at 3 months post-training provided an opportunity to evaluate the same health workers available at the hospital at the time of assessment.

The result of the study was limited by relying on the documentation of newborns in maternity and children ward of both hospitals to determine newborn outcome after pre-training, immediately post-training and at 3 months post-training where trained staff were available to help with documentation. There still exists a large gap in the documentation of medical records and keeping and data use among the health workers, clerks and managers in the hospitals and health facilities in South Sudan. Furthermore, our study was also limited by excluding the still birth and pre mature newborn in our analysis.

Conclusion

The study has highly demonstrated that Health Babies Breathe had significantly improved health worker’s knowledge of neonatal resuscitation, skills and competency as well neonatal mortality over a period of one year. With the ongoing conflict, it was expected that training of health worker on HBB might not have the hypothesized impact but this was proven to be wrong. Knowledge, skill and competency was found to decline after one years. This finding concur with many of the Helping Babies Breathe carried out in low and middle income countries. The implementation of Helping Babies Breathe had a positive effect on the neonatal survival. Overall, there was a significant reduction in the number of neonates dying from asphyxia related illness compared to the pre implementation.

HBB training require minimal equipment for newborn resuscitation and combing this with quality improvement cycle and periodic refresher training will contribute to increased knowledge, skill, competency and most significantly in the reduction of asphyxia related death in hospital setting, and also in long run prepare them for newborn emergencies related to asphyxia. Further, quality improvement cycle and refresher training could minimize the determination of knowledge, skill and competency as seen in the study.

South Sudan could benefit from the scale up of the training project training of health workers in Helping Babies Breathe as it had a very significant impact in reducing the neonatal mortality among newborn with asphyxia within 24 hours. Our training further demonstrated that, health workers even in conflict setting can easily acquire knowledge and retain skills in other normal simulated environment. Closer examination of rolling the educational aspect of Helping Babies Breathe will be paramount in reducing the high mortality among the newborn when combined with quality cycle. Although skills and competency has been proven to be difficult to acquire, careful selection of teaching methodology and focusing on practical aspect of the course during the intervention helped to improve all aspect of knowledge, skill and competency acquisition. Similarly, demonstrated that practical skill like knowledge can easily be acquired and retained.

Most importantly, bag and mask was a challenging part of the Helping Babies Breathe before training of the health workers, which at most times was ignored to be administered pre training to health workers, makingit difficult to focus on the right mastery skill to be taught to the participant. Aware of the difficulty of acquisition of skills and competency as identified pre training, led to focusing on the best approach of educational teaching on bag and mask using various methods to achieve ventilation by golden minute. taking into account the varied training background and clinical exposure to resuscitation education among the health workers. This study has demonstrated high level of practical skills and competency over three months’ period and shown decline to the overall resuscitation knowledge, skill and competency after one year which is which does not remain unique to this study. Several Helping Babies Breathe follow up study raised concern in the decline in basic neonatal resuscitation skill over time. A further large randomized cluster studies is recommended to determine both the effect of training on education retention on knowledge, skill, competency and neonatal mortality. This may prove directly replicable in other similar settings, not only in South Sudan but in other low income countries as well.

Data availability

Underlying data

Data concerning newborn mortality and the performance of health workers are available on OSF. DOI: https://doi.org/10.17605/OSF.IO/3NTJB¹⁶.

Extended data

Consent forms and assessment tools used in this study are available on OSF. DOI: https://doi.org/10.17605/OSF.IO/3NTJB¹⁶.

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).

Reporting guidelines

A completed TREND statement is available on OSF¹⁶.

Grant information

The author(s) declare that no grants were involved in supporting this work.

Acknowledgements

We acknowledge the Government of South Sudan, Ministry of health, Directorate of Reproductive of Health, Juba and Wau teaching Hospital administration, state Ministry of health for their continued support in making the evaluation successful. Our appreciation goes to the health workers (medical doctors, nurses, midwives) in Juba and Wau teaching Hospital for the contribution through their time in accomplishing the result documented in the study. Finally, our Research team for their tireless time through training, simulation rating, data analysis and production of the manuscript. Thank you all.

Faculty Opinions recommended

References

1. Countdown Coverage Writing Group: Countdown to 2015 for maternal, newborn, and child survival. 2015. Reference Source
2. Goldenberg RL, McClure EM, Bann CM: The relationship of intrapartum and antepartum stillbirth rates to measures of obstetric care in developed and developing countries. Acta Obstet Gynecol Scand. 2007; 86(11): 1303–9. PubMed Abstract | Publisher Full Text
3. World Health Organization: Making pregnancy safer: the critical role of the skilled attendant. A joint statement by WHO, ICM and FIGO. Geneva: World Health Organization; 2005. Reference Source
4. Lawn JE, Kerber K, Enweronu-Laryea C, et al.: 3.6 million neonatal deaths--what is progressing and what is not? Semin Perinatol. 2010; 34(6): 371–386. PubMed Abstract | Publisher Full Text
5. Save the Children: Ending Newborn Deaths. London, UK: Author. 2014. Reference Source
6. UNICEF: Committing to Child Survival: A Promise Renewed – Progress Report 2013. New York, 2013. Reference Source
7. Carlo WA, Wright LL, Chomba E, et al.: Educational impact of the neonatal resuscitation program in low-risk delivery centers in a developing country. J Pediatr. 2009; 154(4): 504–508.e5. PubMed Abstract | Publisher Full Text | Free Full Text
8. Lawn JE, Lee AC, Kinney M, et al.: Two million intrapartum-related stillbirths and neonatal deaths: where, why, and what can be done? Int J Gynaecol Obstet. 2009; 107 Suppl 1: S5–18, S19. PubMed Abstract | Publisher Full Text
9. Kak L, Johnson J, McPherson R, et al.: Helping Babies Breathe: Lessons learned guiding the way forward [Internet]. 2015 [cited 2017 Mar 28]. Reference Source
10. Niemeyer S, Keenan WJ, Little GA, et al.: Helping Babies Breathe Learner Workbook. American Academy of Pediatrics, Elk Grove Village, IL, 2010.
11. Singhal N, Lockyer J, Fidler H, et al.: Helping Babies Breathe: global neonatal resuscitation program development and formative educational evaluation. Resuscitation. 2012; 83(1): 90–6. PubMed Abstract | Publisher Full Text
12. Goudar SS, Somannavar MS, Clark R, et al.: Stillbirth and newborn mortality in India after helping babies breathe training. Pediatrics. 2013; 131(2): e344–52. PubMed Abstract | Publisher Full Text
13. American Academy of Pediatrics Helping Babies Breathe. 2014. Reference Source
14. Opiyo N, English M: In-service training for health professionals to improve care of the seriously ill newborn or child in low and middle-income countries (Review). Cochrane Database Syst Rev. 2010; 14(4): CD007071. PubMed Abstract | Publisher Full Text | Free Full Text
15. Pan African Clinical Trial Registry (PACTR). South Africa, PACTR201708002469225, 2017. Reference Source
16. Vunni C: Evaluation of Retention of Knowledge, Skill, Competency of Health Workers One Year After Completion of Helping Babies Breathe Training Program in South Sudan. 2019. http://www.doi.org/10.17605/OSF.IO/3NTJB
17. Seto TL, Tabangin ME, Josyula S, et al.: Educational outcomes of Helping Babies Breathe training at a community hospital in Honduras. Perspect Med Educ. 2015; 4(5): 225–32. PubMed Abstract | Publisher Full Text | Free Full Text
18. Msemo G, Massawe A, Mmbando D, et al.: Newborn mortality and fresh stillbirth rates in Tanzania after helping babies breathe training. Pediatrics. 2013; 131(2): e353–60. PubMed Abstract | Publisher Full Text
19. Kaczorowski J, Levitt C, Hammond M, et al.: Retention of neonatal resuscitation skills and knowledge: a randomized controlled trial. Fam Med. 1998; 30(10): 705–711. PubMed Abstract
20. Darmstadt GL, Kinney MV, Chopra M, et al.: Who has been caring for the baby? Lancet. 2014; 384(9938): 174–88. PubMed Abstract | Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 07 Feb 2019

Author details Author details

¹ College of Public Health Sciences (CPHS), Chulalongkorn University, Bangkok, Thailand
² Department of Gynecology, Juba Teaching Hospital, Juba, Central Equatoria, Sudan

Christopher Vunni Draiko
Roles: Conceptualization, Data Curation, Investigation, Methodology, Project Administration, Resources, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Khemika Yamarat
Roles: Conceptualization, Formal Analysis, Methodology, Project Administration, Supervision, Validation, Writing – Original Draft Preparation, Writing – Review & Editing

Alessio Panza
Roles: Formal Analysis, Investigation, Methodology, Project Administration, Resources, Writing – Original Draft Preparation, Writing – Review & Editing

Judith Draleru
Roles: Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Supervision, Validation, Writing – Original Draft Preparation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (2)

version 2

Revised

Published: 10 Apr 2019, 8:167

https://doi.org/10.12688/f1000research.17560.2

version 1

Published: 07 Feb 2019, 8:167

https://doi.org/10.12688/f1000research.17560.1

Copyright

© 2019 Vunni Draiko C et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

0

SEE MORE DETAILS

CITE

how to cite this article

Vunni Draiko C, Yamarat K, Panza A and Draleru J. Evaluation of retention of knowledge, skill and competency of health workers one year after completion of the Helping Babies Breathe training program in South Sudan [version 2; peer review: 2 approved with reservations]. F1000Research 2019, 8:167 (https://doi.org/10.12688/f1000research.17560.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 2

VERSION 2

PUBLISHED 10 Apr 2019

Revised

Views

4

Reviewer Report 31 Jul 2019

Beena D. Kamath-Rayne, Department of Pediatrics, College of Medicine, University of Cincinnati, Cincinnati, OH, USA

Approved with Reservations

https://doi.org/10.5256/f1000research.20482.r47032

The goal of the authors was to evaluate retention of neonatal resuscitation skills immediately before HBB training and then at three time points after: immediately post-training, 3 months and 1 year after. The authors stated that their results show an ... Continue reading

The goal of the authors was to evaluate retention of neonatal resuscitation skills immediately before HBB training and then at three time points after: immediately post-training, 3 months and 1 year after. The authors stated that their results show an increase in retention in knowledge, skill and competency immediately post-training and 3 months after, but with a decrease by 1 year of training. Again, the authors should be commended for undertaking a study that evaluates retention of skills as such data are needed to make better recommendations for the frequency that practice is needed in different cohorts for retention of skills.

The manuscript is significantly improved from the first submission. However, I did not find the authors to be responsive to all the points of my critique. For example, I am disappointed that the authors did not utilize the list of references that I provided on this topic to do a full literature review about educational outcomes after HBB training that would have been pertinent to their discussion, particularly the Tabingin et al. reference that discusses the performance of different cadres of learners. The manuscript continues to need significant copyediting for basic grammatical, spelling and word choice mistakes. The authors need to review the manuscript to ensure that the references provided at the end of certain statements are correct, and other areas where studies are cited but no reference provided at all. Reference #13 was unclear. There could be better organization to the flow of the writing to tell the story of the study and the use of cited literature to support their findings.

The authors state that the main primary outcome was any improved knowledge, skill and competency of health workers on the HBB protocol. This is vague. Are skills and competency being used interchangeably and how are these being measured? The authors used the knowledge check and skills-based assessments (Objective Structured Clinical Evaluations), but how these assessments correlated with skill and/or competency was vague. These concepts could have been better defined and correlated with specific HBB assessments.

My interpretation of the study design is that in the intervention hospital, HBB training occurred, and then workshop participants were evaluated pre/post and then at 3 and 12 months with the knowledge and skills-based assessments associated with HBB. In the control hospital, no HBB training was offered. Furthermore, in the intervention hospital, bag-mask ventilation drills were used to augment retention of skills over time. Per the manuscript, it appeared that daily drills were done 5 days per week for three months in the intervention hospital, which resulted in retention of skills as evaluated by the HBB skill-based assessments. Since no drills are described after three months’ time, scores on the skills-based assessments were significantly less at one year. If my understanding is correct, this more so reinforces the point made by many other studies that ongoing practice is necessary to maintain skills. The results also beg the question, why did the neonatal mortality rate decline in the control hospital without HBB exposure? Was it the introduction of the skills-based assessments which could also be used as a formative tool?

Long term retention of neonatal resuscitation skills is an important concept that must be studied as there are many unknown factors that could potentially affect competency and skills retention. I continue to applaud the authors for their work and their manuscript has significantly improved. However, I again feel that major revision is necessary before this manuscript is suitable for indexing to fully tell the story of what they did.

Competing Interests: I am the Associate Editor of the 2nd Edition of Helping Babies Breathe.

Reviewer Expertise: neonatal resuscitation, Helping Babies Breathe, educational outcomes

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Respond or Comment

Views

6

Reviewer Report 23 May 2019

Brett D. Nelson, Division of Global Health and Human Rights, Department of Emergency Medicine, Massachusetts General Hospital, Boston, MA, USA

Approved with Reservations

https://doi.org/10.5256/f1000research.20482.r47033

I appreciate the authors efforts in revising their manuscript.

With regard to my earlier concerns that the Abstract's Results section was unclear.... It appears that the authors have just significantly reduced their Results section to two sentences

I appreciate the authors efforts in revising their manuscript.

With regard to my earlier concerns that the Abstract's Results section was unclear.... It appears that the authors have just significantly reduced their Results section to two sentences that contain no specific data. I don't believe this is what either of the Reviewers wanted. The two sentences in the revised Abstract's Results also, unfortunately, remain unclear. For example, "Helping Babies Breathe has shown a significant increase in knowledge, skill and competency post-test and three months." Respectfully, this is inaccurately and very imprecisely worded. "HBB" has not shown an increase - the health workers have. "Post-test and three months" is also quite unclear. I imagine the authors mean that there was a significant increase in knowledge and skills from immediately post-training to three months post-training. Also, how is "competency" different than knowledge and skills? I don't think "competency" is a third, separate indicator separate from knowledge and skills, therefore, it should not be included here as if it were. In the second sentence, "...competency of the health within....," I believe the authors intended to write "health WORKERS" rather than just "health." Furthermore, please don't only write that "there was marked decline over [a] 1-year period." Please provide readers here with your specific data. Regretfully, the authors' revised abstract is now much less clear and less specific than it was previously. Additionally, having just 2 sentences in the Results (compared to the MUCH longer Methods section) doesn't make sense to me. Please provide readers with your key findings in your Abstract's Results. Feel free to examine similar published HBB articles for examples of what can be useful to include in the Abstract's Results.
In the Abstract, the revised Conclusion is also repetitive with the Results; they nearly restate the same thing verbatim. These sentences - or at least their topics - would be appropriate for a Conclusion IF the Results section then focuses on providing the concrete, specific data. However, currently, the Results section of the Abstract does not provide any concrete data. The English in both of these two sentences in the revised Abstract's Conclusion also need to be corrected - if the sentences are retained.
I appreciate the authors adding additional literature to the Intro. In the first paragraph, I wouldn't have necessarily added the worldwide statistics ("136 million...") in between all of the sentences that are otherwise talking specifically about South Sudan. I would either start by talking globally and working towards South Sudan, or vice versa, but not going back and forth. I would also advise not writing that the number of newborns [not infants] born is expected to increase to nearly 137 million "BY 2016." Since 2016 is past, I wouldn't talk about it in future tense, even if that's the most recent data available.
I don't understand the title in Figure 2: "Registry birth at intervention and Control." I think the English needs to be revised slightly. (I realize publishing a scientific article in anything but one's primary native tongue is incredibly difficult, and I commend the authors' efforts. However, I do feel this paper needs to have a native English speaker with technical writing skills carefully review it. There are several minor issues throughout the paper, figures, and tables.)
The new Table 1 is very helpful. Thank you.
The authors' efforts in revising the Discussion is greatly appreciated. However, the first 6 paragraphs of the revised Discussion will need heavy English editing (as will the revised Conclusion).
I also appreciate the authors' efforts in improving the Conclusion. However, the revised Conclusion is now four large paragraphs in length (compared to one in the original article). This is significantly longer than most Conclusions, which are intended to simply restate the key take-home messages rather than to continue the Discussion. Please feel free to review other published HBB articles for examples of appropriate Conclusion length and content. If the authors feel the current content of their revised Conclusion is essential and not a restatement of what is in the Discussion, then much of it can be integrated into the Discussion. But a Conclusion should typically be a brief, concise summary of the main take-home messages and no new ideas should typically be presented in the Conclusion; all new ideas and any discussion of the study results should be presented in the paper's Discussion.

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Newborn medicine, Helping Babies Breathe, global health.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 07 Feb 2019

Views

30

Reviewer Report 12 Mar 2019

Beena D. Kamath-Rayne, Department of Pediatrics, College of Medicine, University of Cincinnati, Cincinnati, OH, USA

Not Approved

https://doi.org/10.5256/f1000research.19204.r44164

Long term retention of neonatal resuscitation skills is an important concept that must be studied as there are many unknown factors that could potentially affect competency and skills retention. Educational efficiency is an important component of the Formula for Survival, ... Continue reading

Long term retention of neonatal resuscitation skills is an important concept that must be studied as there are many unknown factors that could potentially affect competency and skills retention. Educational efficiency is an important component of the Formula for Survival, which denotes potential survival being a product of medical science x educational efficiency x local implementation. The authors are to be commended to undertaking a study that specifically evaluates retention of skills in this population in South Sudan. However, while the topic is important, the way in which these authors have presented their data and results needs to be improved.

My interpretation of the study design is that in the intervention hospital, HBB training occurred, and then workshop participants were evaluated pre/post and then at 3 and 12 months with the knowledge and skills-based assessments associated with HBB. In the control hospital, no HBB training was offered. Furthermore, in the intervention hospital, bag-mask ventilation drills were used to augment retention of skills over time.

The manuscript could use some copyediting to fix basic grammatical, spelling and word choice mistakes. There is also some confusion about the secondary outcome of neonatal mortality, which the authors refer to as neonatal mortality due to birth asphyxia. In the abstract, for example, this is only called neonatal mortality. If this is asphyxia-related mortality only the authors could do a better job of determining how this specifically was assessed as the cause of death rather that something else. Also, some babies, like premature babies, can have both asphyxia and prematurity as contributors to their deaths. How was this sorted out?

Further detail on the methods/results would be helpful to improve the readers' interpretation of the investigators' findings. With the delivery room observations and calculation of neonatal mortality, was it known whether the individual leading the resuscitation had been trained in HBB? Given the frequent turnover in such settings, it would be important to know whether the resuscitator was in the study protocol or not. It would also be helpful to know how many individuals in each hospital did not consent to the study; what proportion of health providers potentially eligible for the study in each hospital who take care of newborns were in the study?

What types of health workers were included in the study, and did they have any previous exposure to neonatal resuscitation or simulation? What individuals refused to be in the study or were lost to follow-up? Were they predominantly one type of health provider? Tabangin et al.¹ showed that doctors performed better in simulated evaluations (OSCE B) due to having had previous experience with neonatal resuscitation and simulation training, which allowed their performance to improve at a greater level than nurses, who had not had prior experience.

How was it known that the daily bag-mask ventilation drills were performed, and that the intervention group received the intervention? The CONSORT diagram was difficult to follow as the numbers did not appear to add up correctly.

The authors could be much more rigorous in their literature search to support their arguments. Because of their lack of a complete literature review, the study is not able to fully analyze their results within the context of the previously studies and their nuances. I believe the study was done with HBB 1st Edition materials, however, the correct citation was not reported; there is a publication date of 2014 which would be incorrect for either the 1st or 2nd Edition. This is important to distinguish because the 2nd Edition has greater emphasis about an ongoing system for practice, an improved OSCE, and more standardization about giving feedback and debriefing after the OSCES. References 10 and 11 do not support their points regarding an immediate increase in knowledge and skills showing improvements in neonatal outcome. Some of the most important papers regarding retention of skills both in the short and long term after HBB training are not cited. These are referenced below.

The findings that the skills declined in the intervention group does not fit with what would be expected nor with what has been seen in previous studies. It also does not fit with neonatal mortality decreasing if the skills are not being performed as well. It would be helpful for the authors to hypothesize why they had such markedly different findings from all the other studies that have been published. It also underscores the importance of knowing whether the intervention such as the ongoing practice and drills were being completed as desired.

In regards to the greater numbers of babies being "assisted to breathe" during the study period - which the authors also referred to as "resuscitation" - could they be more specific about what this resuscitation entailed? Some studies have shown that fewer babies require bag-mask ventilation after HBB because more babies are being stimulated appropriately and respond to that alone. If they are referring to bag-mask ventilation, can they comment on whether the preceding steps to bag-mask ventilation were appropriately performed or not, or the health providers just went right to bag-mask ventilation?

While the authors are to be applauded for their efforts on this study, major revisions are required before this study can be considered worthy for indexing. I am concerned that their studies are framed in a dangerous way, to suggest that ongoing practice is not beneficial for retention of skills, that goes against both common sense and what has been shown in other studies. The authors need to provide some justification for these findings.

Is the work clearly and accurately presented and does it cite the current literature?

No
Is the study design appropriate and is the work technically sound?

No
Are sufficient details of methods and analysis provided to allow replication by others?

No
If applicable, is the statistical analysis and its interpretation appropriate?

No
Are all the source data underlying the results available to ensure full reproducibility?

No
Are the conclusions drawn adequately supported by the results?

No

References

1. Tabangin M, Josyula S, Taylor K, Vasquez J, et al.: Resuscitation skills after Helping Babies Breathe training: a comparison of varying practice frequency and impact on retention of skills in different types of providers. International Health. 2018; 10 (3): 163-171 Publisher Full Text
2. Mduma E, Ersdal H, Svensen E, Kidanto H, et al.: Frequent brief on-site simulation training and reduction in 24-h neonatal mortality--an educational intervention study.Resuscitation. 2015; 93: 1-7 PubMed Abstract | Publisher Full Text
3. Seto TL, Tabangin ME, Josyula S, Taylor KK, et al.: Educational outcomes of Helping Babies Breathe training at a community hospital in Honduras.Perspect Med Educ. 2015; 4 (5): 225-32 PubMed Abstract | Publisher Full Text
4. Rule ARL, Tabangin M, Cheruiyot D, Mueri P, et al.: The Call and the Challenge of Pediatric Resuscitation and Simulation Research in Low-Resource Settings.Simul Healthc. 2017; 12 (6): 402-406 PubMed Abstract | Publisher Full Text
5. Dol J, Campbell-Yeo M, Murphy GT, Aston M, et al.: The impact of the Helping Babies Survive program on neonatal outcomes and health provider skills: a systematic review.JBI Database System Rev Implement Rep. 2018; 16 (3): 701-737 PubMed Abstract | Publisher Full Text
6. Seto TL, Tabangin ME, Taylor KK, Josyula S, et al.: Breaking Down the Objective Structured Clinical Examination: An Evaluation of the Helping Babies Breathe OSCEs.Simul Healthc. 2017; 12 (4): 226-232 PubMed Abstract | Publisher Full Text
7. Eblovi D, Kelly P, Afua G, Agyapong S, et al.: Retention and use of newborn resuscitation skills following a series of helping babies breathe trainings for midwives in rural Ghana.Glob Health Action. 2017; 10 (1): 1387985 PubMed Abstract | Publisher Full Text
8. Rule ARL, Maina E, Cheruiyot D, Mueri P, et al.: Using quality improvement to decrease birth asphyxia rates after 'Helping Babies Breathe' training in Kenya.Acta Paediatr. 2017; 106 (10): 1666-1673 PubMed Abstract | Publisher Full Text
9. Kc A, Wrammert J, Clark RB, Ewald U, et al.: Reducing Perinatal Mortality in Nepal Using Helping Babies Breathe.Pediatrics. 137 (6). PubMed Abstract | Publisher Full Text
10. ME Arabi A, Ibrahim S, Ahmed S, MacGinnea F, et al.: Skills retention in Sudanese village midwives 1 year following Helping Babies Breathe training. Archives of Disease in Childhood. 2016; 101 (5): 439-442 Publisher Full Text
11. Arabi A, Ibrahim S, Manar A, Abdalla M, et al.: Perinatal outcomes following Helping Babies Breathe training and regular peer–peer skills practice among village midwives in Sudan. Archives of Disease in Childhood. 2018; 103 (1): 24-27 Publisher Full Text

Competing Interests: I am the associate editor for the second edition of Helping Babies Breathe.

Reviewer Expertise: neonatal resuscitation, Helping Babies Breathe, educational outcomes

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

CITE

Report a concern

Respond or Comment

Views

25

Reviewer Report 22 Feb 2019

Brett D. Nelson, Division of Global Health and Human Rights, Department of Emergency Medicine, Massachusetts General Hospital, Boston, MA, USA

Approved with Reservations

https://doi.org/10.5256/f1000research.19204.r44166

I congratulate and thank the authors for their important work on newborn health in South Sudan. Having worked on various HBB projects, and having worked for a couple of years in South Sudan, I really enjoyed reading your article.
... Continue reading

I congratulate and thank the authors for their important work on newborn health in South Sudan. Having worked on various HBB projects, and having worked for a couple of years in South Sudan, I really enjoyed reading your article.

The paper is great. However, I would like to humbly provide a strong recommendation that the current abstract please be updated by the authors. The abstract's Results and Conclusion sections currently seem to suggest that the Control group faired better than the Intervention group and had better retention than the Intervention group. (E.g., "Conclusions: Health workers in the control hospital had improvement in retention of their knowledge, skill and competency. Newborn mortality decreased in both hospitals.")

Respectfully, I think this could be inadvertently misleading to many readers. While the Control group's scores did increase slightly (and statistically significantly, for the possible reasons discussed in the paper), the Control group's scores were always much lower than the Intervention group's scores (since the Controls didn't receive the training yet). So I would argue that the Control group never showed any significant knowledge or skills in HBB and, therefore, never "retained" this knowledge and skills. Because they hadn't yet been formally taught the knowledge and skills (again, because they were Controls), they never had the knowledge and skills to lose and never had the ability to "retain" the HBB skills.

Meanwhile, the Intervention group did acquire substantial knowledge and skills following the successful training and lost some knowledge and skills over time, which we've seen in other HBB studies -- unless active measures are taken to retain knowledge and skills via on-the-job training, daily practice, refresher training, supervisory visits, etc. In short, I think the small improvement in the Control group's comparatively low scores should not be the primary focus of the abstract's Results and Conclusion sections, and I think it may be inadvertently misleading to readers to simply conclude that "Health workers in the control hospital had improvement in RETENTION of their knowledge, skill and competency" (my emphasis added). This could give the mis-perception that the Controls did better than the Intervention group.

I hope you don't mind the suggestion, and hopefully this might be some helpful feedback to further strengthen your paper. And again, I congratulate and thank the authors for their important work.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

No

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Newborn medicine, Helping Babies Breathe, global health

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Respond or Comment

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 07 Feb 2019

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 10 Apr 19	read	read
Version 1 07 Feb 19	read	read

Brett D. Nelson, Massachusetts General Hospital, Boston, USA
Beena D. Kamath-Rayne, University of Cincinnati, Cincinnati, USA

Comments on this article

All Comments(0)

Add a comment

Sign up for content alerts

Browse by related subjects

Back to all reports

Reviewer Report

4 Views

31 Jul 2019 | for Version 2

Beena D. Kamath-Rayne, Department of Pediatrics, College of Medicine, University of Cincinnati, Cincinnati, OH, USA

4 Views Cite this report Responses(0)

Approved With Reservations

The goal of the authors was to evaluate retention of neonatal resuscitation skills immediately before HBB training and then at three time points after: immediately post-training, 3 months and 1 year after. The authors stated that their results show an increase in retention in knowledge, skill and competency immediately post-training and 3 months after, but with a decrease by 1 year of training. Again, the authors should be commended for undertaking a study that evaluates retention of skills as such data are needed to make better recommendations for the frequency that practice is needed in different cohorts for retention of skills.

The manuscript is significantly improved from the first submission. However, I did not find the authors to be responsive to all the points of my critique. For example, I am disappointed that the authors did not utilize the list of references that I provided on this topic to do a full literature review about educational outcomes after HBB training that would have been pertinent to their discussion, particularly the Tabingin et al. reference that discusses the performance of different cadres of learners. The manuscript continues to need significant copyediting for basic grammatical, spelling and word choice mistakes. The authors need to review the manuscript to ensure that the references provided at the end of certain statements are correct, and other areas where studies are cited but no reference provided at all. Reference #13 was unclear. There could be better organization to the flow of the writing to tell the story of the study and the use of cited literature to support their findings.

The authors state that the main primary outcome was any improved knowledge, skill and competency of health workers on the HBB protocol. This is vague. Are skills and competency being used interchangeably and how are these being measured? The authors used the knowledge check and skills-based assessments (Objective Structured Clinical Evaluations), but how these assessments correlated with skill and/or competency was vague. These concepts could have been better defined and correlated with specific HBB assessments.

My interpretation of the study design is that in the intervention hospital, HBB training occurred, and then workshop participants were evaluated pre/post and then at 3 and 12 months with the knowledge and skills-based assessments associated with HBB. In the control hospital, no HBB training was offered. Furthermore, in the intervention hospital, bag-mask ventilation drills were used to augment retention of skills over time. Per the manuscript, it appeared that daily drills were done 5 days per week for three months in the intervention hospital, which resulted in retention of skills as evaluated by the HBB skill-based assessments. Since no drills are described after three months’ time, scores on the skills-based assessments were significantly less at one year. If my understanding is correct, this more so reinforces the point made by many other studies that ongoing practice is necessary to maintain skills. The results also beg the question, why did the neonatal mortality rate decline in the control hospital without HBB exposure? Was it the introduction of the skills-based assessments which could also be used as a formative tool?

Long term retention of neonatal resuscitation skills is an important concept that must be studied as there are many unknown factors that could potentially affect competency and skills retention. I continue to applaud the authors for their work and their manuscript has significantly improved. However, I again feel that major revision is necessary before this manuscript is suitable for indexing to fully tell the story of what they did.

Competing Interests

I am the Associate Editor of the 2nd Edition of Helping Babies Breathe.

Reviewer Expertise

neonatal resuscitation, Helping Babies Breathe, educational outcomes

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

6 Views

23 May 2019 | for Version 2

Brett D. Nelson, Division of Global Health and Human Rights, Department of Emergency Medicine, Massachusetts General Hospital, Boston, MA, USA

6 Views Cite this report Responses(0)

Approved With Reservations

I appreciate the authors efforts in revising their manuscript.

With regard to my earlier concerns that the Abstract's Results section was unclear.... It appears that the authors have just significantly reduced their Results section to two sentences that contain no specific data. I don't believe this is what either of the Reviewers wanted. The two sentences in the revised Abstract's Results also, unfortunately, remain unclear. For example, "Helping Babies Breathe has shown a significant increase in knowledge, skill and competency post-test and three months." Respectfully, this is inaccurately and very imprecisely worded. "HBB" has not shown an increase - the health workers have. "Post-test and three months" is also quite unclear. I imagine the authors mean that there was a significant increase in knowledge and skills from immediately post-training to three months post-training. Also, how is "competency" different than knowledge and skills? I don't think "competency" is a third, separate indicator separate from knowledge and skills, therefore, it should not be included here as if it were. In the second sentence, "...competency of the health within....," I believe the authors intended to write "health WORKERS" rather than just "health." Furthermore, please don't only write that "there was marked decline over [a] 1-year period." Please provide readers here with your specific data. Regretfully, the authors' revised abstract is now much less clear and less specific than it was previously. Additionally, having just 2 sentences in the Results (compared to the MUCH longer Methods section) doesn't make sense to me. Please provide readers with your key findings in your Abstract's Results. Feel free to examine similar published HBB articles for examples of what can be useful to include in the Abstract's Results.
In the Abstract, the revised Conclusion is also repetitive with the Results; they nearly restate the same thing verbatim. These sentences - or at least their topics - would be appropriate for a Conclusion IF the Results section then focuses on providing the concrete, specific data. However, currently, the Results section of the Abstract does not provide any concrete data. The English in both of these two sentences in the revised Abstract's Conclusion also need to be corrected - if the sentences are retained.
I appreciate the authors adding additional literature to the Intro. In the first paragraph, I wouldn't have necessarily added the worldwide statistics ("136 million...") in between all of the sentences that are otherwise talking specifically about South Sudan. I would either start by talking globally and working towards South Sudan, or vice versa, but not going back and forth. I would also advise not writing that the number of newborns [not infants] born is expected to increase to nearly 137 million "BY 2016." Since 2016 is past, I wouldn't talk about it in future tense, even if that's the most recent data available.
I don't understand the title in Figure 2: "Registry birth at intervention and Control." I think the English needs to be revised slightly. (I realize publishing a scientific article in anything but one's primary native tongue is incredibly difficult, and I commend the authors' efforts. However, I do feel this paper needs to have a native English speaker with technical writing skills carefully review it. There are several minor issues throughout the paper, figures, and tables.)
The new Table 1 is very helpful. Thank you.
The authors' efforts in revising the Discussion is greatly appreciated. However, the first 6 paragraphs of the revised Discussion will need heavy English editing (as will the revised Conclusion).
I also appreciate the authors' efforts in improving the Conclusion. However, the revised Conclusion is now four large paragraphs in length (compared to one in the original article). This is significantly longer than most Conclusions, which are intended to simply restate the key take-home messages rather than to continue the Discussion. Please feel free to review other published HBB articles for examples of appropriate Conclusion length and content. If the authors feel the current content of their revised Conclusion is essential and not a restatement of what is in the Discussion, then much of it can be integrated into the Discussion. But a Conclusion should typically be a brief, concise summary of the main take-home messages and no new ideas should typically be presented in the Conclusion; all new ideas and any discussion of the study results should be presented in the paper's Discussion.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Newborn medicine, Helping Babies Breathe, global health.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

30 Views

12 Mar 2019 | for Version 1

Beena D. Kamath-Rayne, Department of Pediatrics, College of Medicine, University of Cincinnati, Cincinnati, OH, USA

30 Views Cite this report Responses(0)

Not Approved

Long term retention of neonatal resuscitation skills is an important concept that must be studied as there are many unknown factors that could potentially affect competency and skills retention. Educational efficiency is an important component of the Formula for Survival, which denotes potential survival being a product of medical science x educational efficiency x local implementation. The authors are to be commended to undertaking a study that specifically evaluates retention of skills in this population in South Sudan. However, while the topic is important, the way in which these authors have presented their data and results needs to be improved.

My interpretation of the study design is that in the intervention hospital, HBB training occurred, and then workshop participants were evaluated pre/post and then at 3 and 12 months with the knowledge and skills-based assessments associated with HBB. In the control hospital, no HBB training was offered. Furthermore, in the intervention hospital, bag-mask ventilation drills were used to augment retention of skills over time.

The manuscript could use some copyediting to fix basic grammatical, spelling and word choice mistakes. There is also some confusion about the secondary outcome of neonatal mortality, which the authors refer to as neonatal mortality due to birth asphyxia. In the abstract, for example, this is only called neonatal mortality. If this is asphyxia-related mortality only the authors could do a better job of determining how this specifically was assessed as the cause of death rather that something else. Also, some babies, like premature babies, can have both asphyxia and prematurity as contributors to their deaths. How was this sorted out?

Further detail on the methods/results would be helpful to improve the readers' interpretation of the investigators' findings. With the delivery room observations and calculation of neonatal mortality, was it known whether the individual leading the resuscitation had been trained in HBB? Given the frequent turnover in such settings, it would be important to know whether the resuscitator was in the study protocol or not. It would also be helpful to know how many individuals in each hospital did not consent to the study; what proportion of health providers potentially eligible for the study in each hospital who take care of newborns were in the study?

What types of health workers were included in the study, and did they have any previous exposure to neonatal resuscitation or simulation? What individuals refused to be in the study or were lost to follow-up? Were they predominantly one type of health provider? Tabangin et al.¹ showed that doctors performed better in simulated evaluations (OSCE B) due to having had previous experience with neonatal resuscitation and simulation training, which allowed their performance to improve at a greater level than nurses, who had not had prior experience.

How was it known that the daily bag-mask ventilation drills were performed, and that the intervention group received the intervention? The CONSORT diagram was difficult to follow as the numbers did not appear to add up correctly.

The authors could be much more rigorous in their literature search to support their arguments. Because of their lack of a complete literature review, the study is not able to fully analyze their results within the context of the previously studies and their nuances. I believe the study was done with HBB 1st Edition materials, however, the correct citation was not reported; there is a publication date of 2014 which would be incorrect for either the 1st or 2nd Edition. This is important to distinguish because the 2nd Edition has greater emphasis about an ongoing system for practice, an improved OSCE, and more standardization about giving feedback and debriefing after the OSCES. References 10 and 11 do not support their points regarding an immediate increase in knowledge and skills showing improvements in neonatal outcome. Some of the most important papers regarding retention of skills both in the short and long term after HBB training are not cited. These are referenced below.

The findings that the skills declined in the intervention group does not fit with what would be expected nor with what has been seen in previous studies. It also does not fit with neonatal mortality decreasing if the skills are not being performed as well. It would be helpful for the authors to hypothesize why they had such markedly different findings from all the other studies that have been published. It also underscores the importance of knowing whether the intervention such as the ongoing practice and drills were being completed as desired.

In regards to the greater numbers of babies being "assisted to breathe" during the study period - which the authors also referred to as "resuscitation" - could they be more specific about what this resuscitation entailed? Some studies have shown that fewer babies require bag-mask ventilation after HBB because more babies are being stimulated appropriately and respond to that alone. If they are referring to bag-mask ventilation, can they comment on whether the preceding steps to bag-mask ventilation were appropriately performed or not, or the health providers just went right to bag-mask ventilation?

While the authors are to be applauded for their efforts on this study, major revisions are required before this study can be considered worthy for indexing. I am concerned that their studies are framed in a dangerous way, to suggest that ongoing practice is not beneficial for retention of skills, that goes against both common sense and what has been shown in other studies. The authors need to provide some justification for these findings.

Is the work clearly and accurately presented and does it cite the current literature?

No
Is the study design appropriate and is the work technically sound?

No
Are sufficient details of methods and analysis provided to allow replication by others?

No
If applicable, is the statistical analysis and its interpretation appropriate?

No
Are all the source data underlying the results available to ensure full reproducibility?

No
Are the conclusions drawn adequately supported by the results?

No

References

1. Tabangin M, Josyula S, Taylor K, Vasquez J, et al.: Resuscitation skills after Helping Babies Breathe training: a comparison of varying practice frequency and impact on retention of skills in different types of providers. International Health. 2018; 10 (3): 163-171 Publisher Full Text
2. Mduma E, Ersdal H, Svensen E, Kidanto H, et al.: Frequent brief on-site simulation training and reduction in 24-h neonatal mortality--an educational intervention study.Resuscitation. 2015; 93: 1-7 PubMed Abstract | Publisher Full Text
3. Seto TL, Tabangin ME, Josyula S, Taylor KK, et al.: Educational outcomes of Helping Babies Breathe training at a community hospital in Honduras.Perspect Med Educ. 2015; 4 (5): 225-32 PubMed Abstract | Publisher Full Text
4. Rule ARL, Tabangin M, Cheruiyot D, Mueri P, et al.: The Call and the Challenge of Pediatric Resuscitation and Simulation Research in Low-Resource Settings.Simul Healthc. 2017; 12 (6): 402-406 PubMed Abstract | Publisher Full Text
5. Dol J, Campbell-Yeo M, Murphy GT, Aston M, et al.: The impact of the Helping Babies Survive program on neonatal outcomes and health provider skills: a systematic review.JBI Database System Rev Implement Rep. 2018; 16 (3): 701-737 PubMed Abstract | Publisher Full Text
6. Seto TL, Tabangin ME, Taylor KK, Josyula S, et al.: Breaking Down the Objective Structured Clinical Examination: An Evaluation of the Helping Babies Breathe OSCEs.Simul Healthc. 2017; 12 (4): 226-232 PubMed Abstract | Publisher Full Text
7. Eblovi D, Kelly P, Afua G, Agyapong S, et al.: Retention and use of newborn resuscitation skills following a series of helping babies breathe trainings for midwives in rural Ghana.Glob Health Action. 2017; 10 (1): 1387985 PubMed Abstract | Publisher Full Text
8. Rule ARL, Maina E, Cheruiyot D, Mueri P, et al.: Using quality improvement to decrease birth asphyxia rates after 'Helping Babies Breathe' training in Kenya.Acta Paediatr. 2017; 106 (10): 1666-1673 PubMed Abstract | Publisher Full Text
9. Kc A, Wrammert J, Clark RB, Ewald U, et al.: Reducing Perinatal Mortality in Nepal Using Helping Babies Breathe.Pediatrics. 137 (6). PubMed Abstract | Publisher Full Text
10. ME Arabi A, Ibrahim S, Ahmed S, MacGinnea F, et al.: Skills retention in Sudanese village midwives 1 year following Helping Babies Breathe training. Archives of Disease in Childhood. 2016; 101 (5): 439-442 Publisher Full Text
11. Arabi A, Ibrahim S, Manar A, Abdalla M, et al.: Perinatal outcomes following Helping Babies Breathe training and regular peer–peer skills practice among village midwives in Sudan. Archives of Disease in Childhood. 2018; 103 (1): 24-27 Publisher Full Text

Competing Interests

I am the associate editor for the second edition of Helping Babies Breathe.

Reviewer Expertise

neonatal resuscitation, Helping Babies Breathe, educational outcomes

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

25 Views

22 Feb 2019 | for Version 1

Brett D. Nelson, Division of Global Health and Human Rights, Department of Emergency Medicine, Massachusetts General Hospital, Boston, MA, USA

25 Views Cite this report Responses(0)

Approved With Reservations

I congratulate and thank the authors for their important work on newborn health in South Sudan. Having worked on various HBB projects, and having worked for a couple of years in South Sudan, I really enjoyed reading your article.

The paper is great. However, I would like to humbly provide a strong recommendation that the current abstract please be updated by the authors. The abstract's Results and Conclusion sections currently seem to suggest that the Control group faired better than the Intervention group and had better retention than the Intervention group. (E.g., "Conclusions: Health workers in the control hospital had improvement in retention of their knowledge, skill and competency. Newborn mortality decreased in both hospitals.")

Respectfully, I think this could be inadvertently misleading to many readers. While the Control group's scores did increase slightly (and statistically significantly, for the possible reasons discussed in the paper), the Control group's scores were always much lower than the Intervention group's scores (since the Controls didn't receive the training yet). So I would argue that the Control group never showed any significant knowledge or skills in HBB and, therefore, never "retained" this knowledge and skills. Because they hadn't yet been formally taught the knowledge and skills (again, because they were Controls), they never had the knowledge and skills to lose and never had the ability to "retain" the HBB skills.

Meanwhile, the Intervention group did acquire substantial knowledge and skills following the successful training and lost some knowledge and skills over time, which we've seen in other HBB studies -- unless active measures are taken to retain knowledge and skills via on-the-job training, daily practice, refresher training, supervisory visits, etc. In short, I think the small improvement in the Control group's comparatively low scores should not be the primary focus of the abstract's Results and Conclusion sections, and I think it may be inadvertently misleading to readers to simply conclude that "Health workers in the control hospital had improvement in RETENTION of their knowledge, skill and competency" (my emphasis added). This could give the mis-perception that the Controls did better than the Intervention group.

I hope you don't mind the suggestion, and hopefully this might be some helpful feedback to further strengthen your paper. And again, I congratulate and thank the authors for their important work.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

No

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Newborn medicine, Helping Babies Breathe, global health

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Respond to this report

Responses (0)

[1] 1. Countdown Coverage Writing Group: Countdown to 2015 for maternal, newborn, and child survival. 2015. Reference Source

[2] 2. Goldenberg RL, McClure EM, Bann CM: The relationship of intrapartum and antepartum stillbirth rates to measures of obstetric care in developed and developing countries. Acta Obstet Gynecol Scand. 2007; 86(11): 1303–9. PubMed Abstract | Publisher Full Text

[3] 3. World Health Organization: Making pregnancy safer: the critical role of the skilled attendant. A joint statement by WHO, ICM and FIGO. Geneva: World Health Organization; 2005. Reference Source

[4] 4. Lawn JE, Kerber K, Enweronu-Laryea C, et al.: 3.6 million neonatal deaths--what is progressing and what is not? Semin Perinatol. 2010; 34(6): 371–386. PubMed Abstract | Publisher Full Text

[5] 5. Save the Children: Ending Newborn Deaths. London, UK: Author. 2014. Reference Source

[6] 6. UNICEF: Committing to Child Survival: A Promise Renewed – Progress Report 2013. New York, 2013. Reference Source

[7] 7. Carlo WA, Wright LL, Chomba E, et al.: Educational impact of the neonatal resuscitation program in low-risk delivery centers in a developing country. J Pediatr. 2009; 154(4): 504–508.e5. PubMed Abstract | Publisher Full Text | Free Full Text

[8] 8. Lawn JE, Lee AC, Kinney M, et al.: Two million intrapartum-related stillbirths and neonatal deaths: where, why, and what can be done? Int J Gynaecol Obstet. 2009; 107 Suppl 1: S5–18, S19. PubMed Abstract | Publisher Full Text

[9] 9. Kak L, Johnson J, McPherson R, et al.: Helping Babies Breathe: Lessons learned guiding the way forward [Internet]. 2015 [cited 2017 Mar 28]. Reference Source

[10] 10. Niemeyer S, Keenan WJ, Little GA, et al.: Helping Babies Breathe Learner Workbook. American Academy of Pediatrics, Elk Grove Village, IL, 2010.

[11] 11. Singhal N, Lockyer J, Fidler H, et al.: Helping Babies Breathe: global neonatal resuscitation program development and formative educational evaluation. Resuscitation. 2012; 83(1): 90–6. PubMed Abstract | Publisher Full Text

[12] 12. Goudar SS, Somannavar MS, Clark R, et al.: Stillbirth and newborn mortality in India after helping babies breathe training. Pediatrics. 2013; 131(2): e344–52. PubMed Abstract | Publisher Full Text

[13] 13. American Academy of Pediatrics Helping Babies Breathe. 2014. Reference Source

[14] 14. Opiyo N, English M: In-service training for health professionals to improve care of the seriously ill newborn or child in low and middle-income countries (Review). Cochrane Database Syst Rev. 2010; 14(4): CD007071. PubMed Abstract | Publisher Full Text | Free Full Text

[15] 15. Pan African Clinical Trial Registry (PACTR). South Africa, PACTR201708002469225, 2017. Reference Source

[16] 16. Vunni C: Evaluation of Retention of Knowledge, Skill, Competency of Health Workers One Year After Completion of Helping Babies Breathe Training Program in South Sudan. 2019. http://www.doi.org/10.17605/OSF.IO/3NTJB

[17] 17. Seto TL, Tabangin ME, Josyula S, et al.: Educational outcomes of Helping Babies Breathe training at a community hospital in Honduras. Perspect Med Educ. 2015; 4(5): 225–32. PubMed Abstract | Publisher Full Text | Free Full Text

[18] 18. Msemo G, Massawe A, Mmbando D, et al.: Newborn mortality and fresh stillbirth rates in Tanzania after helping babies breathe training. Pediatrics. 2013; 131(2): e353–60. PubMed Abstract | Publisher Full Text

[19] 19. Kaczorowski J, Levitt C, Hammond M, et al.: Retention of neonatal resuscitation skills and knowledge: a randomized controlled trial. Fam Med. 1998; 30(10): 705–711. PubMed Abstract

[20] 20. Darmstadt GL, Kinney MV, Chopra M, et al.: Who has been caring for the baby? Lancet. 2014; 384(9938): 174–88. PubMed Abstract | Publisher Full Text

Evaluation of retention of knowledge, skill and competency of health workers one year after completion of the Helping Babies Breathe training program in South Sudan

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Methods

Trial background

Sample size

Selection criteria

Recruitment setting

Location of data collection

Period of recruitment

Specific objectives and hypotheses

Primary and secondary outcome measures

Data collection method and instruments

Validity of HBB instruments

Retention evaluation

Training of trainers of trainers

Selection and training of health workers

Content of training

Health worker assessment

Assessment for simple newborn resuscitation

Assessment for complex newborn resuscitation

Daily bag and mask drills Assessment evaluation

The self-evaluation checklist

Hospital registry and forms

Statistical analysis

Results

Figure 1. Flow chart for recruitment and allocation health workers.

Characteristics of health workers

Table 1. Demographic characteristics.

Status of retention of knowledge

Table 2. Retention of HBB-trained health worker’s knowledge, psychomotor skills and competency.

Figure 2. Flow chart for birth registry pre- and post-implementation.

Status of retention of skill among the health workers

Status of health worker’s competency for simple resuscitation

Status of health worker’s competency for complex resuscitation

Status of early newborn mortality

Table 3. Early newborn mortality.

Discussion

HBB Knowledge of the health workers

HBB Psychomotor skill and competency of the health workers

Early neonatal mortality within 24 hours (ENBM)

Conclusion

Data availability

Underlying data

Extended data

Reporting guidelines

Grant information

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated