Validation of the Dominican system for measuring early childhood development

Laura V. Sánchez-Vincitore; María Angélica Alonso Pellerano; María Elena Valdez; Angie Sabrina Jiménez; Carlos B Ruiz-Matuk; Arachu Castro; Felipe Díaz; Daniel Cubilla-Bonnetier

doi:10.12688/f1000research.128657.1

Home Browse Validation of the Dominican system for measuring early childhood development

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Validation of the Dominican system for measuring early childhood development

[version 1; peer review: 1 approved with reservations, 1 not approved]

Laura V. Sánchez-Vincitore ¹, María Angélica Alonso Pellerano², María Elena Valdez², [...] Angie Sabrina Jiménez², Carlos B Ruiz-Matuk¹, Arachu Castro³, Felipe Díaz⁴, Daniel Cubilla-Bonnetier¹

Laura V. Sánchez-Vincitore ¹, María Angélica Alonso Pellerano², [...] María Elena Valdez², Angie Sabrina Jiménez², Carlos B Ruiz-Matuk¹, Arachu Castro³, Felipe Díaz⁴, Daniel Cubilla-Bonnetier¹

PUBLISHED 14 Mar 2023

Author details Author details

¹ Universidad Iberoamericana (Unibe), Santo Domingo, Dominican Republic
² Instituto Nacional de Atención Integral a la Primera Infancia (INAIPI), Santo Domingo, Dominican Republic
³ Tulane University, New Orleans, USA
⁴ United Nations International Children's Emergency Fund (UNICEF-DR), Santo Domingo, Dominican Republic

Laura V. Sánchez-Vincitore
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Supervision, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

María Angélica Alonso Pellerano
Roles: Conceptualization, Investigation, Methodology, Project Administration, Supervision, Writing – Review & Editing

María Elena Valdez
Roles: Conceptualization, Investigation, Methodology, Project Administration, Validation, Writing – Review & Editing

Angie Sabrina Jiménez
Roles: Conceptualization, Investigation, Methodology, Project Administration, Validation, Writing – Review & Editing

Carlos B Ruiz-Matuk
Roles: Conceptualization, Data Curation, Formal Analysis, Methodology, Writing – Review & Editing

Arachu Castro
Roles: Conceptualization, Investigation, Methodology, Supervision, Writing – Review & Editing

Felipe Díaz
Roles: Conceptualization, Funding Acquisition, Supervision, Writing – Review & Editing

Daniel Cubilla-Bonnetier
Roles: Data Curation, Formal Analysis, Investigation, Methodology, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Health Services gateway.

This article is included in the Developmental Psychology and Cognition gateway.

Abstract

Background: The purpose of the study was to determine the psychometric properties of the Dominican System for Measuring Early Childhood Development (SIMEDID, for its Spanish acronym), to adjust the sequence of item presentation, and to provide age-standardized norms for each item, to enable policy and program managers to make decisions based on specific and structured data.
Methods: After approval from an ethics committee, a total of 948 children from 0 to 60 months participated in this study. Participants were evaluated on four early childhood development domains (gross motor, fine motor, language development, and socio-emotional development). The data were collected from November 2021 to February 2022, either at early childhood care centers or at home, using mobile devices that guided the evaluators through the screener. Data were later synced to a global database. Psychometric properties were calculated using Cronbach’s alpha and split-half parallel reliability. For reorganizing item presentation and to obtain age-standardized norms, we conducted a logistic regression analysis for each item on dependent variable item success, and independent variable age.
Results: The instrument showed excellent reliability and additional evidence of validity. The item presentation order was rearranged according to the probability of item success progression. In addition, the study characterized the expected evolution of item success probability across participants' age.
Conclusions: SIMEDID is a valid and reliable instrument for depicting childhood development in national evaluations. Its integration with electronic platforms for national monitoring represents a cost-effective, time-efficient screening tool adapted to the Dominican sociocultural context. This represents a promising tool to strengthen strategies that support early childhood development.

Keywords

Early childhood development, Dominican Republic, Monitoring, Screening

Corresponding author: Laura V. Sánchez-Vincitore

Competing interests: No competing interests were disclosed.

Grant information: This study was supported by the result of an interinstitutional collaboration between the United Nations International Children's Emergency Fund (UNICEF), the Instituto Nacional de Atención Integral a la Primera Infancia (INAIPI), and the Universidad Iberoamericana (UNIBE).

Copyright: © 2023 Sánchez-Vincitore LV et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Sánchez-Vincitore LV, Alonso Pellerano MA, Valdez ME et al. Validation of the Dominican system for measuring early childhood development [version 1; peer review: 1 approved with reservations, 1 not approved]. F1000Research 2023, 12:279 (https://doi.org/10.12688/f1000research.128657.1) First published: 14 Mar 2023, 12:279 (https://doi.org/10.12688/f1000research.128657.1) Latest published: 29 Jan 2024, 12:279 (https://doi.org/10.12688/f1000research.128657.2)

Introduction

The Dominican Republic confirmed its commitment to achieving the United Nation’s sustainable development goal 4.2 for 2030 to guarantee that children are ready for primary education by offering quality early childhood services, including care and early education (United Nations, 2015). One of this goal’s indicators is “[the] proportion of children aged 24-59 months who are developmentally on track in health, learning, and psychosocial well-being, by sex” (UNESCO UIS, n.d., para. 1). Although this indicator is conceptually straightforward, and numerous efforts have been conducted to establish a global methodology for measuring it, some challenges associated with determining cut-off points for early childhood development remain (Daelmans et al., 2017; Richter et al., 2017).

First, high-income countries' screening tools might translate poorly to low- and middle-income countries. This poses the risk of either underestimating or overestimating childhood development, which in turn precludes making accurate, evidence-based decisions regarding early childhood interventions and resource allocation (Gladstone et al., 2008; Sabanathan et al., 2015). The second challenge is the need for more funding for monitoring systems in low- and middle-income countries, which translates into needing more sufficiently qualified personnel to conduct periodic childhood development screenings (Lokuketagoda et al., 2016). And third, there is a critical need to obtain a large pool of data from developing children to identify those at risk for developmental delay (Lokuketagoda et al., 2016), which is particularly challenging in low- and middle-income countries (Richter et al., 2017).

In Latin America and the Caribbean, childhood development measurement has attracted attention, evidenced by the creation of the Regional Network for Measuring Childhood Development (REMDI) (Interamerican Dialogue, 2020). This international network of specialists is dedicated to promoting national measurements of childhood development to obtain data for decision-making and comparison between and within countries. Since the year 2000, the Dominican Republic has collected data on childhood development by participating in a series of Multiple Indicator Cluster Surveys (MICS), a household survey methodology designed by UNICEF that analyzes the situation of women and children across the world. The instrument collects data on children’s health, education, protection, and environment (such as sanitation), among other variables. The Dominican Republic participated in MICS2 (Molina Achécar and Polanco, 2001), MICS5 (ONE and UNICEF, 2016), and MICS6 (ONE and UNICEF, 2021) survey rounds with an evolving early childhood development measurement (Loizillon et al., 2017).

The latest data from 2019 reported that 87.1% of Dominican children meet the minimum development indicators. Data generated by MICS have been useful for guiding the advocacy and system-strengthening plans of early childhood development and children’s rights institutions, including sustainable investments in the multi-year governmental planning for 2020-2024. In addition, the MICS data have been used to create predictive models that quantify the impact of multiple sociodemographic and psychosocial factors in childhood development (Sánchez-Vincitore and Castro, 2022).

Household surveys provide useful information on general trends in childhood development, but they are not comprehensive enough to assess development in its various dimensions or sensitive enough to generate alerts to detect developmental delays. These limitations highlight the need for specific child development screening tools that, although quick and cost-effective in the application, have adequate psychometric properties.

Many private, informal, and some public initiatives have been conducted in the Dominican Republic to provide early childhood services. However, until 2019 there were no standardized instruments to measure the impact of such efforts. In 2019, Sánchez-Vincitore et al. (2019) initiated the validation of the Dominican adaptation of the Malawi Developmental Assessment Tool (MDAT) (Gladstone et al., 2008, 2010). It is a childhood development screener in which an evaluator observes the behavior of a child in four different domains: gross motor, fine motor, language, and socio-emotional development. One of the advantages of this screener is that it optimizes test application time by only presenting items that correspond to the child’s expected evolutionary stage according to the child’s age. Therefore, providing a more precise item order is crucial to prevent bias in obtaining total scores. Although the instrument presented good psychometric properties, some limitations had to be considered before upscaling it as a national surveillance tool. First, as a preliminary pilot implementation within the academic context, research assistants with vast experience and training in data collection administered the instrument, which is unlikely in naturalistic environments. Second, the study had a small sample size (N = 42), meaning that there was no national representativeness. Therefore, age-standardized norms were not obtained for each item, threatening item presentation order. Finally, the study evaluated children up to 24 months, limiting the age range for which data were available.

To overcome these limitations, the National Institute for Early Childhood Comprehensive Care (INAIPI, for its acronym in Spanish) –the state institution responsible for providing quality comprehensive care services to children from 0 to five years and their families–, the Universidad Iberoamericana (UNIBE), and the United Nations Children’s Fund (UNICEF) developed the Dominican System for Measuring Early Childhood Development (SIMEDID, for its acronym in Spanish). SIMEDID was created by re-analyzing the content and structure of the Dominican version of the MDAT (Sánchez-Vincitore et al., 2019) and integrating other items from international and national instruments (Alonso et al., 2022).

SIMEDID is an electronic platform that hosts an early childhood development screener. This platform allows data collection through mobile devices and connects to INAIPI’s servers as part of its monitoring and evaluation system. The mobile application extracts sociodemographic information from the server, configuring individual evaluations according to each child’s age. As a result, INAIPI personnel already in the field can administer the early childhood development screener time-efficiently with little training. Once the assessment is over and the device connects to the internet, the data sync to the server—reducing the risk of losing the data.

Given the current stage of development of SIMEDID, the aims of this study were: (1) To determine the psychometric properties of SIMEDID; (2) To adjust the sequence of item presentation according to developmental milestones obtained from data from a large sample; (3) To provide age-standardized norms for each item.

Methods

Ethical statement

The Universidad Iberoamericana’s ethics committee approved this study (CEI2021-3). Written informed consent was obtained from participant’s parents or guardians before participation in the study.

Study design

This is a cross-sectional, non-experimental, and descriptive study that evaluated children who receive services at INAIPI.

Setting

Data collection occurred in Santo Domingo, Dominican Republic, from November 1^st, 2021, to February 17^th, 2022. INAIPI participants who attended Comprehensive Care Centers for Early Childhood (CAIPIs, for its Spanish acronym) were assessed at their centers. In contrast, those who participated in Comprehensive Care Centers for Children and the Family (CAFI, for its Spanish acronym) received community and family-based services at their home, which is where children were assessed. The instrument was applied during regular service hours. The evaluation personnel consisted of 20 educational agents (who work at CAIPIs) and 20 community agents (who work at CAFIs).

Participants

An intentional sample of 948 children who live in Santo Domingo was selected from the INAIPI’s System of Information and Management for Early Childhood (SIGEPI, for its Spanish acronym) according to their age and type of service received (CAIPI or CAFI, which were kept proportional to the actual service: 36% and 64%, respectively). The inclusion criterion was to be beneficiaries of INAIPI. Participants were 428 girls (45.1%) and 520 boys (54.9%).

Instruments

Sociodemographic variables: This set of questions addressed general demographic variables: sex assigned at birth (male and female), age (in days at the moment of evaluation), and type of service (CAIPI vs CAFI). These variables, obtained directly from SIGEPI, determine the starting item of SIMEDID 's subscales.

SIMEDID: This electronic instrument assesses childhood development in four development areas: gross motor, fine motor, language development, and social development. Each subtest has 33 items (except for language development, which has 34 items) in ascending difficulty levels. The presentation of the first item depends on the child’s age. Once the first item is completed on each dimension, the instrument is presented backward until the participant passes three items. Then the presentation continues forward until the participant misses three items. This allows evaluators to know the actual state of child development, in addition to detecting alerts, either delays in childhood development (backward presentation) or above the average development (forward presentation). Since the purpose of the study was to determine the age range in which each item is accomplished, we used the Malawi MDAT norms for this first trial. Subsequent data collection will use the sequence based on the findings from this study.

The evaluation is conducted using a mobile device with the SIMEDID app that connects to the INAIPI server and instructs the evaluator to assess a specific child. The app calculates the first item in each development area and presents items backward and forward. Passed items were scored 1, while missed items scored 0. Items not shown (for not corresponding to the participant's age range) were automatically completed: items before the three first achieved items were scored 1 (since it is assumed that the child has already passed these), and items after three consecutive misses scored 0 since the child is not ready to perform these.

We obtained an expert panel's consensus before collecting data to guarantee the instrument's construct definition (Sireci and Sukin 2013). The expert panel consisted of a group of professionals who represent Dominican institutions that provide early childhood services, including the Early Childhood Education Department and the Special Education Department from the Ministry of Education; the Ministry of Health; the National Health Services (SNS for its Spanish acronym); the National Council for Childhood and Adolescence (CONANI for its Spanish acronym); and the National Council for Disabilities (CONADIS for its Spanish acronym). The experts had the opportunity to revise each item and their definitions. The experts suggested to modify the definition of the item “[They] walk on tiptoes”. They included a statement that differentiates walking on tiptoes as a possible disability indicator vs. the intentional action of walking on tiptoes. The experts unanimously approved all changes.

The items were presented in Spanish and were translated to English by the research team for this publication.

The variables included in this study are listed in Table 1.

Table 1. Study variables.

Variable	Description
Age at evaluation	Numerical variable. Age is calculated in days for the analyses but displayed in months and years in the figures.
Age group	Ordinal variable. Age group was calculated by merging age in days intervals into the following month interval: 0-2, 2-4, 4-6, 6-9, 9-12, 12-15, 15-18, 18-24, 24-30, 30-36, 36-42, 42-48, 48-54, 54-60.
Service	Categorical variable. CAFI vs. CAIPI
Item success (for each item)	Categorical variable. 0 = no success, 1 = success.
Gross motor development score	Numerical variable. Scores range from 0-33, calculated by the sum of item success from the gross motor development sub-scale.
Fine motor development score	Numerical variable. Scores range from 0-33, calculated by the sum of item success from the fine motor development sub-scale.
Language development score	Numerical variable. Scores range from 0-34, calculated by the sum of item success from the language development sub-scale.
Socioemotional development score	Numerical variable. Scores range from 0-33, calculated by the sum of item success from the socioemotional development sub-scale.

Procedure

A total of 40 evaluators received a six-hour training session and conducted two practice evaluations. During the practice evaluations, the research team coached and supervised the evaluators. Recruitment was conducted by an institutional message indicating that either a CAIPI or CAFI was selected to participate. Children from CAIPI attending services during the data collection day were evaluated after a parent signed the consent form when dropping off their children at the centers. For CAFI participants, evaluation was conducted at home, and the assigned in-field INAIPI personnel contacted their families. Parents signed the informed consent before the interview took place at home. Each evaluation had a duration of 25-30 minutes.

Statistical methods

To determine the instrument's psychometric properties, which corresponds to the first aim, we calculated Cronbach’s alpha and split half-parallel reliability. Then, for additional evidence of content validity, we conducted descriptive analyses (means and standard deviations) of each sub-scale score for each age group to confirm alignment of the instrument with development by age.

To determine the most appropriate item presentation order according to these data (second aim) and age-standardized norms (third aim), we conducted a logistic regression analysis on each item with item success (0 and 1) as the dependent variable and age in days as the independent variable. Following the methodology used by Gladstone et al. (2008, 2010), after ensuring a good model fit, the alpha and beta coefficients were used to calculate the cut-off age associated with a.9 probability of success following the formula in Equation 1:

(1)

P (Y) = \frac{1}{1 + e^{- (\propto + βx)}}

The formula was also used to calculate the .75, .50, and .25 probabilities of success for each item. We used predictive probabilities from the regressions to calculate ages corresponding to 25%, 50%, 75%, and 90% of children passing each item, as proposed by Gladstone et al. (2008, 2010). For the statistical analyses, we used the IBM SPSS Statistics 25 program (https://www.ibm.com/products/spss-statistics). An open-source alternative to SPSS that can conduct the same processes is JASP (https://jasp-stats.org/).

Results

Psychometric properties of SIMEDID (first aim)

To confirm the content validity of SIMEDID, we obtained descriptive statistics on each age group for the evaluated developmental areas. Table 2 shows that mean growth is progressive through age groups.

Table 2. Descriptive statistics for total scores by development area and age group.

Age group	n		Gross motor	Fine motor	Language	Social-emotional
0-2 month	2	Mean	*	*	*	*
0-2 month	2	SD	*	*	*	*
2-4 month	30	Mean	4.97	5.13	5.53	7.23
2-4 month	30	SD	2.62	3.75	1.36	3.73
4-6 month	43	Mean	6.49	8.98	6.84	9.74
4-6 month	43	SD	2.10	2.20	1.29	3.07
6-9 month	65	Mean	9.31	11.66	8.31	13.88
6-9 month	65	SD	2.66	2.94	2.08	4.24
9-12 month	52	Mean	12.35	13.00	9.52	16.23
9-12 month	52	SD	1.61	1.61	1.41	2.58
12-15 month	48	Mean	15.94	14.54	11.23	18.90
12-15 month	48	SD	4.25	2.82	2.89	3.45
15-18 month	37	Mean	19.16	18.03	14.19	21.86
15-18 month	37	SD	4.36	5.16	5.33	3.96
18-24 month	82	Mean	22.44	21.27	17.23	22.95
18-24 month	82	SD	3.90	4.49	6.70	4.30
24-30 month	85	Mean	25.74	24.42	21.61	25.96
24-30 month	85	SD	4.50	4.56	6.57	4.36
30-36 month	89	Mean	27.61	25.79	25.90	27.78
30-36 month	89	SD	3.98	3.75	5.56	3.99
36-42 month	111	Mean	28.86	28.38	28.86	30.50
36-42 month	111	SD	3.75	3.11	4.98	3.26
42-48 month	101	Mean	28.98	28.76	29.85	31.33
42-48 month	101	SD	4.29	3.44	5.34	2.50
48-54 month	117	Mean	31.67	30.89	32.05	32.05
48-54 month	117	SD	3.27	3.06	3.59	2.98
54-60 month	86	Mean	32.20	32.14	33.30	32.22
54-60 month	86	SD	1.83	1.16	1.42	2.12

* Sample size for this group was insufficient for an accurate representation.

Two internal consistency indices were calculated for each developmental area to confirm the instrument reliability, Cronbach's alpha, and split-half Spearman-Brown’s correlation (see Table 3).

Table 3. Reliability of the instrument.

Development area	Cronbach's α	Split-half correlation
Gross motor	0.97	0.78
Fine motor	0.96	0.79
Language development	0.97	0.88
Socioemotional development	0.96	0.73

Adjustment of items order presentation (second aim) and age references per item (third aim)

To complete the second and third aims, we conducted a logistic regression analysis on each item with independent variable age and dependent variable item success. The results show a good fit (p < .05), except for the first item in the gross motor area and three of the first four items in the language development area.

The item presentation order was determined by sorting the age at which each item had a .9 probability of success from the results of logistic regression analyses on each item. To describe the expected evolution of item success probability across age, we also determined the .75, .50, and .25 probability of success. We thus provided the range amplitude for each item's success predicted by age. Figures 1 to 4 contain a visual representation of sorted items and corresponding probabilities of success. Note that results are presented in two scales for the X axis: for the first two years, the X axis corresponds to age in months, and for the following age ranges, in years.

Figure 1. Age reference values for gross motor items.

Notes: MG1 Hold their head when carried; MG2 Lift their chin off the floor; MG3 From the prone position, they can lift their head to 90 degrees; MG4 Support head when lifted by hands; MG5 Flip over; MG6 Raise head, shoulders, and chest from a prone position; MG7 Start creeping; MG8 Start crawling position; MG9 Stand up with support; MG10 Sits up unassisted; MG11 Crawl with displacement alternating knees and hands; MG12 Take steps with help; MG13 Stand up unassisted; MG14 Walk without help; MG15 They crouch and stand up; MG16 Walk well with cross-scroll; MG17 Run, they may fall; MG18 Throw ball; MG19 Kick the ball; MG20 Run showing coordination in their movements; MG21 Run well, stops and start again without falling; MG22 Jump with feet together; MG23 Jump moving with both feet; MG24 Stand on one foot for 3 seconds; MG25 Stand on tiptoe with both feet; MG26 Walk on tiptoe; MG27 Walk in a straight line keeping balance; MG28 Jump on one foot without support; MG29 Can catch a ball with both hands; MG30 Bounce and catch the ball; MG31 Stand on one foot for 5 seconds; MG32 Jump moving with one foot; MG33 Jump alternating feet.

Figure 2. Age reference values for fine motor items.

Notes: MF1 Palmar grasp reflex; MF2 Stare the midline; MF3 Visually focuses on an object and tracks it horizontally; MF4 Keep hands open when awake; MF5 Hold an object in hand; MF6 Show interest in putting an object in their mouth; MF7 Visually focus on an object and follow it from top to bottom; MF8 Grasp large objects voluntarily; MF9 Hold an object in each hand; MF10 Pass an object from one hand to another; MF11 Pick up small objects as if their fingers were a rake; MF12 Find the object under a blanket; MF13 Put and take out objects from the container; MF14 Grasp with thumb and forefinger (tweezers); MF15 Pick up a spoon and brings it to their mouth; MF16 Scribble; MF17 Push a car; MF18 Turn pages of a book; MF19 Make a tower of two cubes; MF20 Put nails on a board; MF21 Make a tower of six cubes; MF22 Make a ball of paper; MF23 Tear paper with both hands; MF24 Make shapes with putty; MF25 Rotate hand to unscrew; MF26 String; MF27 Copy a horizontal and vertical line; MF28 Copy a circle; MF29 Copy a cross; MF30 Know how to button and unbutton; MF31 Color without leaving the outline of the drawing; MF32 Draw a human figure; MF33 Cut paper with scissors.

Figure 3. Age reference values for language development items.

Notes: DL1 Calm down when speaking to them; DL2 Startle or jump in response to sounds; DL3 Cry to express needs; DL4 They laugh; DL5 Make sounds with the throat; DL6 Turn their head when they search for a sound; DL7 React when called by their name; DL8 Pronounce syllables like Ma, Pa, Ba, Ta; DL9 Point with their finger when they want something; DL10 Repeat the same syllable twice “Dada, Mama, Mimi, Tata, Papa, Yaya, Baba”; DL11 Understand the meaning of the word no; DL12 Follow one-step commands; DL13 Follow two-step instructions; DL14 Answer with yes or no; DL15 Pronounce their first words with communicative intention; DL16 Recognize at least 6 objects; DL17 Point to 5 parts of their body; DL18 Use a two-word phrase; DL19 Say 6 words; DL20 Say their name; DL21 Know the use of three or more objects; DL22 Can identify 10 objects by name; DL23 Pronounce sentences of three words; DL24 Use more than 15 words; DL25 Use long sentences; DL26 Know the qualities or characteristics of an object; DL27 Pronounce the sounds of words correctly; DL28 Describe the drawing; DL29 Name at least three things in a category; DL30 Recognize opposites; DL31 Can count up to 5 or more objects; DL32 Answer two comprehension questions; DL33 Compare objects; DL34 Tell a story from a sequence of images.

Figure 4. Age reference values for socioemotional development items.

Notes: DS1 Calm down with family members or caregivers; DS2 Smile spontaneously; DS3 Smile in response to a person; DS4 Recognize the voice of the main caregiver; DS5 Make eye contact; DS6 Touch the examiner's hands; DS7 They are aware of their hands (body); DS8 Try to hold a cup when being fed; DS9 Respond to a conversation; DS10 Raise their arms or indicate that they want to be carried; DS11 Laugh out loud; DS12 Explore their face when they are in front of the mirror; DS13 Show interest or intention to feed themselves; DS14 Look for continuing the game; DS15 Explore the environment; DS16 Participate in games; DS17 Wave or verbally greet; DS18 Express their satisfaction when they achieve something; DS19 Take a glass without spilling; DS20 Imitate adult actions; DS21 Recognize their belongings; DS22 Express interest in playing with other children; DS23 Symbolic game; DS24 Refer to themselves as “I”; DS25 Say the names of the people with whom they live; DS26 They urinate or defecate independently without dirtying their clothes; DS27 Indicate in some way that they need to urinate or defecate; DS28 Identify basic emotions in images; DS29 Come up with games; DS30 Share their belongings; DS31 Recognize basic emotions in themselves and express them verbally; DS32 Recognize and express basic emotions in others; DS33 Participate in games respecting rules and turns.

Discussion

This paper collects evidence for the validity of SIMEDID, an electronic early childhood development screening tool adapted to the Dominican context conducted by INAIPI’s personnel through an electronic application. Regarding the first objective, we found that the instrument has adequate psychometric properties: the instrument’s subscales showed high internal consistency scores, evidencing excellent reliability. Furthermore, total scores for each sub-scale increased progressively across age, which evidenced alignment with standards already provided by an expert panel (Alonso et al., 2022) and criterion validity with a previous version of this instrument on a small sample size (Sánchez-Vincitore et al., 2019).

Regarding the second aim, we found that age predicted item success in most items, which supports similar results in children from Malawi Gladstone et al. (2008, 2010). However, age did not predict four initial items from the gross motor and language development subscales. We attribute these null findings to the fact that our study did not include an acceptable sample size for the age group for which these items were relevant. This is because INAIPI’s services to very young children were scarce at the time of this study, which will be considered for future studies. To comply with the third aim, age-standardized norms for each item were established, obtaining correspondences between ages and different probabilities of success for each item, which will allow comparing the achievement of participants with what is expected at their age.

Adapting this instrument to the Dominican context guarantees that cultural aspects of childrearing do not overshadow developmental scores (Suchdev et al., 2017) and that development is not under or overestimated. Having the instrument in an electronic platform solves two main challenges. First, personnel training is kept to the minimum since the platform guides the evaluator throughout the evaluation, presenting the items that only pertain to the child according to their age, with suggested videos and additional testing resources. Second, having SIMEDID connected to INAIPI servers and incorporating the data on national services provided by INAIPI creates a continuous stream that otherwise would be costly and logistically convolute data. This data stream will allow the development of new research agendas that include correlational modeling, intervention studies, and longitudinal studies to understand better the factors associated with childhood development in the Dominican Republic in a timely and cost-efficient way.

The study findings demonstrate that SIMEDID passes three elements of a checklist of critical methodological elements to consider when appraising a childhood development assessment tool: (1) the instrument measures domains affected by the risk factor or intervention; (2) reliability and validity of the instrument in the population of interest; (3) sensitivity of the instrument to identify changes; (4) logistics and methodology is suitable for evaluating the outcome; and (5) consideration of control group (Sabanathan et al., 2015). SIMEDID passes the first two elements from this checklist, as it measures specific domains of early childhood development previously identified as risk factors in the Dominican Republic, such as the sociodemographic and psychosocial factors that predict childhood development (Sánchez-Vincitore and Castro, 2022) and low levels of oral language comprehension in school-aged children that should be addressed during early childhood development before children enter primary school (Sánchez-Vincitore et al., 2020, 2022) among other risk factors. In addition, it passes the fourth element, given that the logistic and methodology was specifically designed to assess the outcome in the Dominican context. Future studies will address the third and fifth items when SIMEDID is used as a monitoring tool on a population basis.

This study has some limitations that should be considered before its interpretation. Children from the sample for which these age standards were obtained receive services at INAIPI. Socio-economic vulnerability is one of the main inclusion criteria for receiving such services. This means that the results of this study can only be generalizable for this population. Future studies should consider the whole socio-economic position spectrum to obtain national norms. The data generated in the Dominican context using SIMEDID has limited comparability to data from other countries.

Another important limitation is that the experience of creating this platform in the Dominican Republic was cost-effective due to the already existing infrastructure within INAIPI, which should be considered when transferring it to other countries. The institution is the national administrator of early childhood services, which gives them access to the population of interest and trained personnel already working with children. In addition, INAIPI has the Division of Early Childhood Development Measurement, with dedicated personnel to designing, creating, supervising, and training the personnel in childhood development measurement. And finally, INAIPI has a dedicated Information and Communications Technology Department, which developed the online platform and made it to synchronize with SIGEPI, the national database for managing data from early childhood services. Further studies should conduct a cost-per-user analysis to evaluate its efficiency.

Even with these limitations, these results will allow the pertinent institutions of the Dominican Republic to implement and report more accurate early childhood development indicators. They will also contribute to creating a robust monitoring system with a high-quality data collection process that allows evidence-based and timely decision-making. Furthermore, such a system will contribute to generating longitudinal data that can establish the association between childhood development and sociodemographic and psychosocial variables and determine the impact of initiatives and interventions (Richter et al., 2017), which is not sufficiently evaluated in most countries (Daelmans et al., 2017).

Although SIMEDID was created to be integrated into the services provided by INAIPI, and the instrument so far has only been administered to INAIPI beneficiaries, efforts to make a paper version of SIMEDID are on the way under the name of TADID (Tamizaje de Desarrollo Infantil Dominicano). This will allow other institutions, clinicians, schools, and pediatricians to use this validated tool at no cost.

Conclusion

In conclusion, this study provides evidence for the validity of SIMEDID, an electronic early childhood development screening tool adapted to the Dominican context, with adequate psychometric properties and age-standardized norms for each item. Adapting this instrument to the Dominican context ensures that cultural aspects of childrearing do not overshadow developmental scores. While SIMEDID is a screening tool and not intended for diagnosis, it offers valuable insights for caregivers and stakeholders, both at a group level for program design and decision-making, as well as at the individual level to monitor each child's progress.

Data availability

Underlying data

Open Science Framework: Database for Validation of the Dominican System for Measuring Early Childhood Development. https://doi.org/10.17605/OSF.IO/KW3B8 (Sánchez-Vincitore et al., 2023b).

The project contains the following underlying data:

• Codebook SIMEDID.docx (names and values of each variable).
• Database – Validation study – SIMEDID.csv (database).

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Extended data

Open Science Framework: Questionnaire and item presentation for SIMEDID validation study. https://doi.org/10.17605/OSF.IO/SWN8C (Sánchez-Vincitore et al., 2023a).

This project contains the following extended data:

• SIMEDID – Presentation order.xlsx. (Order of item presentation before and after data collection. Spanish and English translations)

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Acknowledgements

The authors sincerely thank Besaida Manola Santana and Penélope Melo Ballesteros for their exemplary leadership throughout the project. We also extend our heartfelt appreciation to the dedicated team of data collectors whose hard work and commitment to quality ensured the success of this study.

References

Alonso MA, Valdez ME, Jiménez A, et al.: Creación del Sistema de Medición de Desarrollo Infantil Dominicano (SIMEDID). Ciencia y Educación. 2022; 6(3): 71–77. Publisher Full Text
Daelmans B, Darmstadt GL, Lombardi J, et al.: Early childhood development: The foundation of sustainable development. Lancet. 2017; 389(10064): 9–11. Publisher Full Text
Inter-American Dialogue: Medición del desarrollo infantil en América: Desafíos para la medición y respuestas de política. Washington DC: Inter-American Dialogue; 2020.
Gladstone M, Lancaster GA, Jones AP, et al.: Can Western developmental screening tools be modified for use in a rural Malawian setting? Arch. Dis. Child. 2008; 93(1): 23–29. PubMed Abstract | Publisher Full Text
Gladstone M, Lancaster GA, Umar E, et al.: The Malawi developmental assessment tool (MDAT): The creation, validation, and reliability of a tool to assess child development in rural African settings. PLoS Med. 2010; 7(5): e1000273. PubMed Abstract | Publisher Full Text | Free Full Text
Loizillon A, Petrowski N, Britto P, et al.: Development of the Early Childhood Development Index in MICS surveys. MICS Methodological Papers, No. 6.2017.
Lokuketagoda BUWP, Thalagala N, Fonseka P, et al.: Early development standards for children aged 2 to 12 months in a low-income setting. SAGE Open. 2016; 6(4): 215824401667312. Publisher Full Text
Molina Achécar M, Polanco JJ: Encuesta por Conglomerados de Indicadores Múltiples (MICS-2000).2001.
ONE, & UNICEF: Encuesta nacional de hogares de propósitos múltiples - Encuesta de Indicadores Múltiples por Conglomerados 2014, Informe final.2016.
ONE, & UNICEF: ENHOGAR-MICS Encuesta de Indicadores Múltiples por Conglomerados 2019, Informe de resultados de la encuesta.2021.
Richter LM, Daelmans B, Lombardi J, et al.: Investing in the foundation of sustainable development: Pathways to scale up for early childhood development. Lancet. Elsevier B.V; 2017; 389(10064): 103–118. PubMed Abstract | Publisher Full Text | Free Full Text
Sabanathan S, Wills B, Gladstone M: Child development assessment tools in low-income and middle-income countries: How can we use them more appropriately? Arch. Dis. Child. 2015; 100(5): 482–488. PubMed Abstract | Publisher Full Text | Free Full Text
Sánchez-Vincitore LV, Castro A: The role of sociodemographic and psychosocial variables in early childhood development: A secondary data analysis of the 2014 and 2019 Multiple Indicator Cluster Surveys in the Dominican Republic. PLOS Global Public Health. 2022; 2(7): e0000465. Publisher Full Text
Sánchez-Vincitore LV, Mencía-Ripley A, Veras-Díaz C, et al.: Efectos de una intervención de alfabetización en las habilidades lectoras de estudiantes de primaria: Proyecto USAID Leer. Revista Caribeña de Investigación Educativa. 2020; 4(2): 78–95. Publisher Full Text
Sánchez-Vincitore LV, Schaettle P, Castro A: Validation of the Malawi Developmental Assessment Tool for children in the Dominican Republic: Preliminary results. PLoS One. 2019; 14(8): e0221162. PubMed Abstract | Publisher Full Text | Free Full Text
Sánchez-Vincitore LV, Veras C, Mencía-Ripley A, et al.: Reading comprehension precursors: Evidence of the simple view of reading in a transparent orthography. Front. Educ. 2022; 7: 914414. Publisher Full Text
Sánchez-Vincitore LV, Alonso Pellerano MA, Valdez ME, et al.: Questionnaire and item presentation for SIMEDID validation study. Dataset. 2023a, February 21. Publisher Full Text
Sánchez-Vincitore LV, Alonso Pellerano MA, Valdez ME, et al.: Database for Validation of the Dominican System for Measuring Early Childhood Development. Dataset. 2023b, February 7. Publisher Full Text
Sireci SG, Sukin T: Test validity.Geisinger KF, Bracken BA, Carlson JF, et al., editors. APA handbook of testing and assessment in psychology, Vol. 1. Test theory and testing and assessment in industrial and organizational psychology. American Psychological Association; 2013; pp. 61–84. Publisher Full Text
Suchdev PS, Boivin MJ, Forsyth BW, et al.: Assessment of neurodevelopment, nutrition, and inflammation from fetal life to adolescence in low-resource settings. Pediatrics. 2017; 139: S23–S37. PubMed Abstract | Publisher Full Text
UNESCO Institute of Statistics (UIS): Proportion of children aged 24-59 months who are developmentally on track in health, learning and psychosocial well-being, by sex.n.d.Reference Source
United Nations: Transforming our world: The 2030 agenda for sustainable development (A/RES/70/1).2015. Reference Source

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 14 Mar 2023

Author details Author details

María Angélica Alonso Pellerano
Roles: Conceptualization, Investigation, Methodology, Project Administration, Supervision, Writing – Review & Editing

María Elena Valdez
Roles: Conceptualization, Investigation, Methodology, Project Administration, Validation, Writing – Review & Editing

Angie Sabrina Jiménez
Roles: Conceptualization, Investigation, Methodology, Project Administration, Validation, Writing – Review & Editing

Carlos B Ruiz-Matuk
Roles: Conceptualization, Data Curation, Formal Analysis, Methodology, Writing – Review & Editing

Arachu Castro
Roles: Conceptualization, Investigation, Methodology, Supervision, Writing – Review & Editing

Felipe Díaz
Roles: Conceptualization, Funding Acquisition, Supervision, Writing – Review & Editing

Daniel Cubilla-Bonnetier
Roles: Data Curation, Formal Analysis, Investigation, Methodology, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

This study was supported by the result of an interinstitutional collaboration between the United Nations International Children's Emergency Fund (UNICEF), the Instituto Nacional de Atención Integral a la Primera Infancia (INAIPI), and the Universidad Iberoamericana (UNIBE).

Article Versions (2)

version 2

Revised

Published: 29 Jan 2024, 12:279

https://doi.org/10.12688/f1000research.128657.2

version 1

Published: 14 Mar 2023, 12:279

https://doi.org/10.12688/f1000research.128657.1

© 2023 Sánchez-Vincitore LV et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Sánchez-Vincitore LV, Alonso Pellerano MA, Valdez ME et al. Validation of the Dominican system for measuring early childhood development [version 1; peer review: 1 approved with reservations, 1 not approved]. F1000Research 2023, 12:279 (https://doi.org/10.12688/f1000research.128657.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 14 Mar 2023

Views

Reviewer Report 31 Aug 2023

Mphelekedzeni Caroline Mulaudzi, Paediatrics and Child Health, Charlotte Maxeke Johannesburg Academic Hospital, Johannesburg, Gauteng, South Africa; University of Witwater'srand, Johannesburg, South Africa

Approved with Reservations

https://doi.org/10.5256/f1000research.141269.r194708

I commend the researchers for doing research in the area of instruments for early child development in low/middle-income countries.

Literature review

The researchers should have shared more on their initial work of adapting MDAT to the Dominican instrument and how it differs.

Methodology and statistical analysis

There is no description of the sampling technique used and no information on how they determined the population was normal

As far as whether the correct statistical analysis was done to meet the aims of the study.

There is clear information and a report on reliability but not validity.

Results

Figures should be labeled appropriately with an explanation of the range of colors' meanings.

The Y axis of the item should not only mention the domain but also what analysis

Discussion

Interpretation of the result is poor. The researchers gave results of the Chronbach alpha and split-half correlation but did not elaborate on the relevance

They did not comment on how the instrument should be used in the future, especially with items that had a standard deviation that was higher than 3 and even some items that had a standard deviation greater or equal to 6.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Partly
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Child health and Early Childhood Development

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 22 Mar 2024

Laura V. Sánchez-Vincitore, Universidad Iberoamericana (Unibe), Santo Domingo, Dominican Republic

22 Mar 2024

Author Response

Dear Dr. Mulaudzi,

We thank you for your comments and questions, which have helped us improve the quality of our manuscript. We address your questions below and in the ... Continue reading Dear Dr. Mulaudzi,

We thank you for your comments and questions, which have helped us improve the quality of our manuscript. We address your questions below and in the manuscript:

1. The researchers should have shared more on their initial work of adapting MDAT to the Dominican instrument and how it differs.
How was this addressed in the manuscript?
We have described the process of adapting the instrument in more detail, summarizing Alonso et al. 2022.

2. There is no description of the sampling technique used and no information on how they determined the population was normal
How was this addressed in the manuscript?
We used an intentional sampling with an N=948. We have clarified that the sample came from INAIPI beneficiaries who lived in the capital city of Santo Domingo and provided additional details.

3. There is clear information and a report on reliability but not validity.
How was this addressed in the manuscript?
We have included additional details on the creation of SIMEDID that includes more validity information, such as collaboration with experts, content analysis, and ongoing data collection activities. As SIMEDID transitions into the national surveillance system, continued data collection will contribute further evidence of its validity.

4. Figures should be labeled appropriately with an explanation of the range of colors' meanings.
How was this addressed in the manuscript?
We have added a note to each figure explaining the color range.

5. The Y axis of the item should not only mention the domain but also what analysis
How was this addressed in the manuscript?
The Y represents the distribution of the probability of success, as explained by the color ranges. We have added this information to each figure.

6. Interpretation of the result is poor. The researchers gave results of the Chronbach alpha and split-half correlation but did not elaborate on the relevance
How was this addressed in the manuscript?
We have expanded the conclusion with an additional interpretation of the results.

7. They did not comment on how the instrument should be used in the future, especially with items that had a standard deviation that was higher than 3 and even some items that had a standard deviation greater or equal to 6.
How was this addressed in the manuscript?
In the future, SIMEDID will identify children at risk for developmental delays. The current study aimed to enhance item presentation order. The observed large standard deviations may result from presenting items in an order that doesn't fully adapt to the Dominican context. The upcoming standardization study, which includes a more fitting presentation order of items obtained in the present validation study, seeks to establish national norms by calculating cut-off points based on standard deviations once the presentation order is appropriately configured. We have explained these plans in the discussion.
Dear Dr. Mulaudzi,

We thank you for your comments and questions, which have helped us improve the quality of our manuscript. We address your questions below and in the manuscript:

1. The researchers should have shared more on their initial work of adapting MDAT to the Dominican instrument and how it differs.
How was this addressed in the manuscript?
We have described the process of adapting the instrument in more detail, summarizing Alonso et al. 2022.

2. There is no description of the sampling technique used and no information on how they determined the population was normal
How was this addressed in the manuscript?
We used an intentional sampling with an N=948. We have clarified that the sample came from INAIPI beneficiaries who lived in the capital city of Santo Domingo and provided additional details.

3. There is clear information and a report on reliability but not validity.
How was this addressed in the manuscript?
We have included additional details on the creation of SIMEDID that includes more validity information, such as collaboration with experts, content analysis, and ongoing data collection activities. As SIMEDID transitions into the national surveillance system, continued data collection will contribute further evidence of its validity.

4. Figures should be labeled appropriately with an explanation of the range of colors' meanings.
How was this addressed in the manuscript?
We have added a note to each figure explaining the color range.

5. The Y axis of the item should not only mention the domain but also what analysis
How was this addressed in the manuscript?
The Y represents the distribution of the probability of success, as explained by the color ranges. We have added this information to each figure.

6. Interpretation of the result is poor. The researchers gave results of the Chronbach alpha and split-half correlation but did not elaborate on the relevance
How was this addressed in the manuscript?
We have expanded the conclusion with an additional interpretation of the results.

7. They did not comment on how the instrument should be used in the future, especially with items that had a standard deviation that was higher than 3 and even some items that had a standard deviation greater or equal to 6.
How was this addressed in the manuscript?
In the future, SIMEDID will identify children at risk for developmental delays. The current study aimed to enhance item presentation order. The observed large standard deviations may result from presenting items in an order that doesn't fully adapt to the Dominican context. The upcoming standardization study, which includes a more fitting presentation order of items obtained in the present validation study, seeks to establish national norms by calculating cut-off points based on standard deviations once the presentation order is appropriately configured. We have explained these plans in the discussion.
Competing Interests: N/A Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 22 Mar 2024

Laura V. Sánchez-Vincitore, Universidad Iberoamericana (Unibe), Santo Domingo, Dominican Republic

22 Mar 2024

Author Response

Dear Dr. Mulaudzi,

We thank you for your comments and questions, which have helped us improve the quality of our manuscript. We address your questions below and in the ... Continue reading Dear Dr. Mulaudzi,

We thank you for your comments and questions, which have helped us improve the quality of our manuscript. We address your questions below and in the manuscript:

1. The researchers should have shared more on their initial work of adapting MDAT to the Dominican instrument and how it differs.
How was this addressed in the manuscript?
We have described the process of adapting the instrument in more detail, summarizing Alonso et al. 2022.

2. There is no description of the sampling technique used and no information on how they determined the population was normal
How was this addressed in the manuscript?
We used an intentional sampling with an N=948. We have clarified that the sample came from INAIPI beneficiaries who lived in the capital city of Santo Domingo and provided additional details.

3. There is clear information and a report on reliability but not validity.
How was this addressed in the manuscript?
We have included additional details on the creation of SIMEDID that includes more validity information, such as collaboration with experts, content analysis, and ongoing data collection activities. As SIMEDID transitions into the national surveillance system, continued data collection will contribute further evidence of its validity.

4. Figures should be labeled appropriately with an explanation of the range of colors' meanings.
How was this addressed in the manuscript?
We have added a note to each figure explaining the color range.

5. The Y axis of the item should not only mention the domain but also what analysis
How was this addressed in the manuscript?
The Y represents the distribution of the probability of success, as explained by the color ranges. We have added this information to each figure.

6. Interpretation of the result is poor. The researchers gave results of the Chronbach alpha and split-half correlation but did not elaborate on the relevance
How was this addressed in the manuscript?
We have expanded the conclusion with an additional interpretation of the results.

7. They did not comment on how the instrument should be used in the future, especially with items that had a standard deviation that was higher than 3 and even some items that had a standard deviation greater or equal to 6.
How was this addressed in the manuscript?
In the future, SIMEDID will identify children at risk for developmental delays. The current study aimed to enhance item presentation order. The observed large standard deviations may result from presenting items in an order that doesn't fully adapt to the Dominican context. The upcoming standardization study, which includes a more fitting presentation order of items obtained in the present validation study, seeks to establish national norms by calculating cut-off points based on standard deviations once the presentation order is appropriately configured. We have explained these plans in the discussion.
Dear Dr. Mulaudzi,

We thank you for your comments and questions, which have helped us improve the quality of our manuscript. We address your questions below and in the manuscript:

1. The researchers should have shared more on their initial work of adapting MDAT to the Dominican instrument and how it differs.
How was this addressed in the manuscript?
We have described the process of adapting the instrument in more detail, summarizing Alonso et al. 2022.

2. There is no description of the sampling technique used and no information on how they determined the population was normal
How was this addressed in the manuscript?
We used an intentional sampling with an N=948. We have clarified that the sample came from INAIPI beneficiaries who lived in the capital city of Santo Domingo and provided additional details.

3. There is clear information and a report on reliability but not validity.
How was this addressed in the manuscript?
We have included additional details on the creation of SIMEDID that includes more validity information, such as collaboration with experts, content analysis, and ongoing data collection activities. As SIMEDID transitions into the national surveillance system, continued data collection will contribute further evidence of its validity.

4. Figures should be labeled appropriately with an explanation of the range of colors' meanings.
How was this addressed in the manuscript?
We have added a note to each figure explaining the color range.

5. The Y axis of the item should not only mention the domain but also what analysis
How was this addressed in the manuscript?
The Y represents the distribution of the probability of success, as explained by the color ranges. We have added this information to each figure.

6. Interpretation of the result is poor. The researchers gave results of the Chronbach alpha and split-half correlation but did not elaborate on the relevance
How was this addressed in the manuscript?
We have expanded the conclusion with an additional interpretation of the results.

7. They did not comment on how the instrument should be used in the future, especially with items that had a standard deviation that was higher than 3 and even some items that had a standard deviation greater or equal to 6.
How was this addressed in the manuscript?
In the future, SIMEDID will identify children at risk for developmental delays. The current study aimed to enhance item presentation order. The observed large standard deviations may result from presenting items in an order that doesn't fully adapt to the Dominican context. The upcoming standardization study, which includes a more fitting presentation order of items obtained in the present validation study, seeks to establish national norms by calculating cut-off points based on standard deviations once the presentation order is appropriately configured. We have explained these plans in the discussion.
Competing Interests: N/A Close
Report a concern

Views

Reviewer Report 11 Apr 2023

Melissa J. Gladstone, Department of Women and Children's Health, Institute of Life Course and Medical Sciences, University of Liverpool, Liverpool, UK

Not Approved

https://doi.org/10.5256/f1000research.141269.r166639

This article provides information about the recent validation of a system for measuring child development in Dominican Republic.
This is quite a large study which provides some initial information as to how a new tool for measuring child development in the Dominican Republic is working. This is extremely laudable and very important. Tools such as this are required to ensure that better developmental surveillance can be done across countries globally.

I do have some reservations that could be addressed but presently the article is not in a fit state to publish. I outline these below.

Methods:

It is not clear if the tool has been adapted from the mentioned MDAT tool and whether it is parent report or directly observed or a mixture of the two. It is not clear if it needs a kit to support assessment or not. It is also not clear if there are any pictures or materials that have been created to support assessment. It is also not entirely clear if it has been piloted prior to this use.

The authors describe how the tool was reviewed by an expert pane to provide information on construct validity but also to revise the items and their definitions. The authors provide examples of items they modified but it is not clear what the initial item was. It might be helpful to have a list of items and what was and was not changed. It also might be helpful to use the COSMIN framework to clarify the various validation procedures to ensure they were done as would be recommended and if not, this could be discussed more in the discussion part of the paper.

Do the items have to be passed in succession? Can they move from one area to another within the app?

Methods

The authors describe that a Normative sample of children were assessed. Can the authors be clearer about how this is known. It looks like the children who were assessed were those who attended Comprehensive Care Centers for Early Childhood but it is not clear how much or little these represent the general population of children in the Dominican Republic. The authors do not make it clear if any other demographics were taken into account e.g. born prematurely? Birth weight? Adversities and sociodemographic status (maternal education), health status and frequency of admissions to hospital, nutritional status etc?

The authors describe the use of Cronbach’s alpha internal reliability and “split half parallel reliability” but it is not clear if any reliability was done at the time of training and whether any quality control was done between examiners?

What did the 6 hour session for training consist of? And could it be clearer who did this and what is needed for the future? Were there online sessions? In person sessions? Any materials that were created? Certification of assessors?

Furthermore, could the authors provide more information about whether children were assessed with a disability and if not, how did they decide to exclude them? Can the present tool go down further to assess a child who is not performing? How did the team decide if they had a child that they felt was below the norm?

Discussion

The authors describe that age did not predict certain items. Is this because these items are not acceptable or work in the setting or is it because there were not enough children to assess these items? It seems the latter.

The authors conclude that the study passes three elements which include that it is reliable and valid. The authors may want to check the COSMIN criteria to make decisions on what reliability and validity they are describing as having definitively concluded on through this study. It is not clear that the authors have definitively shown that the tool can sensitively identify changes? Could they make that clearer and that they have demonstrated its use in another population who are known to have a problem through doing a Gold Standard assessment of these children.

Could the authors provide a little more information on what they plan to do with the info from this study? At the moment, it seems like it goes into a data collection system but what and how will they create “cut offs” decided for referral? These next steps are very important and could be provided more clearly for the reader.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

No
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

No
Are the conclusions drawn adequately supported by the results?

No

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Neurodevelopmental Paediatrics and International Child Health

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

CITE

Report a concern

Author Response 22 Mar 2024

Laura V. Sánchez-Vincitore, Universidad Iberoamericana (Unibe), Santo Domingo, Dominican Republic

22 Mar 2024

Author Response

Dear Dr. Gladstone,

We are thankful for your constructive feedback, which has contributed to improving our manuscript. We have addressed all your questions and comments, as explained below and ... Continue reading Dear Dr. Gladstone,

We are thankful for your constructive feedback, which has contributed to improving our manuscript. We have addressed all your questions and comments, as explained below and included in the manuscript’s new version.

1. It is not clear if the tool has been adapted from the mentioned MDAT tool.

The SIMEDID tool has been adapted from the MDAT tool and other instruments, as reported by Alonso et al. 2022, which provides more details on the process of adaptation and validation.

How was this addressed in the manuscript?
We described the process of adapting the instrument in more detail, summarizing Alonso et al. 2022.

2. It is not clear if the assessment tool requires direct observation or is based on parent
report, or a combination of both.

The SIMEDID tool combines direct observation of the child's behavior by educational agents and reports from individuals who have spent enough time with the child. That person must have observed the desired behavior. For example, in INAIPI’s case, the evaluator is the person in contact with the child, the educational agent, the community agent, and/or the parent.

How was this addressed in the manuscript?
We included the following statement in the methods section: “The assessment tool requires an observation of the child’s behavior, either by the evaluator, parent, or the person in charge of the child at a given time. In case the child does not show the expected behavior at the time of testing, the informant’s report is considered a valid response as long as they have witnessed the expected behavior from the child.”

3. It is not clear if the assessment tool requires a kit to support the assessment or not.

Yes, there is a kit available to support the assessment with the SIMEDID tool.

How was this addressed in the manuscript?
We included the following statement in the methods section: “The INAIPI team created a kit to support the assessment tool (which includes props, pictures, and everyday objects). Materials are easily found in the Dominican Republic as a strategy for upscaling data collection.”

4. It is not entirely clear if there are any pictures or materials that have been created to support the assessment.

Yes, the SIMEDID tool includes pictures and materials to support the assessment.

How was this addressed in the manuscript?
See comment #3.

5. It is not entirely clear if the assessment tool has been piloted before this use.

The SIMEDID assessment tool underwent an initial pilot study involving 100 children before its use in this study. The pilot study, which is unpublished, provided valuable insights into the tool's performance and informed the validation process.

How was this addressed in the manuscript?
The following statement has been included “The pilot study included 100 children aged 45 days to 5 years who were receiving INAIPI services. The study confirmed the platform’s offline functionality, speed, and user-friendly interface, along with the willingness of educators and animators to participate. In addition, the instrument showed good internal consistency.”

6. It might be helpful to have a list of items and what was and was not changed.

The extended data now include a list of the items in the SIMEDID tool and details on what was and was not changed during the adaptation process.

How was this addressed in the manuscript?
The extended data are referenced in the manuscript.

7. It might be helpful to use the COSMIN framework to clarify the various validation procedures to ensure they were done as would be recommended.

A new supplementary table that follows the COSMIN framework has been included in the extended data. This table will serve as a live document to track data collection activities and ensure that the instrument continues to improve over time.

How was this addressed in the manuscript?
The extended data are referenced in the manuscript.

8. Do the items have to be passed in succession? Can they move from one area to another within the app?

Items must be completed in sequence as they are organized based on increasing difficulty and follow an algorithm that assesses children's development from minimum to maximum levels.
Evaluators can easily switch between different developmental areas by clicking on tabs with corresponding names. However, to progress to the next item, the current one must be answered. Additionally, to finish the test and submit results, all age-specific items must be filled out. Any missing responses will trigger an alert from the platform.

How was this addressed in the manuscript?
We included the following statement in the instruments section:
“Each subtest comprises 33 items, except for language development, which has 34 items; all are arranged in ascending difficulty levels. The initial presentation of the first item is age-dependent. Once the first item on each dimension is completed, the instrument presents items in reverse until the participant successfully completes three items. Subsequently, the tool proceeds forward until the participant misses three items. This approach enables evaluators to ascertain the child's current developmental state, identifying potential delays (backward presentation) or advanced development (forward presentation). Items must be completed in sequence as they are organized based on increasing difficulty and follow an algorithm that assesses children's development from minimum to maximum levels. Evaluators can easily switch between different developmental areas by clicking on tabs with corresponding names.
The study's primary aim was to determine the age range at which each item is accomplished, utilizing the Malawi MDAT norms for the initial trial. Subsequent data collection will adopt the sequence based on the findings from the present validation study.”

9. Can the authors be clearer about how they obtained the sample of children for the study and if these children are representative of the general population in the Dominican Republic?

The children in this study attended CAIPIs and CAFIs in the metropolitan area. While this may not be a fully representative sample of the general population of children in the Dominican Republic, we are currently obtaining norms from a representative sample of INAIPI beneficiaries from all regions. Future data collection activities will also aim to include a sample of children outside of INAIPI to ensure greater representativeness.

How was this addressed in the manuscript?
We have included the following statement in the discussion section: “One limitation of this study is that the sample may not accurately reflect all children in the Dominican Republic since it comprises only those involved in INAIPI programs within the metropolitan region. During the study period, 77,000 children enrolled in INAIPI from the metropolitan region accounted for 1.23 percent of the total population of children ages 0 to 5. The limited sample size was influenced by the challenges posed by the COVID-19 pandemic, including restrictions and prevention measures.”

10. Were other demographic factors taken into account, such as birth weight, adversities, sociodemographic status, health status, frequency of admissions to hospital, nutritional status, etc.?

For the the current validation study, no other demographic factors were considered since we were looking for a progression in item order. However, future data collection activities, which will involve a larger sample of children from INAIPI, will take into account variables such as birth weight, prematurity, and other health factors.

How was this addressed in the manuscript?
The following statement has been included in the method section. “No other demographic factors were considered for this analysis.” In addition, the following statement was included in the discussion section: “Another limitation is that the study did not account for other relevant factors in childhood development such as low birth weight and other health factors. However, future data collection activities, which will involve a larger sample of children from INAIPI, will consider variables such as birth weight, prematurity, and other health factors to determine cutoff points that better identify children at risk for developmental delay and to further understand the sensitivity of SIMEDID.”

11. The authors describe the use of Cronbach’s alpha for internal reliability and “split half parallel reliability”, but it is unclear if any reliability was done at the time of training and whether any quality control was done between examiners?

During the pilot (unpublished), Cronbach’s alpha and split-half parallel reliability were conducted to ensure the reliability of the instrument. Additionally, we conducted a quality control analysis to check for consistency between examiners. This involved comparing the mean scores of participants per evaluator.

How was this addressed in the manuscript?
The following statement has been included in the method section: “To guarantee uniformity in assessments among evaluators, we conducted an analysis of variance (ANOVA), comparing the mean scores of participants within each age group per evaluator. The results revealed no significant differences
between evaluators within each age group across all dimensions.”

12. What did the 6-hour session for training consist of? Could it be clearer who did this and what is needed for the future? Were there online sessions? In-person sessions? Any materials that were created? Certification of assessors?

The 6-hour training session, conducted by INAIPI personnel, encompassed practical and theoretical components based on a module designed by experts in training and child development. The training, mainly conducted face-to-face, included virtual follow-up by the Child Development Evaluation Division. While there was no initial certification process for assessors, facilitators of the training received certification as official SIMEDID facilitators. The module covered essential topics such as the concept of child development, the Dominican Child Development Measurement System (SIMEDID), its structure, and dimensions. It explained who applies SIMEDID in INAIPI's service modalities and provided guidelines on application rules. Evaluators learned the assessment process in homes, including guidance for parents, the periodicity of the system, and the specifications of the tool. The practical section involved the hands-on use of evaluation kit materials in simulated assessments, emphasizing adherence to instructions for standardized data. The training concluded with a presentation on the app's functionality, using visual aids and videos to reinforce the covered content.

How was this addressed in the manuscript?
The following statement has been included in the method section: “A total of 40 evaluators received a six-hour training session and conducted two practice evaluations. The training session comprised both theoretical and practical components. The training, facilitated by experts in training and child development, covered essential topics such as the concept of child development and SIMEDID. It included virtual follow-ups by the Child Development Evaluation Division and certification for facilitators. The module provided insights into SIMEDID's structure, dimensions, and its application in INAIPI's service modalities. Evaluators learned about assessment processes in homes, including guidance for parents, system periodicity, and tool specifications. The practical section involved hands-on use of evaluation kit materials in simulated assessments, emphasizing adherence to standardized data collection. The training concluded with a presentation on the SIMEDID app's functionality, utilizing visual aids and videos to reinforce the covered content. At the end of the training, evaluators were certified as official SIMEDID evaluators.”

13. Furthermore, could the authors provide more information about whether children were assessed with a disability, and if not, how did they decide to exclude them? Can the present tool go down further to assess a child who is not performing? How did the team decide if they had a child that they felt was below the norm?

For this specific data collection activity, we did not exclude children with disabilities because such data were not available. However, for the normalization study, in which we are setting the cutoff points to detect developmental delays, we have a mechanism to exclude them from the analysis, not from the evaluation. This way, we will be able to determine SIMEDID's sensitivity to disability. The tool is designed to assess children from ages 0 to 5, and there are items that can be used to assess children who are not performing at their age level.

How was this addressed in the manuscript?
The following statement has been included in the discussion “Another limitation is that the study did not account for other relevant factors in childhood development, such as low birth weight and other health factors or disabilities. However, future data collection activities, which will involve a larger sample of children from INAIPI, will consider variables such as birth weight, prematurity, and other health factors to determine cutoff points that better identify children at risk for developmental delays and to further understand the sensitivity of SIMEDID.”

14. The authors describe that age did not predict certain items. Is this because these items are not acceptable or work in the setting, or is it because there were not enough children to assess these items? It seems to be the latter.

We did not have enough children to assess these items, and we are currently working on obtaining a sample of children within the age range to evaluate these items. We are meeting with other governmental institutions that work with infants younger than 45 days to expand the age range of the study.

How was this addressed in the manuscript?
The following statement has been included in the discussion section: “We could not validate initial items of SIMEDID given that INAIPI does not provide services to infants younger than 45 days, given that mothers are on maternity leave during that period. However, the research team is meeting other governmental institutions that work with infants that young to expand the age range of the study. “

15. It is not clear that the authors have definitively shown that the tool can sensitively identify changes. Could they make that clearer and that they have demonstrated its use in another population who are known to have a problem through doing a Gold Standard assessment of these children?

We have yet to demonstrate the tool's ability to sensitively identify changes, as this is the first large-scale data collection using this instrument. Our current focus is to establish the reliability and validity of the SIMEDID tool. However, we are designing a study to compare the SIMEDID tool with another widely used instrument by UNICEF that is applied during household surveys.

How was this addressed in the manuscript?
The following statement has been included in the discussion section: “We have yet to demonstrate the tool's ability to sensitively identify changes, as this is the first large-scale data collection using this instrument. However, since SIMEDID is already part of the monitoring strategy, with constant data collection, it would be relatively simple to demonstrate its sensitivity to changes.”

16. Could the authors provide a little more information on what they plan to do with the information from this study? At the moment, it seems like it goes into a data collection system, but what and how will they create “cut-offs” decided for referral? These next steps are very important and could be provided more clearly for the reader.

The information collected from this study will be used to establish cut-off scores for referral to early intervention services. We will use statistical methods to determine the optimal cut-offs.  A subsequent standardization study allowed us to establish correction parameters and normative scores, with which it will be possible to establish when alerts occur in the development of a child.

How was this addressed in the manuscript?
This was addressed in the Discussion section:
“These results will allow the pertinent institutions of the Dominican Republic to implement and report more accurate early childhood development indicators. Still, they will also contribute to creating a robust monitoring system with a high-quality data collection process that allows evidence-based and timely decision-making. Furthermore, such a system will contribute to generating longitudinal data that establish the association of childhood development with sociodemographic and psychosocial variables and determine the impact of initiatives and interventions (Richter et al., 2017), which is not sufficiently evaluated in most countries (Daelmans et al., 2017).”

Dear Dr. Gladstone,

We are thankful for your constructive feedback, which has contributed to improving our manuscript. We have addressed all your questions and comments, as explained below and included in the manuscript’s new version.

1. It is not clear if the tool has been adapted from the mentioned MDAT tool.

The SIMEDID tool has been adapted from the MDAT tool and other instruments, as reported by Alonso et al. 2022, which provides more details on the process of adaptation and validation.

How was this addressed in the manuscript?
We described the process of adapting the instrument in more detail, summarizing Alonso et al. 2022.

2. It is not clear if the assessment tool requires direct observation or is based on parent
report, or a combination of both.

The SIMEDID tool combines direct observation of the child's behavior by educational agents and reports from individuals who have spent enough time with the child. That person must have observed the desired behavior. For example, in INAIPI’s case, the evaluator is the person in contact with the child, the educational agent, the community agent, and/or the parent.

How was this addressed in the manuscript?
We included the following statement in the methods section: “The assessment tool requires an observation of the child’s behavior, either by the evaluator, parent, or the person in charge of the child at a given time. In case the child does not show the expected behavior at the time of testing, the informant’s report is considered a valid response as long as they have witnessed the expected behavior from the child.”

3. It is not clear if the assessment tool requires a kit to support the assessment or not.

Yes, there is a kit available to support the assessment with the SIMEDID tool.

How was this addressed in the manuscript?
We included the following statement in the methods section: “The INAIPI team created a kit to support the assessment tool (which includes props, pictures, and everyday objects). Materials are easily found in the Dominican Republic as a strategy for upscaling data collection.”

4. It is not entirely clear if there are any pictures or materials that have been created to support the assessment.

Yes, the SIMEDID tool includes pictures and materials to support the assessment.

How was this addressed in the manuscript?
See comment #3.

5. It is not entirely clear if the assessment tool has been piloted before this use.

The SIMEDID assessment tool underwent an initial pilot study involving 100 children before its use in this study. The pilot study, which is unpublished, provided valuable insights into the tool's performance and informed the validation process.

How was this addressed in the manuscript?
The following statement has been included “The pilot study included 100 children aged 45 days to 5 years who were receiving INAIPI services. The study confirmed the platform’s offline functionality, speed, and user-friendly interface, along with the willingness of educators and animators to participate. In addition, the instrument showed good internal consistency.”

6. It might be helpful to have a list of items and what was and was not changed.

The extended data now include a list of the items in the SIMEDID tool and details on what was and was not changed during the adaptation process.

How was this addressed in the manuscript?
The extended data are referenced in the manuscript.

7. It might be helpful to use the COSMIN framework to clarify the various validation procedures to ensure they were done as would be recommended.

A new supplementary table that follows the COSMIN framework has been included in the extended data. This table will serve as a live document to track data collection activities and ensure that the instrument continues to improve over time.

How was this addressed in the manuscript?
The extended data are referenced in the manuscript.

8. Do the items have to be passed in succession? Can they move from one area to another within the app?

Items must be completed in sequence as they are organized based on increasing difficulty and follow an algorithm that assesses children's development from minimum to maximum levels.
Evaluators can easily switch between different developmental areas by clicking on tabs with corresponding names. However, to progress to the next item, the current one must be answered. Additionally, to finish the test and submit results, all age-specific items must be filled out. Any missing responses will trigger an alert from the platform.

How was this addressed in the manuscript?
We included the following statement in the instruments section:
“Each subtest comprises 33 items, except for language development, which has 34 items; all are arranged in ascending difficulty levels. The initial presentation of the first item is age-dependent. Once the first item on each dimension is completed, the instrument presents items in reverse until the participant successfully completes three items. Subsequently, the tool proceeds forward until the participant misses three items. This approach enables evaluators to ascertain the child's current developmental state, identifying potential delays (backward presentation) or advanced development (forward presentation). Items must be completed in sequence as they are organized based on increasing difficulty and follow an algorithm that assesses children's development from minimum to maximum levels. Evaluators can easily switch between different developmental areas by clicking on tabs with corresponding names.
The study's primary aim was to determine the age range at which each item is accomplished, utilizing the Malawi MDAT norms for the initial trial. Subsequent data collection will adopt the sequence based on the findings from the present validation study.”

9. Can the authors be clearer about how they obtained the sample of children for the study and if these children are representative of the general population in the Dominican Republic?

The children in this study attended CAIPIs and CAFIs in the metropolitan area. While this may not be a fully representative sample of the general population of children in the Dominican Republic, we are currently obtaining norms from a representative sample of INAIPI beneficiaries from all regions. Future data collection activities will also aim to include a sample of children outside of INAIPI to ensure greater representativeness.

How was this addressed in the manuscript?
We have included the following statement in the discussion section: “One limitation of this study is that the sample may not accurately reflect all children in the Dominican Republic since it comprises only those involved in INAIPI programs within the metropolitan region. During the study period, 77,000 children enrolled in INAIPI from the metropolitan region accounted for 1.23 percent of the total population of children ages 0 to 5. The limited sample size was influenced by the challenges posed by the COVID-19 pandemic, including restrictions and prevention measures.”

10. Were other demographic factors taken into account, such as birth weight, adversities, sociodemographic status, health status, frequency of admissions to hospital, nutritional status, etc.?

For the the current validation study, no other demographic factors were considered since we were looking for a progression in item order. However, future data collection activities, which will involve a larger sample of children from INAIPI, will take into account variables such as birth weight, prematurity, and other health factors.

How was this addressed in the manuscript?
The following statement has been included in the method section. “No other demographic factors were considered for this analysis.” In addition, the following statement was included in the discussion section: “Another limitation is that the study did not account for other relevant factors in childhood development such as low birth weight and other health factors. However, future data collection activities, which will involve a larger sample of children from INAIPI, will consider variables such as birth weight, prematurity, and other health factors to determine cutoff points that better identify children at risk for developmental delay and to further understand the sensitivity of SIMEDID.”

11. The authors describe the use of Cronbach’s alpha for internal reliability and “split half parallel reliability”, but it is unclear if any reliability was done at the time of training and whether any quality control was done between examiners?

During the pilot (unpublished), Cronbach’s alpha and split-half parallel reliability were conducted to ensure the reliability of the instrument. Additionally, we conducted a quality control analysis to check for consistency between examiners. This involved comparing the mean scores of participants per evaluator.

How was this addressed in the manuscript?
The following statement has been included in the method section: “To guarantee uniformity in assessments among evaluators, we conducted an analysis of variance (ANOVA), comparing the mean scores of participants within each age group per evaluator. The results revealed no significant differences
between evaluators within each age group across all dimensions.”

12. What did the 6-hour session for training consist of? Could it be clearer who did this and what is needed for the future? Were there online sessions? In-person sessions? Any materials that were created? Certification of assessors?

The 6-hour training session, conducted by INAIPI personnel, encompassed practical and theoretical components based on a module designed by experts in training and child development. The training, mainly conducted face-to-face, included virtual follow-up by the Child Development Evaluation Division. While there was no initial certification process for assessors, facilitators of the training received certification as official SIMEDID facilitators. The module covered essential topics such as the concept of child development, the Dominican Child Development Measurement System (SIMEDID), its structure, and dimensions. It explained who applies SIMEDID in INAIPI's service modalities and provided guidelines on application rules. Evaluators learned the assessment process in homes, including guidance for parents, the periodicity of the system, and the specifications of the tool. The practical section involved the hands-on use of evaluation kit materials in simulated assessments, emphasizing adherence to instructions for standardized data. The training concluded with a presentation on the app's functionality, using visual aids and videos to reinforce the covered content.

How was this addressed in the manuscript?
The following statement has been included in the method section: “A total of 40 evaluators received a six-hour training session and conducted two practice evaluations. The training session comprised both theoretical and practical components. The training, facilitated by experts in training and child development, covered essential topics such as the concept of child development and SIMEDID. It included virtual follow-ups by the Child Development Evaluation Division and certification for facilitators. The module provided insights into SIMEDID's structure, dimensions, and its application in INAIPI's service modalities. Evaluators learned about assessment processes in homes, including guidance for parents, system periodicity, and tool specifications. The practical section involved hands-on use of evaluation kit materials in simulated assessments, emphasizing adherence to standardized data collection. The training concluded with a presentation on the SIMEDID app's functionality, utilizing visual aids and videos to reinforce the covered content. At the end of the training, evaluators were certified as official SIMEDID evaluators.”

13. Furthermore, could the authors provide more information about whether children were assessed with a disability, and if not, how did they decide to exclude them? Can the present tool go down further to assess a child who is not performing? How did the team decide if they had a child that they felt was below the norm?

For this specific data collection activity, we did not exclude children with disabilities because such data were not available. However, for the normalization study, in which we are setting the cutoff points to detect developmental delays, we have a mechanism to exclude them from the analysis, not from the evaluation. This way, we will be able to determine SIMEDID's sensitivity to disability. The tool is designed to assess children from ages 0 to 5, and there are items that can be used to assess children who are not performing at their age level.

How was this addressed in the manuscript?
The following statement has been included in the discussion “Another limitation is that the study did not account for other relevant factors in childhood development, such as low birth weight and other health factors or disabilities. However, future data collection activities, which will involve a larger sample of children from INAIPI, will consider variables such as birth weight, prematurity, and other health factors to determine cutoff points that better identify children at risk for developmental delays and to further understand the sensitivity of SIMEDID.”

14. The authors describe that age did not predict certain items. Is this because these items are not acceptable or work in the setting, or is it because there were not enough children to assess these items? It seems to be the latter.

We did not have enough children to assess these items, and we are currently working on obtaining a sample of children within the age range to evaluate these items. We are meeting with other governmental institutions that work with infants younger than 45 days to expand the age range of the study.

How was this addressed in the manuscript?
The following statement has been included in the discussion section: “We could not validate initial items of SIMEDID given that INAIPI does not provide services to infants younger than 45 days, given that mothers are on maternity leave during that period. However, the research team is meeting other governmental institutions that work with infants that young to expand the age range of the study. “

15. It is not clear that the authors have definitively shown that the tool can sensitively identify changes. Could they make that clearer and that they have demonstrated its use in another population who are known to have a problem through doing a Gold Standard assessment of these children?

We have yet to demonstrate the tool's ability to sensitively identify changes, as this is the first large-scale data collection using this instrument. Our current focus is to establish the reliability and validity of the SIMEDID tool. However, we are designing a study to compare the SIMEDID tool with another widely used instrument by UNICEF that is applied during household surveys.

How was this addressed in the manuscript?
The following statement has been included in the discussion section: “We have yet to demonstrate the tool's ability to sensitively identify changes, as this is the first large-scale data collection using this instrument. However, since SIMEDID is already part of the monitoring strategy, with constant data collection, it would be relatively simple to demonstrate its sensitivity to changes.”

16. Could the authors provide a little more information on what they plan to do with the information from this study? At the moment, it seems like it goes into a data collection system, but what and how will they create “cut-offs” decided for referral? These next steps are very important and could be provided more clearly for the reader.

The information collected from this study will be used to establish cut-off scores for referral to early intervention services. We will use statistical methods to determine the optimal cut-offs.  A subsequent standardization study allowed us to establish correction parameters and normative scores, with which it will be possible to establish when alerts occur in the development of a child.

How was this addressed in the manuscript?
This was addressed in the Discussion section:
“These results will allow the pertinent institutions of the Dominican Republic to implement and report more accurate early childhood development indicators. Still, they will also contribute to creating a robust monitoring system with a high-quality data collection process that allows evidence-based and timely decision-making. Furthermore, such a system will contribute to generating longitudinal data that establish the association of childhood development with sociodemographic and psychosocial variables and determine the impact of initiatives and interventions (Richter et al., 2017), which is not sufficiently evaluated in most countries (Daelmans et al., 2017).”

Competing Interests: N/A Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 22 Mar 2024

Laura V. Sánchez-Vincitore, Universidad Iberoamericana (Unibe), Santo Domingo, Dominican Republic

22 Mar 2024

Author Response

Dear Dr. Gladstone,

We are thankful for your constructive feedback, which has contributed to improving our manuscript. We have addressed all your questions and comments, as explained below and ... Continue reading Dear Dr. Gladstone,

We are thankful for your constructive feedback, which has contributed to improving our manuscript. We have addressed all your questions and comments, as explained below and included in the manuscript’s new version.

1. It is not clear if the tool has been adapted from the mentioned MDAT tool.

The SIMEDID tool has been adapted from the MDAT tool and other instruments, as reported by Alonso et al. 2022, which provides more details on the process of adaptation and validation.

How was this addressed in the manuscript?
We described the process of adapting the instrument in more detail, summarizing Alonso et al. 2022.

2. It is not clear if the assessment tool requires direct observation or is based on parent
report, or a combination of both.

The SIMEDID tool combines direct observation of the child's behavior by educational agents and reports from individuals who have spent enough time with the child. That person must have observed the desired behavior. For example, in INAIPI’s case, the evaluator is the person in contact with the child, the educational agent, the community agent, and/or the parent.

How was this addressed in the manuscript?
We included the following statement in the methods section: “The assessment tool requires an observation of the child’s behavior, either by the evaluator, parent, or the person in charge of the child at a given time. In case the child does not show the expected behavior at the time of testing, the informant’s report is considered a valid response as long as they have witnessed the expected behavior from the child.”

3. It is not clear if the assessment tool requires a kit to support the assessment or not.

Yes, there is a kit available to support the assessment with the SIMEDID tool.

How was this addressed in the manuscript?
We included the following statement in the methods section: “The INAIPI team created a kit to support the assessment tool (which includes props, pictures, and everyday objects). Materials are easily found in the Dominican Republic as a strategy for upscaling data collection.”

4. It is not entirely clear if there are any pictures or materials that have been created to support the assessment.

Yes, the SIMEDID tool includes pictures and materials to support the assessment.

How was this addressed in the manuscript?
See comment #3.

5. It is not entirely clear if the assessment tool has been piloted before this use.

The SIMEDID assessment tool underwent an initial pilot study involving 100 children before its use in this study. The pilot study, which is unpublished, provided valuable insights into the tool's performance and informed the validation process.

How was this addressed in the manuscript?
The following statement has been included “The pilot study included 100 children aged 45 days to 5 years who were receiving INAIPI services. The study confirmed the platform’s offline functionality, speed, and user-friendly interface, along with the willingness of educators and animators to participate. In addition, the instrument showed good internal consistency.”

6. It might be helpful to have a list of items and what was and was not changed.

The extended data now include a list of the items in the SIMEDID tool and details on what was and was not changed during the adaptation process.

How was this addressed in the manuscript?
The extended data are referenced in the manuscript.

7. It might be helpful to use the COSMIN framework to clarify the various validation procedures to ensure they were done as would be recommended.

A new supplementary table that follows the COSMIN framework has been included in the extended data. This table will serve as a live document to track data collection activities and ensure that the instrument continues to improve over time.

How was this addressed in the manuscript?
The extended data are referenced in the manuscript.

8. Do the items have to be passed in succession? Can they move from one area to another within the app?

Items must be completed in sequence as they are organized based on increasing difficulty and follow an algorithm that assesses children's development from minimum to maximum levels.
Evaluators can easily switch between different developmental areas by clicking on tabs with corresponding names. However, to progress to the next item, the current one must be answered. Additionally, to finish the test and submit results, all age-specific items must be filled out. Any missing responses will trigger an alert from the platform.

How was this addressed in the manuscript?
We included the following statement in the instruments section:
“Each subtest comprises 33 items, except for language development, which has 34 items; all are arranged in ascending difficulty levels. The initial presentation of the first item is age-dependent. Once the first item on each dimension is completed, the instrument presents items in reverse until the participant successfully completes three items. Subsequently, the tool proceeds forward until the participant misses three items. This approach enables evaluators to ascertain the child's current developmental state, identifying potential delays (backward presentation) or advanced development (forward presentation). Items must be completed in sequence as they are organized based on increasing difficulty and follow an algorithm that assesses children's development from minimum to maximum levels. Evaluators can easily switch between different developmental areas by clicking on tabs with corresponding names.
The study's primary aim was to determine the age range at which each item is accomplished, utilizing the Malawi MDAT norms for the initial trial. Subsequent data collection will adopt the sequence based on the findings from the present validation study.”

9. Can the authors be clearer about how they obtained the sample of children for the study and if these children are representative of the general population in the Dominican Republic?

The children in this study attended CAIPIs and CAFIs in the metropolitan area. While this may not be a fully representative sample of the general population of children in the Dominican Republic, we are currently obtaining norms from a representative sample of INAIPI beneficiaries from all regions. Future data collection activities will also aim to include a sample of children outside of INAIPI to ensure greater representativeness.

How was this addressed in the manuscript?
We have included the following statement in the discussion section: “One limitation of this study is that the sample may not accurately reflect all children in the Dominican Republic since it comprises only those involved in INAIPI programs within the metropolitan region. During the study period, 77,000 children enrolled in INAIPI from the metropolitan region accounted for 1.23 percent of the total population of children ages 0 to 5. The limited sample size was influenced by the challenges posed by the COVID-19 pandemic, including restrictions and prevention measures.”

10. Were other demographic factors taken into account, such as birth weight, adversities, sociodemographic status, health status, frequency of admissions to hospital, nutritional status, etc.?

For the the current validation study, no other demographic factors were considered since we were looking for a progression in item order. However, future data collection activities, which will involve a larger sample of children from INAIPI, will take into account variables such as birth weight, prematurity, and other health factors.

How was this addressed in the manuscript?
The following statement has been included in the method section. “No other demographic factors were considered for this analysis.” In addition, the following statement was included in the discussion section: “Another limitation is that the study did not account for other relevant factors in childhood development such as low birth weight and other health factors. However, future data collection activities, which will involve a larger sample of children from INAIPI, will consider variables such as birth weight, prematurity, and other health factors to determine cutoff points that better identify children at risk for developmental delay and to further understand the sensitivity of SIMEDID.”

11. The authors describe the use of Cronbach’s alpha for internal reliability and “split half parallel reliability”, but it is unclear if any reliability was done at the time of training and whether any quality control was done between examiners?

During the pilot (unpublished), Cronbach’s alpha and split-half parallel reliability were conducted to ensure the reliability of the instrument. Additionally, we conducted a quality control analysis to check for consistency between examiners. This involved comparing the mean scores of participants per evaluator.

How was this addressed in the manuscript?
The following statement has been included in the method section: “To guarantee uniformity in assessments among evaluators, we conducted an analysis of variance (ANOVA), comparing the mean scores of participants within each age group per evaluator. The results revealed no significant differences
between evaluators within each age group across all dimensions.”

12. What did the 6-hour session for training consist of? Could it be clearer who did this and what is needed for the future? Were there online sessions? In-person sessions? Any materials that were created? Certification of assessors?

The 6-hour training session, conducted by INAIPI personnel, encompassed practical and theoretical components based on a module designed by experts in training and child development. The training, mainly conducted face-to-face, included virtual follow-up by the Child Development Evaluation Division. While there was no initial certification process for assessors, facilitators of the training received certification as official SIMEDID facilitators. The module covered essential topics such as the concept of child development, the Dominican Child Development Measurement System (SIMEDID), its structure, and dimensions. It explained who applies SIMEDID in INAIPI's service modalities and provided guidelines on application rules. Evaluators learned the assessment process in homes, including guidance for parents, the periodicity of the system, and the specifications of the tool. The practical section involved the hands-on use of evaluation kit materials in simulated assessments, emphasizing adherence to instructions for standardized data. The training concluded with a presentation on the app's functionality, using visual aids and videos to reinforce the covered content.

How was this addressed in the manuscript?
The following statement has been included in the method section: “A total of 40 evaluators received a six-hour training session and conducted two practice evaluations. The training session comprised both theoretical and practical components. The training, facilitated by experts in training and child development, covered essential topics such as the concept of child development and SIMEDID. It included virtual follow-ups by the Child Development Evaluation Division and certification for facilitators. The module provided insights into SIMEDID's structure, dimensions, and its application in INAIPI's service modalities. Evaluators learned about assessment processes in homes, including guidance for parents, system periodicity, and tool specifications. The practical section involved hands-on use of evaluation kit materials in simulated assessments, emphasizing adherence to standardized data collection. The training concluded with a presentation on the SIMEDID app's functionality, utilizing visual aids and videos to reinforce the covered content. At the end of the training, evaluators were certified as official SIMEDID evaluators.”

13. Furthermore, could the authors provide more information about whether children were assessed with a disability, and if not, how did they decide to exclude them? Can the present tool go down further to assess a child who is not performing? How did the team decide if they had a child that they felt was below the norm?

For this specific data collection activity, we did not exclude children with disabilities because such data were not available. However, for the normalization study, in which we are setting the cutoff points to detect developmental delays, we have a mechanism to exclude them from the analysis, not from the evaluation. This way, we will be able to determine SIMEDID's sensitivity to disability. The tool is designed to assess children from ages 0 to 5, and there are items that can be used to assess children who are not performing at their age level.

How was this addressed in the manuscript?
The following statement has been included in the discussion “Another limitation is that the study did not account for other relevant factors in childhood development, such as low birth weight and other health factors or disabilities. However, future data collection activities, which will involve a larger sample of children from INAIPI, will consider variables such as birth weight, prematurity, and other health factors to determine cutoff points that better identify children at risk for developmental delays and to further understand the sensitivity of SIMEDID.”

14. The authors describe that age did not predict certain items. Is this because these items are not acceptable or work in the setting, or is it because there were not enough children to assess these items? It seems to be the latter.

We did not have enough children to assess these items, and we are currently working on obtaining a sample of children within the age range to evaluate these items. We are meeting with other governmental institutions that work with infants younger than 45 days to expand the age range of the study.

How was this addressed in the manuscript?
The following statement has been included in the discussion section: “We could not validate initial items of SIMEDID given that INAIPI does not provide services to infants younger than 45 days, given that mothers are on maternity leave during that period. However, the research team is meeting other governmental institutions that work with infants that young to expand the age range of the study. “

15. It is not clear that the authors have definitively shown that the tool can sensitively identify changes. Could they make that clearer and that they have demonstrated its use in another population who are known to have a problem through doing a Gold Standard assessment of these children?

We have yet to demonstrate the tool's ability to sensitively identify changes, as this is the first large-scale data collection using this instrument. Our current focus is to establish the reliability and validity of the SIMEDID tool. However, we are designing a study to compare the SIMEDID tool with another widely used instrument by UNICEF that is applied during household surveys.

How was this addressed in the manuscript?
The following statement has been included in the discussion section: “We have yet to demonstrate the tool's ability to sensitively identify changes, as this is the first large-scale data collection using this instrument. However, since SIMEDID is already part of the monitoring strategy, with constant data collection, it would be relatively simple to demonstrate its sensitivity to changes.”

16. Could the authors provide a little more information on what they plan to do with the information from this study? At the moment, it seems like it goes into a data collection system, but what and how will they create “cut-offs” decided for referral? These next steps are very important and could be provided more clearly for the reader.

The information collected from this study will be used to establish cut-off scores for referral to early intervention services. We will use statistical methods to determine the optimal cut-offs.  A subsequent standardization study allowed us to establish correction parameters and normative scores, with which it will be possible to establish when alerts occur in the development of a child.

How was this addressed in the manuscript?
This was addressed in the Discussion section:
“These results will allow the pertinent institutions of the Dominican Republic to implement and report more accurate early childhood development indicators. Still, they will also contribute to creating a robust monitoring system with a high-quality data collection process that allows evidence-based and timely decision-making. Furthermore, such a system will contribute to generating longitudinal data that establish the association of childhood development with sociodemographic and psychosocial variables and determine the impact of initiatives and interventions (Richter et al., 2017), which is not sufficiently evaluated in most countries (Daelmans et al., 2017).”

Dear Dr. Gladstone,

We are thankful for your constructive feedback, which has contributed to improving our manuscript. We have addressed all your questions and comments, as explained below and included in the manuscript’s new version.

1. It is not clear if the tool has been adapted from the mentioned MDAT tool.

The SIMEDID tool has been adapted from the MDAT tool and other instruments, as reported by Alonso et al. 2022, which provides more details on the process of adaptation and validation.

How was this addressed in the manuscript?
We described the process of adapting the instrument in more detail, summarizing Alonso et al. 2022.

2. It is not clear if the assessment tool requires direct observation or is based on parent
report, or a combination of both.

The SIMEDID tool combines direct observation of the child's behavior by educational agents and reports from individuals who have spent enough time with the child. That person must have observed the desired behavior. For example, in INAIPI’s case, the evaluator is the person in contact with the child, the educational agent, the community agent, and/or the parent.

How was this addressed in the manuscript?
We included the following statement in the methods section: “The assessment tool requires an observation of the child’s behavior, either by the evaluator, parent, or the person in charge of the child at a given time. In case the child does not show the expected behavior at the time of testing, the informant’s report is considered a valid response as long as they have witnessed the expected behavior from the child.”

3. It is not clear if the assessment tool requires a kit to support the assessment or not.

Yes, there is a kit available to support the assessment with the SIMEDID tool.

How was this addressed in the manuscript?
We included the following statement in the methods section: “The INAIPI team created a kit to support the assessment tool (which includes props, pictures, and everyday objects). Materials are easily found in the Dominican Republic as a strategy for upscaling data collection.”

4. It is not entirely clear if there are any pictures or materials that have been created to support the assessment.

Yes, the SIMEDID tool includes pictures and materials to support the assessment.

How was this addressed in the manuscript?
See comment #3.

5. It is not entirely clear if the assessment tool has been piloted before this use.

The SIMEDID assessment tool underwent an initial pilot study involving 100 children before its use in this study. The pilot study, which is unpublished, provided valuable insights into the tool's performance and informed the validation process.

How was this addressed in the manuscript?
The following statement has been included “The pilot study included 100 children aged 45 days to 5 years who were receiving INAIPI services. The study confirmed the platform’s offline functionality, speed, and user-friendly interface, along with the willingness of educators and animators to participate. In addition, the instrument showed good internal consistency.”

6. It might be helpful to have a list of items and what was and was not changed.

The extended data now include a list of the items in the SIMEDID tool and details on what was and was not changed during the adaptation process.

How was this addressed in the manuscript?
The extended data are referenced in the manuscript.

7. It might be helpful to use the COSMIN framework to clarify the various validation procedures to ensure they were done as would be recommended.

A new supplementary table that follows the COSMIN framework has been included in the extended data. This table will serve as a live document to track data collection activities and ensure that the instrument continues to improve over time.

How was this addressed in the manuscript?
The extended data are referenced in the manuscript.

8. Do the items have to be passed in succession? Can they move from one area to another within the app?

Items must be completed in sequence as they are organized based on increasing difficulty and follow an algorithm that assesses children's development from minimum to maximum levels.
Evaluators can easily switch between different developmental areas by clicking on tabs with corresponding names. However, to progress to the next item, the current one must be answered. Additionally, to finish the test and submit results, all age-specific items must be filled out. Any missing responses will trigger an alert from the platform.

How was this addressed in the manuscript?
We included the following statement in the instruments section:
“Each subtest comprises 33 items, except for language development, which has 34 items; all are arranged in ascending difficulty levels. The initial presentation of the first item is age-dependent. Once the first item on each dimension is completed, the instrument presents items in reverse until the participant successfully completes three items. Subsequently, the tool proceeds forward until the participant misses three items. This approach enables evaluators to ascertain the child's current developmental state, identifying potential delays (backward presentation) or advanced development (forward presentation). Items must be completed in sequence as they are organized based on increasing difficulty and follow an algorithm that assesses children's development from minimum to maximum levels. Evaluators can easily switch between different developmental areas by clicking on tabs with corresponding names.
The study's primary aim was to determine the age range at which each item is accomplished, utilizing the Malawi MDAT norms for the initial trial. Subsequent data collection will adopt the sequence based on the findings from the present validation study.”

9. Can the authors be clearer about how they obtained the sample of children for the study and if these children are representative of the general population in the Dominican Republic?

The children in this study attended CAIPIs and CAFIs in the metropolitan area. While this may not be a fully representative sample of the general population of children in the Dominican Republic, we are currently obtaining norms from a representative sample of INAIPI beneficiaries from all regions. Future data collection activities will also aim to include a sample of children outside of INAIPI to ensure greater representativeness.

How was this addressed in the manuscript?
We have included the following statement in the discussion section: “One limitation of this study is that the sample may not accurately reflect all children in the Dominican Republic since it comprises only those involved in INAIPI programs within the metropolitan region. During the study period, 77,000 children enrolled in INAIPI from the metropolitan region accounted for 1.23 percent of the total population of children ages 0 to 5. The limited sample size was influenced by the challenges posed by the COVID-19 pandemic, including restrictions and prevention measures.”

10. Were other demographic factors taken into account, such as birth weight, adversities, sociodemographic status, health status, frequency of admissions to hospital, nutritional status, etc.?

For the the current validation study, no other demographic factors were considered since we were looking for a progression in item order. However, future data collection activities, which will involve a larger sample of children from INAIPI, will take into account variables such as birth weight, prematurity, and other health factors.

How was this addressed in the manuscript?
The following statement has been included in the method section. “No other demographic factors were considered for this analysis.” In addition, the following statement was included in the discussion section: “Another limitation is that the study did not account for other relevant factors in childhood development such as low birth weight and other health factors. However, future data collection activities, which will involve a larger sample of children from INAIPI, will consider variables such as birth weight, prematurity, and other health factors to determine cutoff points that better identify children at risk for developmental delay and to further understand the sensitivity of SIMEDID.”

11. The authors describe the use of Cronbach’s alpha for internal reliability and “split half parallel reliability”, but it is unclear if any reliability was done at the time of training and whether any quality control was done between examiners?

During the pilot (unpublished), Cronbach’s alpha and split-half parallel reliability were conducted to ensure the reliability of the instrument. Additionally, we conducted a quality control analysis to check for consistency between examiners. This involved comparing the mean scores of participants per evaluator.

How was this addressed in the manuscript?
The following statement has been included in the method section: “To guarantee uniformity in assessments among evaluators, we conducted an analysis of variance (ANOVA), comparing the mean scores of participants within each age group per evaluator. The results revealed no significant differences
between evaluators within each age group across all dimensions.”

12. What did the 6-hour session for training consist of? Could it be clearer who did this and what is needed for the future? Were there online sessions? In-person sessions? Any materials that were created? Certification of assessors?

The 6-hour training session, conducted by INAIPI personnel, encompassed practical and theoretical components based on a module designed by experts in training and child development. The training, mainly conducted face-to-face, included virtual follow-up by the Child Development Evaluation Division. While there was no initial certification process for assessors, facilitators of the training received certification as official SIMEDID facilitators. The module covered essential topics such as the concept of child development, the Dominican Child Development Measurement System (SIMEDID), its structure, and dimensions. It explained who applies SIMEDID in INAIPI's service modalities and provided guidelines on application rules. Evaluators learned the assessment process in homes, including guidance for parents, the periodicity of the system, and the specifications of the tool. The practical section involved the hands-on use of evaluation kit materials in simulated assessments, emphasizing adherence to instructions for standardized data. The training concluded with a presentation on the app's functionality, using visual aids and videos to reinforce the covered content.

How was this addressed in the manuscript?
The following statement has been included in the method section: “A total of 40 evaluators received a six-hour training session and conducted two practice evaluations. The training session comprised both theoretical and practical components. The training, facilitated by experts in training and child development, covered essential topics such as the concept of child development and SIMEDID. It included virtual follow-ups by the Child Development Evaluation Division and certification for facilitators. The module provided insights into SIMEDID's structure, dimensions, and its application in INAIPI's service modalities. Evaluators learned about assessment processes in homes, including guidance for parents, system periodicity, and tool specifications. The practical section involved hands-on use of evaluation kit materials in simulated assessments, emphasizing adherence to standardized data collection. The training concluded with a presentation on the SIMEDID app's functionality, utilizing visual aids and videos to reinforce the covered content. At the end of the training, evaluators were certified as official SIMEDID evaluators.”

13. Furthermore, could the authors provide more information about whether children were assessed with a disability, and if not, how did they decide to exclude them? Can the present tool go down further to assess a child who is not performing? How did the team decide if they had a child that they felt was below the norm?

For this specific data collection activity, we did not exclude children with disabilities because such data were not available. However, for the normalization study, in which we are setting the cutoff points to detect developmental delays, we have a mechanism to exclude them from the analysis, not from the evaluation. This way, we will be able to determine SIMEDID's sensitivity to disability. The tool is designed to assess children from ages 0 to 5, and there are items that can be used to assess children who are not performing at their age level.

How was this addressed in the manuscript?
The following statement has been included in the discussion “Another limitation is that the study did not account for other relevant factors in childhood development, such as low birth weight and other health factors or disabilities. However, future data collection activities, which will involve a larger sample of children from INAIPI, will consider variables such as birth weight, prematurity, and other health factors to determine cutoff points that better identify children at risk for developmental delays and to further understand the sensitivity of SIMEDID.”

14. The authors describe that age did not predict certain items. Is this because these items are not acceptable or work in the setting, or is it because there were not enough children to assess these items? It seems to be the latter.

We did not have enough children to assess these items, and we are currently working on obtaining a sample of children within the age range to evaluate these items. We are meeting with other governmental institutions that work with infants younger than 45 days to expand the age range of the study.

How was this addressed in the manuscript?
The following statement has been included in the discussion section: “We could not validate initial items of SIMEDID given that INAIPI does not provide services to infants younger than 45 days, given that mothers are on maternity leave during that period. However, the research team is meeting other governmental institutions that work with infants that young to expand the age range of the study. “

15. It is not clear that the authors have definitively shown that the tool can sensitively identify changes. Could they make that clearer and that they have demonstrated its use in another population who are known to have a problem through doing a Gold Standard assessment of these children?

We have yet to demonstrate the tool's ability to sensitively identify changes, as this is the first large-scale data collection using this instrument. Our current focus is to establish the reliability and validity of the SIMEDID tool. However, we are designing a study to compare the SIMEDID tool with another widely used instrument by UNICEF that is applied during household surveys.

How was this addressed in the manuscript?
The following statement has been included in the discussion section: “We have yet to demonstrate the tool's ability to sensitively identify changes, as this is the first large-scale data collection using this instrument. However, since SIMEDID is already part of the monitoring strategy, with constant data collection, it would be relatively simple to demonstrate its sensitivity to changes.”

16. Could the authors provide a little more information on what they plan to do with the information from this study? At the moment, it seems like it goes into a data collection system, but what and how will they create “cut-offs” decided for referral? These next steps are very important and could be provided more clearly for the reader.

The information collected from this study will be used to establish cut-off scores for referral to early intervention services. We will use statistical methods to determine the optimal cut-offs.  A subsequent standardization study allowed us to establish correction parameters and normative scores, with which it will be possible to establish when alerts occur in the development of a child.

How was this addressed in the manuscript?
This was addressed in the Discussion section:
“These results will allow the pertinent institutions of the Dominican Republic to implement and report more accurate early childhood development indicators. Still, they will also contribute to creating a robust monitoring system with a high-quality data collection process that allows evidence-based and timely decision-making. Furthermore, such a system will contribute to generating longitudinal data that establish the association of childhood development with sociodemographic and psychosocial variables and determine the impact of initiatives and interventions (Richter et al., 2017), which is not sufficiently evaluated in most countries (Daelmans et al., 2017).”

Competing Interests: N/A Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 14 Mar 2023

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 29 Jan 24	read	read
Version 1 14 Mar 23	read	read

Melissa J. Gladstone, University of Liverpool, Liverpool, UK
Mphelekedzeni Caroline Mulaudzi, Charlotte Maxeke Johannesburg Academic Hospital, Johannesburg, South Africa; University of Witwater'srand, Johannesburg, South Africa

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

4 Views

22 Apr 2024 | for Version 2

Melissa J. Gladstone, Department of Women and Children's Health, Institute of Life Course and Medical Sciences, University of Liverpool, Liverpool, UK

4 Views Cite this report Responses(0)

Approved

Laura Sanchez-Vincitore and team have worked hard to address comments and the paper is now much better than it was and clearer for the reader to understand the work done. I agree that the results need to consider the epidemiology of the population utilised for this study and that future studies in other populations or larger more well characterised populations may help with this.

Competing Interests

I created the original MDAT tool along with the team at Kamuzu University of Health Sciences in Malawi.

Reviewer Expertise

Neurodevelopmental Paediatrics and International Child Health

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

4 Views

13 Mar 2024 | for Version 2

4 Views Cite this report Responses(0)

Approved

I commend the researchers for addressing my previous concerned very well.
I am still of the opinion that
1. the age standardization the way it was done may underestimate the developmental potential because known physical and medical risk factors were not excluded in the sample for age standardization
2. the SIMEDID is adequate and sufficient for the domains tested. older children around 5 year may need more on cognitive.

In conclusion the above points are for the researchers to consider in their future projects as stated in the article.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Child health and Early Childhood Development

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

18 Views

31 Aug 2023 | for Version 1

18 Views Cite this report Responses(1)

Approved With Reservations

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Partly
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Child health and Early Childhood Development

Respond to this report

Responses (1)

Author Response

22 Mar 2024

Laura V. Sánchez-Vincitore, Universidad Iberoamericana (Unibe), Santo Domingo, Dominican Republic

Dear Dr. Mulaudzi,

We thank you for your comments and questions, which have helped us improve the quality of our manuscript. We address your questions below and in the manuscript:

1. The researchers should have shared more on their initial work of adapting MDAT to the Dominican instrument and how it differs.
How was this addressed in the manuscript?
We have described the process of adapting the instrument in more detail, summarizing Alonso et al. 2022.

2. There is no description of the sampling technique used and no information on how they determined the population was normal
How was this addressed in the manuscript?
We used an intentional sampling with an N=948. We have clarified that the sample came from INAIPI beneficiaries who lived in the capital city of Santo Domingo and provided additional details.

3. There is clear information and a report on reliability but not validity.
How was this addressed in the manuscript?
We have included additional details on the creation of SIMEDID that includes more validity information, such as collaboration with experts, content analysis, and ongoing data collection activities. As SIMEDID transitions into the national surveillance system, continued data collection will contribute further evidence of its validity.

4. Figures should be labeled appropriately with an explanation of the range of colors' meanings.
How was this addressed in the manuscript?
We have added a note to each figure explaining the color range.

5. The Y axis of the item should not only mention the domain but also what analysis
How was this addressed in the manuscript?
The Y represents the distribution of the probability of success, as explained by the color ranges. We have added this information to each figure.

6. Interpretation of the result is poor. The researchers gave results of the Chronbach alpha and split-half correlation but did not elaborate on the relevance
How was this addressed in the manuscript?
We have expanded the conclusion with an additional interpretation of the results.

7. They did not comment on how the instrument should be used in the future, especially with items that had a standard deviation that was higher than 3 and even some items that had a standard deviation greater or equal to 6.
How was this addressed in the manuscript?
In the future, SIMEDID will identify children at risk for developmental delays. The current study aimed to enhance item presentation order. The observed large standard deviations may result from presenting items in an order that doesn't fully adapt to the Dominican context. The upcoming standardization study, which includes a more fitting presentation order of items obtained in the present validation study, seeks to establish national norms by calculating cut-off points based on standard deviations once the presentation order is appropriately configured. We have explained these plans in the discussion.

View more View less

Competing Interests

N/A

Back to all reports

Reviewer Report

37 Views

11 Apr 2023 | for Version 1

Melissa J. Gladstone, Department of Women and Children's Health, Institute of Life Course and Medical Sciences, University of Liverpool, Liverpool, UK

37 Views Cite this report Responses(1)

Not Approved

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

No
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

No
Are the conclusions drawn adequately supported by the results?

No

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Neurodevelopmental Paediatrics and International Child Health

Respond to this report

Responses (1)

Author Response

22 Mar 2024

Laura V. Sánchez-Vincitore, Universidad Iberoamericana (Unibe), Santo Domingo, Dominican Republic

Dear Dr. Gladstone,

We are thankful for your constructive feedback, which has contributed to improving our manuscript. We have addressed all your questions and comments, as explained below and included in the manuscript’s new version.

1. It is not clear if the tool has been adapted from the mentioned MDAT tool.

The SIMEDID tool has been adapted from the MDAT tool and other instruments, as reported by Alonso et al. 2022, which provides more details on the process of adaptation and validation.

How was this addressed in the manuscript?
We described the process of adapting the instrument in more detail, summarizing Alonso et al. 2022.

2. It is not clear if the assessment tool requires direct observation or is based on parent
report, or a combination of both.

The SIMEDID tool combines direct observation of the child's behavior by educational agents and reports from individuals who have spent enough time with the child. That person must have observed the desired behavior. For example, in INAIPI’s case, the evaluator is the person in contact with the child, the educational agent, the community agent, and/or the parent.

How was this addressed in the manuscript?
We included the following statement in the methods section: “The assessment tool requires an observation of the child’s behavior, either by the evaluator, parent, or the person in charge of the child at a given time. In case the child does not show the expected behavior at the time of testing, the informant’s report is considered a valid response as long as they have witnessed the expected behavior from the child.”

3. It is not clear if the assessment tool requires a kit to support the assessment or not.

Yes, there is a kit available to support the assessment with the SIMEDID tool.

How was this addressed in the manuscript?
We included the following statement in the methods section: “The INAIPI team created a kit to support the assessment tool (which includes props, pictures, and everyday objects). Materials are easily found in the Dominican Republic as a strategy for upscaling data collection.”

4. It is not entirely clear if there are any pictures or materials that have been created to support the assessment.

Yes, the SIMEDID tool includes pictures and materials to support the assessment.

How was this addressed in the manuscript?
See comment #3.

5. It is not entirely clear if the assessment tool has been piloted before this use.

The SIMEDID assessment tool underwent an initial pilot study involving 100 children before its use in this study. The pilot study, which is unpublished, provided valuable insights into the tool's performance and informed the validation process.

How was this addressed in the manuscript?
The following statement has been included “The pilot study included 100 children aged 45 days to 5 years who were receiving INAIPI services. The study confirmed the platform’s offline functionality, speed, and user-friendly interface, along with the willingness of educators and animators to participate. In addition, the instrument showed good internal consistency.”

6. It might be helpful to have a list of items and what was and was not changed.

The extended data now include a list of the items in the SIMEDID tool and details on what was and was not changed during the adaptation process.

How was this addressed in the manuscript?
The extended data are referenced in the manuscript.

7. It might be helpful to use the COSMIN framework to clarify the various validation procedures to ensure they were done as would be recommended.

A new supplementary table that follows the COSMIN framework has been included in the extended data. This table will serve as a live document to track data collection activities and ensure that the instrument continues to improve over time.

How was this addressed in the manuscript?
The extended data are referenced in the manuscript.

8. Do the items have to be passed in succession? Can they move from one area to another within the app?

Items must be completed in sequence as they are organized based on increasing difficulty and follow an algorithm that assesses children's development from minimum to maximum levels.
Evaluators can easily switch between different developmental areas by clicking on tabs with corresponding names. However, to progress to the next item, the current one must be answered. Additionally, to finish the test and submit results, all age-specific items must be filled out. Any missing responses will trigger an alert from the platform.

How was this addressed in the manuscript?
We included the following statement in the instruments section:
“Each subtest comprises 33 items, except for language development, which has 34 items; all are arranged in ascending difficulty levels. The initial presentation of the first item is age-dependent. Once the first item on each dimension is completed, the instrument presents items in reverse until the participant successfully completes three items. Subsequently, the tool proceeds forward until the participant misses three items. This approach enables evaluators to ascertain the child's current developmental state, identifying potential delays (backward presentation) or advanced development (forward presentation). Items must be completed in sequence as they are organized based on increasing difficulty and follow an algorithm that assesses children's development from minimum to maximum levels. Evaluators can easily switch between different developmental areas by clicking on tabs with corresponding names.
The study's primary aim was to determine the age range at which each item is accomplished, utilizing the Malawi MDAT norms for the initial trial. Subsequent data collection will adopt the sequence based on the findings from the present validation study.”

9. Can the authors be clearer about how they obtained the sample of children for the study and if these children are representative of the general population in the Dominican Republic?

The children in this study attended CAIPIs and CAFIs in the metropolitan area. While this may not be a fully representative sample of the general population of children in the Dominican Republic, we are currently obtaining norms from a representative sample of INAIPI beneficiaries from all regions. Future data collection activities will also aim to include a sample of children outside of INAIPI to ensure greater representativeness.

How was this addressed in the manuscript?
We have included the following statement in the discussion section: “One limitation of this study is that the sample may not accurately reflect all children in the Dominican Republic since it comprises only those involved in INAIPI programs within the metropolitan region. During the study period, 77,000 children enrolled in INAIPI from the metropolitan region accounted for 1.23 percent of the total population of children ages 0 to 5. The limited sample size was influenced by the challenges posed by the COVID-19 pandemic, including restrictions and prevention measures.”

10. Were other demographic factors taken into account, such as birth weight, adversities, sociodemographic status, health status, frequency of admissions to hospital, nutritional status, etc.?

For the the current validation study, no other demographic factors were considered since we were looking for a progression in item order. However, future data collection activities, which will involve a larger sample of children from INAIPI, will take into account variables such as birth weight, prematurity, and other health factors.

How was this addressed in the manuscript?
The following statement has been included in the method section. “No other demographic factors were considered for this analysis.” In addition, the following statement was included in the discussion section: “Another limitation is that the study did not account for other relevant factors in childhood development such as low birth weight and other health factors. However, future data collection activities, which will involve a larger sample of children from INAIPI, will consider variables such as birth weight, prematurity, and other health factors to determine cutoff points that better identify children at risk for developmental delay and to further understand the sensitivity of SIMEDID.”

11. The authors describe the use of Cronbach’s alpha for internal reliability and “split half parallel reliability”, but it is unclear if any reliability was done at the time of training and whether any quality control was done between examiners?

During the pilot (unpublished), Cronbach’s alpha and split-half parallel reliability were conducted to ensure the reliability of the instrument. Additionally, we conducted a quality control analysis to check for consistency between examiners. This involved comparing the mean scores of participants per evaluator.

How was this addressed in the manuscript?
The following statement has been included in the method section: “To guarantee uniformity in assessments among evaluators, we conducted an analysis of variance (ANOVA), comparing the mean scores of participants within each age group per evaluator. The results revealed no significant differences
between evaluators within each age group across all dimensions.”

12. What did the 6-hour session for training consist of? Could it be clearer who did this and what is needed for the future? Were there online sessions? In-person sessions? Any materials that were created? Certification of assessors?

The 6-hour training session, conducted by INAIPI personnel, encompassed practical and theoretical components based on a module designed by experts in training and child development. The training, mainly conducted face-to-face, included virtual follow-up by the Child Development Evaluation Division. While there was no initial certification process for assessors, facilitators of the training received certification as official SIMEDID facilitators. The module covered essential topics such as the concept of child development, the Dominican Child Development Measurement System (SIMEDID), its structure, and dimensions. It explained who applies SIMEDID in INAIPI's service modalities and provided guidelines on application rules. Evaluators learned the assessment process in homes, including guidance for parents, the periodicity of the system, and the specifications of the tool. The practical section involved the hands-on use of evaluation kit materials in simulated assessments, emphasizing adherence to instructions for standardized data. The training concluded with a presentation on the app's functionality, using visual aids and videos to reinforce the covered content.

How was this addressed in the manuscript?
The following statement has been included in the method section: “A total of 40 evaluators received a six-hour training session and conducted two practice evaluations. The training session comprised both theoretical and practical components. The training, facilitated by experts in training and child development, covered essential topics such as the concept of child development and SIMEDID. It included virtual follow-ups by the Child Development Evaluation Division and certification for facilitators. The module provided insights into SIMEDID's structure, dimensions, and its application in INAIPI's service modalities. Evaluators learned about assessment processes in homes, including guidance for parents, system periodicity, and tool specifications. The practical section involved hands-on use of evaluation kit materials in simulated assessments, emphasizing adherence to standardized data collection. The training concluded with a presentation on the SIMEDID app's functionality, utilizing visual aids and videos to reinforce the covered content. At the end of the training, evaluators were certified as official SIMEDID evaluators.”

13. Furthermore, could the authors provide more information about whether children were assessed with a disability, and if not, how did they decide to exclude them? Can the present tool go down further to assess a child who is not performing? How did the team decide if they had a child that they felt was below the norm?

For this specific data collection activity, we did not exclude children with disabilities because such data were not available. However, for the normalization study, in which we are setting the cutoff points to detect developmental delays, we have a mechanism to exclude them from the analysis, not from the evaluation. This way, we will be able to determine SIMEDID's sensitivity to disability. The tool is designed to assess children from ages 0 to 5, and there are items that can be used to assess children who are not performing at their age level.

How was this addressed in the manuscript?
The following statement has been included in the discussion “Another limitation is that the study did not account for other relevant factors in childhood development, such as low birth weight and other health factors or disabilities. However, future data collection activities, which will involve a larger sample of children from INAIPI, will consider variables such as birth weight, prematurity, and other health factors to determine cutoff points that better identify children at risk for developmental delays and to further understand the sensitivity of SIMEDID.”

14. The authors describe that age did not predict certain items. Is this because these items are not acceptable or work in the setting, or is it because there were not enough children to assess these items? It seems to be the latter.

We did not have enough children to assess these items, and we are currently working on obtaining a sample of children within the age range to evaluate these items. We are meeting with other governmental institutions that work with infants younger than 45 days to expand the age range of the study.

How was this addressed in the manuscript?
The following statement has been included in the discussion section: “We could not validate initial items of SIMEDID given that INAIPI does not provide services to infants younger than 45 days, given that mothers are on maternity leave during that period. However, the research team is meeting other governmental institutions that work with infants that young to expand the age range of the study. “

15. It is not clear that the authors have definitively shown that the tool can sensitively identify changes. Could they make that clearer and that they have demonstrated its use in another population who are known to have a problem through doing a Gold Standard assessment of these children?

We have yet to demonstrate the tool's ability to sensitively identify changes, as this is the first large-scale data collection using this instrument. Our current focus is to establish the reliability and validity of the SIMEDID tool. However, we are designing a study to compare the SIMEDID tool with another widely used instrument by UNICEF that is applied during household surveys.

How was this addressed in the manuscript?
The following statement has been included in the discussion section: “We have yet to demonstrate the tool's ability to sensitively identify changes, as this is the first large-scale data collection using this instrument. However, since SIMEDID is already part of the monitoring strategy, with constant data collection, it would be relatively simple to demonstrate its sensitivity to changes.”

16. Could the authors provide a little more information on what they plan to do with the information from this study? At the moment, it seems like it goes into a data collection system, but what and how will they create “cut-offs” decided for referral? These next steps are very important and could be provided more clearly for the reader.

The information collected from this study will be used to establish cut-off scores for referral to early intervention services. We will use statistical methods to determine the optimal cut-offs.  A subsequent standardization study allowed us to establish correction parameters and normative scores, with which it will be possible to establish when alerts occur in the development of a child.

How was this addressed in the manuscript?
This was addressed in the Discussion section:
“These results will allow the pertinent institutions of the Dominican Republic to implement and report more accurate early childhood development indicators. Still, they will also contribute to creating a robust monitoring system with a high-quality data collection process that allows evidence-based and timely decision-making. Furthermore, such a system will contribute to generating longitudinal data that establish the association of childhood development with sociodemographic and psychosocial variables and determine the impact of initiatives and interventions (Richter et al., 2017), which is not sufficiently evaluated in most countries (Daelmans et al., 2017).”

View more View less

Competing Interests

N/A

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] Alonso MA, Valdez ME, Jiménez A, et al.: Creación del Sistema de Medición de Desarrollo Infantil Dominicano (SIMEDID). Ciencia y Educación. 2022; 6(3): 71–77. Publisher Full Text

[2] Daelmans B, Darmstadt GL, Lombardi J, et al.: Early childhood development: The foundation of sustainable development. Lancet. 2017; 389(10064): 9–11. Publisher Full Text

[3] Inter-American Dialogue: Medición del desarrollo infantil en América: Desafíos para la medición y respuestas de política. Washington DC: Inter-American Dialogue; 2020.

[4] Gladstone M, Lancaster GA, Jones AP, et al.: Can Western developmental screening tools be modified for use in a rural Malawian setting? Arch. Dis. Child. 2008; 93(1): 23–29. PubMed Abstract | Publisher Full Text

[5] Gladstone M, Lancaster GA, Umar E, et al.: The Malawi developmental assessment tool (MDAT): The creation, validation, and reliability of a tool to assess child development in rural African settings. PLoS Med. 2010; 7(5): e1000273. PubMed Abstract | Publisher Full Text | Free Full Text

[6] Loizillon A, Petrowski N, Britto P, et al.: Development of the Early Childhood Development Index in MICS surveys. MICS Methodological Papers, No. 6.2017.

[7] Lokuketagoda BUWP, Thalagala N, Fonseka P, et al.: Early development standards for children aged 2 to 12 months in a low-income setting. SAGE Open. 2016; 6(4): 215824401667312. Publisher Full Text

[8] Molina Achécar M, Polanco JJ: Encuesta por Conglomerados de Indicadores Múltiples (MICS-2000).2001.

[9] ONE, & UNICEF: Encuesta nacional de hogares de propósitos múltiples - Encuesta de Indicadores Múltiples por Conglomerados 2014, Informe final.2016.

[10] ONE, & UNICEF: ENHOGAR-MICS Encuesta de Indicadores Múltiples por Conglomerados 2019, Informe de resultados de la encuesta.2021.

[11] Richter LM, Daelmans B, Lombardi J, et al.: Investing in the foundation of sustainable development: Pathways to scale up for early childhood development. Lancet. Elsevier B.V; 2017; 389(10064): 103–118. PubMed Abstract | Publisher Full Text | Free Full Text

[12] Sabanathan S, Wills B, Gladstone M: Child development assessment tools in low-income and middle-income countries: How can we use them more appropriately? Arch. Dis. Child. 2015; 100(5): 482–488. PubMed Abstract | Publisher Full Text | Free Full Text

[13] Sánchez-Vincitore LV, Castro A: The role of sociodemographic and psychosocial variables in early childhood development: A secondary data analysis of the 2014 and 2019 Multiple Indicator Cluster Surveys in the Dominican Republic. PLOS Global Public Health. 2022; 2(7): e0000465. Publisher Full Text

[14] Sánchez-Vincitore LV, Mencía-Ripley A, Veras-Díaz C, et al.: Efectos de una intervención de alfabetización en las habilidades lectoras de estudiantes de primaria: Proyecto USAID Leer. Revista Caribeña de Investigación Educativa. 2020; 4(2): 78–95. Publisher Full Text

[15] Sánchez-Vincitore LV, Schaettle P, Castro A: Validation of the Malawi Developmental Assessment Tool for children in the Dominican Republic: Preliminary results. PLoS One. 2019; 14(8): e0221162. PubMed Abstract | Publisher Full Text | Free Full Text

[16] Sánchez-Vincitore LV, Veras C, Mencía-Ripley A, et al.: Reading comprehension precursors: Evidence of the simple view of reading in a transparent orthography. Front. Educ. 2022; 7: 914414. Publisher Full Text

[17] Sánchez-Vincitore LV, Alonso Pellerano MA, Valdez ME, et al.: Questionnaire and item presentation for SIMEDID validation study. Dataset. 2023a, February 21. Publisher Full Text

[18] Sánchez-Vincitore LV, Alonso Pellerano MA, Valdez ME, et al.: Database for Validation of the Dominican System for Measuring Early Childhood Development. Dataset. 2023b, February 7. Publisher Full Text

[19] Sireci SG, Sukin T: Test validity.Geisinger KF, Bracken BA, Carlson JF, et al., editors. APA handbook of testing and assessment in psychology, Vol. 1. Test theory and testing and assessment in industrial and organizational psychology. American Psychological Association; 2013; pp. 61–84. Publisher Full Text

[20] Suchdev PS, Boivin MJ, Forsyth BW, et al.: Assessment of neurodevelopment, nutrition, and inflammation from fetal life to adolescence in low-resource settings. Pediatrics. 2017; 139: S23–S37. PubMed Abstract | Publisher Full Text

[21] UNESCO Institute of Statistics (UIS): Proportion of children aged 24-59 months who are developmentally on track in health, learning and psychosocial well-being, by sex.n.d.Reference Source

[22] United Nations: Transforming our world: The 2030 agenda for sustainable development (A/RES/70/1).2015. Reference Source

Validation of the Dominican system for measuring early childhood development

Abstract

Keywords

Introduction

Methods

Ethical statement

Study design

Setting

Participants

Instruments

Table 1. Study variables.

Procedure

Statistical methods

(1)

Results

Psychometric properties of SIMEDID (first aim)

Table 2. Descriptive statistics for total scores by development area and age group.

Table 3. Reliability of the instrument.

Adjustment of items order presentation (second aim) and age references per item (third aim)

Figure 1. Age reference values for gross motor items.

Figure 2. Age reference values for fine motor items.

Figure 3. Age reference values for language development items.

Figure 4. Age reference values for socioemotional development items.

Discussion

Conclusion

Data availability

Underlying data

Extended data

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated