Validation of the Dominican system for measuring early childhood development

Introduction

The Dominican Republic confirmed its commitment to achieving the United Nation’s sustainable development goal 4.2 for 2030 to guarantee that children are ready for primary education by offering quality early childhood services, including care and early education (United Nations, 2015). One of this goal’s indicators is “[the] proportion of children aged 24-59 months who are developmentally on track in health, learning, and psychosocial well-being, by sex” (UNESCO UIS, n.d., para. 1). Although this indicator is conceptually straightforward, and numerous efforts have been conducted to establish a global methodology for measuring it, some challenges associated with determining cut-off points for early childhood development remain (Daelmans et al., 2017; Richter et al., 2017).

First, high-income countries’ screening tools might translate poorly to low- and middle-income countries. This poses the risk of either underestimating or overestimating childhood development, which in turn precludes making accurate, evidence-based decisions regarding early childhood interventions and resource allocation (Gladstone et al., 2008; Sabanathan et al., 2015). The second challenge is the need for more funding for monitoring systems in low- and middle-income countries, which translates into needing more sufficiently qualified personnel to conduct periodic childhood development screenings (Lokuketagoda et al., 2016). And third, there is a critical need to obtain a large pool of data from developing children to identify those at risk for developmental delay (Lokuketagoda et al., 2016), which is particularly challenging in low- and middle-income countries (Richter et al., 2017).

In Latin America and the Caribbean, childhood development measurement has attracted attention, evidenced by the creation of the Regional Network for Measuring Childhood Development (REMDI) (Interamerican Dialogue, 2020). This international network of specialists is dedicated to promoting national measurements of childhood development to obtain data for decision-making and comparison between and within countries. Since the year 2000, the Dominican Republic has collected data on childhood development by participating in a series of Multiple Indicator Cluster Surveys (MICS), a household survey methodology designed by UNICEF that analyzes the situation of women and children across the world. The instrument collects data on children’s health, education, protection, and environment (such as sanitation), among other variables. The Dominican Republic participated in MICS2 (Molina Achécar and Polanco, 2001), MICS5 (ONE and UNICEF, 2016), and MICS6 (ONE and UNICEF, 2021) survey rounds with an evolving early childhood development measurement (Loizillon et al., 2017).

The latest data from 2019 reported that 87.1% of Dominican children meet the minimum development indicators. Data generated by MICS have been useful for guiding the advocacy and system-strengthening plans of early childhood development and children’s rights institutions, including sustainable investments in the multi-year governmental planning for 2020-2024. In addition, the MICS data have been used to create predictive models that quantify the impact of multiple sociodemographic and psychosocial factors in childhood development (Sánchez-Vincitore and Castro, 2022).

Household surveys provide useful information on general trends in childhood development, but they are not comprehensive enough to assess development in its various dimensions or sensitive enough to generate alerts to detect developmental delays. These limitations highlight the need for specific child development screening tools that, although quick and cost-effective in the application, have adequate psychometric properties.

Many private, informal, and some public initiatives have been conducted in the Dominican Republic to provide early childhood services. However, until 2019 there were no standardized instruments to measure the impact of such efforts. In 2019, Sánchez-Vincitore et al. (2019) initiated the validation of the Dominican adaptation of the Malawi Developmental Assessment Tool (MDAT) (Gladstone et al., 2008, 2010). The MDAT is a childhood development screener in which an evaluator observes the behavior of a child in four different domains: gross motor, fine motor, language, and socio-emotional development. One of the advantages of MDAT is that it optimizes test application time by only presenting items that correspond to the child’s expected evolutionary stage according to the child’s age. Therefore, providing a more precise item order is crucial to prevent bias in obtaining total scores. The adaptation of the MDAT to the Dominican Republic (MDAT-DR) (Sánchez-Vincitore et al., 2019) presented good psychometric properties. However, some limitations had to be considered before upscaling it as a national surveillance tool. First, as a preliminary pilot implementation within the academic context, research assistants with vast experience and training in data collection administered the instrument, which is unlikely in naturalistic environments. Second, the MDAT-DR adaptation had a small sample size (N = 42), meaning there was no national representation. Therefore, age-standardized norms were not obtained for each item, threatening item presentation order. Finally, the study evaluated children up to 24 months, limiting the age range for which data were available.

To overcome these limitations, the National Institute for Early Childhood Comprehensive Care (INAIPI, for its acronym in Spanish), established as a national state institution in 2013 to ensure quality comprehensive care services to children aged 0 to five and their families, collaborated with the Universidad Iberoamericana (UNIBE) and the United Nations Children’s Fund (UNICEF) to develop the Dominican System for Measuring Early Childhood Development (SIMEDID, for its acronym in Spanish). To create SIMEDID, the team re-analyzed the content and structure of the MDAT-DR (Sánchez-Vincitore et al., 2019) and integrated other items from international and national instruments (Alonso et al., 2022). The research team obtained an expert panel’s consensus before collecting data to guarantee the instrument’s construct definition (Sireci and Sukin, 2013). The expert panel consisted of a group of professionals who represent Dominican institutions that provide early childhood services, including the Early Childhood Education Department and the Special Education Department from the Ministry of Education; the Ministry of Health; the National Health Services (SNS for its Spanish acronym); the National Council for Childhood and Adolescence (CONANI for its Spanish acronym); and the National Council for Disabilities (CONADIS for its Spanish acronym). The experts had the opportunity to revise each item and their definitions.

The team adopted all four of MDAT-DR’s dimensions—including fine motor skills, gross motor skills, social and emotional development, and language development—updated them for relevance, created a progressive item list for each developmental dimension through expert input, and revised the instrument twice based on surveys among educators and facilitators for clarity. Next, the team conducted an initial pilot of SIMEDID aimed to assess its application conditions, digital platform functionality, internal consistency, and user experience (INAIPI, 2020; Sánchez-Vincitore, 2020). The pilot study included 100 children aged 45 days to 5 years who were receiving INAIPI services. The study confirmed the platform’s offline functionality, speed, and user-friendly interface, along with the willingness of educators and animators to participate. In addition, the instrument showed good internal consistency.

SIMEDID is an electronic platform that hosts an early childhood development screener. This platform allows data collection through mobile devices and connects to INAIPI’s servers as part of its monitoring and evaluation system. The mobile application extracts sociodemographic information from the server, configuring individual evaluations for each child’s age. As a result, INAIPI personnel already in the field can administer the early childhood development screener time-efficiently with little training. Once the assessment is over and the device connects to the internet, the data sync to the server—reducing the risk of losing the data.

We conducted this study to validate SIMEDID, with three aims: (1) To determine the psychometric properties of SIMEDID; (2) To adjust the sequence of item presentation according to developmental milestones obtained from data from a large sample; (3) To provide age-standardized norms for each item.

Methods

Results

Adjustment of items order presentation (second aim) and age references per item (third aim)

To complete the second and third aims, we conducted a logistic regression analysis on each item with independent variable age and dependent variable item success. The results show a good fit (p < .05), except for the first item in the gross motor area and three of the first four items in the language development area.

The item presentation order was determined by sorting the age at which each item had a .9 probability of success from the results of logistic regression analyses on each item. To describe the expected evolution of item success probability across age, we also determined the .75, .50, and .25 probability of success. We thus provided the range amplitude for each item’s success predicted by age. Figures 1 to 4 contain a visual representation of sorted items and corresponding probabilities of success. Note that results are presented in two scales for the X axis: for the first two years, the X axis corresponds to age in months, and for the following age ranges, in years.

Figure 1. Age reference values for gross motor items.

Notes: MG1 Hold their head when carried; MG2 Lift their chin off the floor; MG3 From the prone position, they can lift their head to 90 degrees; MG4 Support head when lifted by hands; MG5 Flip over; MG6 Raise head, shoulders, and chest from a prone position; MG7 Start creeping; MG8 Start crawling position; MG9 Stand up with support; MG10 Sits up unassisted; MG11 Crawl with displacement alternating knees and hands; MG12 Take steps with help; MG13 Stand up unassisted; MG14 Walk without help; MG15 They crouch and stand up; MG16 Walk well with cross-scroll; MG17 Run, they may fall; MG18 Throw ball; MG19 Kick the ball; MG20 Run showing coordination in their movements; MG21 Run well, stops and start again without falling; MG22 Jump with feet together; MG23 Jump moving with both feet; MG24 Stand on one foot for 3 seconds; MG25 Stand on tiptoe with both feet; MG26 Walk on tiptoe; MG27 Walk in a straight line keeping balance; MG28 Jump on one foot without support; MG29 Can catch a ball with both hands; MG30 Bounce and catch the ball; MG31 Stand on one foot for 5 seconds; MG32 Jump moving with one foot; MG33 Jump alternating feet. The turquoise range signifies a probability of success between .25 and .50, the red range indicates a probability of success between .50 and .75, and the yellow range denotes a probability of success between .75 and .90. The y axis represents the distribution of the probability of success as explained by the color ranges.

Figure 2. Age reference values for fine motor items.

Notes: MF1 Palmar grasp reflex; MF2 Stare the midline; MF3 Visually focuses on an object and tracks it horizontally; MF4 Keep hands open when awake; MF5 Hold an object in hand; MF6 Show interest in putting an object in their mouth; MF7 Visually focus on an object and follow it from top to bottom; MF8 Grasp large objects voluntarily; MF9 Hold an object in each hand; MF10 Pass an object from one hand to another; MF11 Pick up small objects as if their fingers were a rake; MF12 Find the object under a blanket; MF13 Put and take out objects from the container; MF14 Grasp with thumb and forefinger (tweezers); MF15 Pick up a spoon and brings it to their mouth; MF16 Scribble; MF17 Push a car; MF18 Turn pages of a book; MF19 Make a tower of two cubes; MF20 Put nails on a board; MF21 Make a tower of six cubes; MF22 Make a ball of paper; MF23 Tear paper with both hands; MF24 Make shapes with putty; MF25 Rotate hand to unscrew; MF26 String; MF27 Copy a horizontal and vertical line; MF28 Copy a circle; MF29 Copy a cross; MF30 Know how to button and unbutton; MF31 Color without leaving the outline of the drawing; MF32 Draw a human figure; MF33 Cut paper with scissors. The turquoise range signifies a probability of success between .25 and .50, the red range indicates a probability of success between .50 and .75, and the yellow range denotes a probability of success between .75 and .90. The y axis represents the distribution of the probability of success as explained by the color ranges.

Figure 3. Age reference values for language development items.

Notes: DL1 Calm down when speaking to them; DL2 Startle or jump in response to sounds; DL3 Cry to express needs; DL4 They laugh; DL5 Make sounds with the throat; DL6 Turn their head when they search for a sound; DL7 React when called by their name; DL8 Pronounce syllables like Ma, Pa, Ba, Ta; DL9 Point with their finger when they want something; DL10 Repeat the same syllable twice “Dada, Mama, Mimi, Tata, Papa, Yaya, Baba”; DL11 Understand the meaning of the word no; DL12 Follow one-step commands; DL13 Follow two-step instructions; DL14 Answer with yes or no; DL15 Pronounce their first words with communicative intention; DL16 Recognize at least 6 objects; DL17 Point to 5 parts of their body; DL18 Use a two-word phrase; DL19 Say 6 words; DL20 Say their name; DL21 Know the use of three or more objects; DL22 Can identify 10 objects by name; DL23 Pronounce sentences of three words; DL24 Use more than 15 words; DL25 Use long sentences; DL26 Know the qualities or characteristics of an object; DL27 Pronounce the sounds of words correctly; DL28 Describe the drawing; DL29 Name at least three things in a category; DL30 Recognize opposites; DL31 Can count up to 5 or more objects; DL32 Answer two comprehension questions; DL33 Compare objects; DL34 Tell a story from a sequence of images. The turquoise range signifies a probability of success between .25 and .50, the red range indicates a probability of success between .50 and .75, and the yellow range denotes a probability of success between .75 and .90. The y axis represents the distribution of the probability of success as explained by the color ranges.

Figure 4. Age reference values for socioemotional development items.

Notes: DS1 Calm down with family members or caregivers; DS2 Smile spontaneously; DS3 Smile in response to a person; DS4 Recognize the voice of the main caregiver; DS5 Make eye contact; DS6 Touch the examiner's hands; DS7 They are aware of their hands (body); DS8 Try to hold a cup when being fed; DS9 Respond to a conversation; DS10 Raise their arms or indicate that they want to be carried; DS11 Laugh out loud; DS12 Explore their face when they are in front of the mirror; DS13 Show interest or intention to feed themselves; DS14 Look for continuing the game; DS15 Explore the environment; DS16 Participate in games; DS17 Wave or verbally greet; DS18 Express their satisfaction when they achieve something; DS19 Take a glass without spilling; DS20 Imitate adult actions; DS21 Recognize their belongings; DS22 Express interest in playing with other children; DS23 Symbolic game; DS24 Refer to themselves as “I”; DS25 Say the names of the people with whom they live; DS26 They urinate or defecate independently without dirtying their clothes; DS27 Indicate in some way that they need to urinate or defecate; DS28 Identify basic emotions in images; DS29 Come up with games; DS30 Share their belongings; DS31 Recognize basic emotions in themselves and express them verbally; DS32 Recognize and express basic emotions in others; DS33 Participate in games respecting rules and turns. The turquoise range signifies a probability of success between .25 and .50, the red range indicates a probability of success between .50 and .75, and the yellow range denotes a probability of success between .75 and .90. The y axis represents the distribution of the probability of success as explained by the color ranges.

Discussion

This paper collects evidence for the validity of SIMEDID, an electronic early childhood development screening tool adapted to the Dominican context conducted by INAIPI’s personnel through an electronic application. Regarding the first aim, we found that the instrument has adequate psychometric properties: the instrument’s subscales showed high internal consistency scores, evidencing excellent reliability. Furthermore, total scores for each sub-scale increased progressively across age, which evidenced alignment with standards already provided by an expert panel (Alonso et al., 2022) and criterion validity with a previous version of this instrument on a small sample size (Sánchez-Vincitore et al., 2019).

Regarding the second aim, we found that age predicted item success in most items, which supports similar results in children from Malawi Gladstone et al. (2008, 2010). However, age did not predict four initial items from the gross motor and language development subscales. We attribute these null findings to the fact that our study did not include an acceptable sample size for the age group for which these items were relevant. This is because INAIPI’s services to very young children were scarce at the time of this study, which will be considered for future studies. To comply with the third aim, age-standardized norms for each item were established, obtaining correspondences between ages and different probabilities of success for each item, which will allow comparing the achievement of participants with what is expected at their age.

Adapting this instrument to the Dominican context guarantees that cultural aspects of childrearing do not overshadow developmental scores (Suchdev et al., 2017) and that development is not under or overestimated. Having the instrument in an electronic platform solves two main challenges. First, personnel training is kept to the minimum since the platform guides the evaluator throughout the evaluation, presenting the items that only pertain to the child according to their age, with suggested videos and additional testing resources. Second, having SIMEDID connected to INAIPI servers and incorporating the data on national services provided by INAIPI creates a continuous stream that otherwise would be costly and logistically convolute data. This data stream will allow the development of new research agendas that include correlational modeling, intervention studies, and longitudinal studies to understand better the factors associated with childhood development in the Dominican Republic in a timely and cost-efficient way.

The study findings demonstrate that SIMEDID passes three elements of a checklist of critical methodological elements to consider when appraising a childhood development assessment tool: (1) the instrument measures domains affected by the risk factor or intervention; (2) reliability and validity of the instrument in the population of interest; (3) sensitivity of the instrument to identify changes; (4) logistics and methodology is suitable for evaluating the outcome; and (5) consideration of control group (Sabanathan et al., 2015). SIMEDID passes the first two elements from this checklist, as it measures specific domains of early childhood development previously identified as risk factors in the Dominican Republic, such as the sociodemographic and psychosocial factors that predict childhood development (Sánchez-Vincitore and Castro, 2022) and low levels of oral language comprehension in school-aged children that should be addressed during early childhood development before children enter primary school (Sánchez-Vincitore et al., 2020, 2022) among other risk factors. In addition, it passes the fourth element, given that the logistic and methodology was specifically designed to assess the outcome in the Dominican context. Future studies will address the third and fifth items when SIMEDID is used as a monitoring tool on a population basis.

This study has some limitations that should be considered before its interpretation. Children from the sample for which these age standards were obtained received services at INAIPI. Socio-economic vulnerability is one of the main criteria for receiving such services. This means that the sample may not accurately reflect all children in the Dominican Republic since it comprises only those involved in INAIPI programs within the metropolitan region. During the study period, 77,000 children enrolled in INAIPI from the metropolitan region accounted for 1.23 percent of the total population of children ages 0 to 5. The limited sample size was influenced by the challenges posed by the COVID-19 pandemic, including restrictions and prevention measures. Future studies should consider the whole socio-economic position spectrum to obtain national norms. The data generated in the Dominican context using SIMEDID has limited comparability to data from other countries.

Another important limitation is that the experience of creating this platform in the Dominican Republic was cost-effective due to the already existing infrastructure within INAIPI, which should be considered when transferring it to other countries. The institution is the national administrator of early childhood services, which gives them access to the population of interest and trained personnel already working with children. In addition, INAIPI has the Division of Early Childhood Development Measurement, with dedicated personnel to designing, creating, supervising, and training the personnel in childhood development measurement. Also, INAIPI has a dedicated Information and Communications Technology Department, which developed the online platform and made it synchronized with SIGEPI, the national database for managing data from early childhood services. Further studies should conduct a cost-per-user analysis to evaluate its efficiency.

Finally, the study did not account for other relevant factors in childhood development, such as low birth weight and other health factors or disabilities. However, future data collection activities, which will involve a larger sample of children from INAIPI, will consider variables such as birth weight, prematurity, and other health factors to determine cutoff points that better identify children at risk for developmental delays and to further understand the sensitivity of SIMEDID.

Additionally, we could not validate the initial items of SIMEDID given that INAIPI does not provide services to infants younger than 45 days, given that mothers are on maternity leave during that period. However, the research team is meeting other governmental institutions that work with infants that young to expand the age range of the study.

We have yet to demonstrate the tool’s ability to sensitively identify changes, as this is the first large-scale data collection using this instrument. However, since SIMEDID is already part of the monitoring strategy, with constant data collection, it would be relatively simple to demonstrate its sensitivity to changes.

Even with these limitations, these results will allow the pertinent institutions of the Dominican Republic to implement and report more accurate early childhood development indicators. They will also contribute to creating a robust monitoring system with a high-quality data collection process that allows evidence-based and timely decision-making. Furthermore, such a system will contribute to generating longitudinal data that can establish the association between childhood development and sociodemographic and psychosocial variables and determine the impact of initiatives and interventions (Richter et al., 2017), which is not sufficiently evaluated in most countries (Daelmans et al., 2017).

In the future, SIMEDID will identify children at risk for developmental delays using data from a standardization study which will incorporate the item presentation order determined by the current validation study. Ensuring the appropriate order will narrow standard deviations, enabling the determination of developmental cut-off points at -1 and -2 standard deviations for different levels of developmental delay risk.

Although SIMEDID was created to be integrated into the services provided by INAIPI, and the instrument so far has only been administered to INAIPI beneficiaries, efforts to make a paper version of SIMEDID are on the way under the name of TADID (Tamizaje de Desarrollo Infantil Dominicano). This will allow other institutions, clinicians, schools, and pediatricians to use this validated tool at no cost.

Conclusion

This study provides compelling evidence for the validity and utility of SIMEDID, an electronic early childhood development screening tool adapted to the Dominican context. Demonstrating robust psychometric properties and high internal consistency, SIMEDID aligns with expert panel standards and exhibits criterion validity. Age-standardized norms enhance its precision, allowing nuanced assessments of developmental progress. The cultural adaptation addresses potential biases in childrearing practices. Future considerations include broader accessibility through the TADID paper version and the tool’s potential contribution to generating longitudinal data for evidence-based decision-making. Overall, SIMEDID emerges as a promising and versatile screening tool with implications for both program design and individual monitoring of childhood development in the Dominican Republic.

In conclusion, this study provides evidence for the validity of SIMEDID, an electronic early childhood development screening tool adapted to the Dominican context, with adequate psychometric properties and age-standardized norms for each item. Adapting this instrument to the Dominican context ensures that cultural aspects of childrearing do not overshadow developmental scores. While SIMEDID is a screening tool and not intended for diagnosis, it offers valuable insights for caregivers and stakeholders, both at a group level for program design and decision-making, as well as at the individual level to monitor each child’s progress.