F1000Research F1000Research 2046-1402 F1000 Research Limited London, UK 10.12688/f1000research.141458.1 Research Article Articles Predicting stunting in Rwanda using artificial neural networks: a demographic health survey 2020 analysis

[version 1; peer review: 3 approved with reservations]

 Ndagijimana Similien Conceptualization Data Curation Formal Analysis Funding Acquisition Investigation Methodology Project Administration Resources Software Supervision Validation Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0002-5636-3551 a 1 Kabano Ignace Visualization Writing – Original Draft Preparation 1 2 Masabo Emmanuel Supervision Validation Visualization Writing – Original Draft Preparation Writing – Review & Editing 1 3 Ntaganda Jean Marie Conceptualization Methodology Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0003-2464-2377 2 African Centre of Excellence in Data Science, Kigali, Kigali, Rwanda College of Business and Economics, University of Rwanda, Kigali, Kigali, Rwanda College of science and Technology, University f Rwanda, Kigali, Kigali, Rwanda similienn@gmail.com

No competing interests were disclosed.

 20 2 2024 2024 13 128 14 12 2023 Copyright: © 2024 Ndagijimana S et al. 2024 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Background

Stunting is a serious public health concern in Rwanda, affecting around 33.3% of children under the age of five in 2020. Several examples of research have employed machine learning algorithms to predict stunting in Rwanda; however, no study used artificial neural networks (ANNs), despite their strong capacity to predict stunting. The purpose of this study was to predict stunting in Rwanda using ANNs and the most recent Demographic and Health Survey (DHS) data from 2020.

 Methods

We used a multilayer perceptron (MLP) architecture to train and test the ANN model on a subset of the DHS dataset. The input variables for the model included child, parental and socio-demographic’s characteristics. The output variable was a binary indicator of stunting status (stunted vs. not stunted).

 Results

An overall accuracy of 72.0% on the test set was observed, with an area under the receiver operating characteristic curve (AUC-ROC) of 0.84, indicating the model’s good performance. Several factors appear as important contributors to the probability of stunting among the negative value aspects. First and foremost, the mother’s height is important, as a lower height suggests an increased risk of stunting in children. Positive value characteristics, on the other hand, emphasie elements that reduce the likelihood of stunting. The timing of the initiation of breastfeeding stands out as a crucial factor, showing that early breastfeeding initiation has been linked with a decreased risk of stunting.

 Conclusions

Our findings suggest that ANNs can be a useful tool for predicting stunting in Rwanda and identifying the most important associated factors for stunting. These insights can inform targeted interventions to reduce the burden of stunting in Rwanda and other low- and middle-income countries.

 Feature importance Artificial Neural Networks Stunting Children Rwanda. African Centre of Excellence in Data Science at the University of Rwanda and World Bank finance (ID:ESC91) We declare that this study was funded by the African Centre of Excellence in Data Science at the University of Rwanda, as well as World Bank finance University of Rwanda, as well as World Bank finance (ID: ESC 91). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Introduction

Stunting remains a significant public health issue worldwide, particularly in low-and middle-income countries. According to the latest estimates by the World Health Organisation (WHO), in 2020, around 149.2 million children under the age of five years (about 22% of all children in this age group) were affected by stunting globally, with the highest burden in low and middle-income countries. 1 The COVID-19 pandemic has exacerbated the situation, since disruptions in food systems and health services are likely leading to an increase in stunting rates. 2 Therefore, stunting is still a serious public health problem across the world, particularly in Africa. 2 In 2020, about 58 million children under the age of five were stunted in Africa, accounting for nearly 40% of all stunted children worldwide. The prevalence of stunting in these EAC countries highlights the importance of focused interventions to address the underlying causes of stunting, which include poverty, poor nutrition, and a lack of access to healthcare and sanitary services. 3

Artificial Neural Networks (ANNs) are a type of machine learning (ML) technique that has gained prominence in recent years due to its ability to learn and generalise from data, making them ideal for predictive modelling applications. 4 ANNs are a kind of deep learning, which is a technique that consists of training models with numerous layers of connected nodes to replicate human brain function. 4 ANNs stand out in significance when contrasted with other ML algorithms for a multitude of compelling reasons. In many circumstances, ANNs can simulate complicated, non-linear interactions between input and output variables, allowing for accurate predictions. 5 Linear models, such as linear regression, have limitations in capturing nonlinear correlations, but more complex models, such as decision trees or random forests, may overfit the data or be computationally costly, however, ANNs perform better. 6 ANNs are usually resistant to noisy or incomplete data, making them useful in real-world situations where data is frequently poor. 6 ANNs can automatically extract significant features from data, removing the need for manual feature engineering. This can save time and effort while modelling, especially with high-dimensional data. 7 ANNs are suited for big data applications because they can be scaled to accommodate massive datasets, hence, it is critical to apply them to Rwanda Demographic Health Survey (RDHS). 7 The RDHS is nationally representative research that collects data on a number of health indicators, including stunting, through house interviews. 8

They can also be parallelised over many processors, increasing computing efficiency. 9 ANNs may be used for transfer learning, which involves fine-tuning a pre-trained model for a new task with little data. This is especially important when data is scarce or expensive to collect. 9 Using an ANN to predict stunting offers various advantages, including increased accuracy and the capacity to identify significant predictors of stunting. ANNs can analyse vast volumes of data and detect patterns that typical statistical approaches may miss, allowing for more accurate stunting predictions. Furthermore, ANNs can identify major predictors of stunting, such as poverty, low maternal education, and a lack of access to sanitary facilities, allowing for focused interventions to address the core causes of stunting. 10 A few studies have been conducted in Rwanda using other machine learning technics like logistic regression, Supportive Vector Machine (SVM), Naive Bayes Random Forest (RF), XGBoost gradient model. 11

However, based on the existing knowledge there has been few researches in Rwanda that attempted to utilise ML to predict stunting like the study conducted by Similien et al. 11 ANNs have proven to be highly effective in predicting illnesses. However, Similien’s publication did not delve into the utilisation of ANNs for this purpose, despite their demonstrated effectiveness in prediction. Recognising this gap, a supplementary study was essential to explore the application of ANNs in predicting stunting in children, using data from the 2020 RDHS. Given the crucial importance of addressing the root causes of stunting for effective treatments and policymaking, the researcher chose to conduct this study on the application of ANNs in the specific context of stunting in Rwanda, using the same dataset as the aforementioned publication. 11 The remaining party of this study is organised as follow: Methods, Results, Discussion, Conclusion and recommendation.

 Methods

DHS is a large-scale household survey program that is carried out in low- and middle-income nations. DHS surveys are meant to collect high-quality data on health, demographic, and nutrition indicators to help policymakers and program administrators make better decisions. The surveys are normally conducted every five years and give information on a variety of areas including fertility, mother and child health, family planning, HIV/AIDS, nutrition, and gender-based violence. 12 The secondary data from the 2019-2020 RDHS were analysed in this study. The RDHS is a five-year quantitative, cross-sectional study-based national survey. The RDHS used a two-phase stratified sampling approach. In the first step, 500 clusters were chosen from a pool of 112 urban enumeration areas and 388 rural enumeration areas. In the second stage, homes were systematically sampled, involving the selection of a random subsample of 26 households within each cluster, resulting in a total of 13,000 surveyed households. This subsample specifically included 3,814 children under the age of five, from whom height and weight measurements were collected. 13

Quantitative variables

Explanatory variables

The explanatory factors for stunting that were associated with the characteristics of mothers, households, and children are summarised below ( Table 1). The selection of variables from the DHS was guided by UNICEF conceptual framework children nutrition and tailored to the specific context of Rwanda. 14

Explanatory factors are assessed and documented for use in the analysis.
Variables Description Categories
Variables related to a child’s characteristics
Baby’s age Age of the child in months 0: <6, 1: 6-11, 2: 12-23, 3: 24-35, 4: 36-47, 5: 48-59
Sex Sex of the child 0: female, 1: male
Size of a child Size of the child at birth 0: large, 1: average, 2: small
Birthweight The weight of the child at birth 0: ≥2.5 kg, 1: <2.5 kg
Breastfeeding start Time when the child starts breastfeeding 0: within the first hour, 1: 1-24 hours, 1-2: 30 days
Presence of diarrhoea The child had diarrhoea in the last 2 weeks 0: No, 1: Yes
Variables related to the child’s mother
Maternal age Age of the mother 0: less than 18 years, 1: Between 19-35 years, 2: greater than 35 years
Maternal education Education level of the mother 0: no education, 1: primary, 2: secondary or higher
Maternal anaemia Anaemia status the of mother 0: not anaemic, 1: anaemic
Marital status Mother’s marital status 0: single, 1: married, 2: separated
Maternal height Respondent’s height in centimetres 0: <160 cm, 1: ≥160 cm
Antenatal Number of antenatal visits during pregnancy 0: no antenatal care, 1: 1-4 antenatal care visits, 2: more than 5 antenatal care visits
Variables related to households
Residence Type of place of residence of the child 0: rural, 1: urban
Source of drinking water Source of drinking water in the household 0: unimproved, 1: improved
Toilet facilities Type of toilet facilities in the household 0: unimproved, 1: improved
Place of delivery Distribution of live births by place of delivery 0: other, 1: delivery at home, 2: delivery at health facility
Province Region 0: Kigali, 1: south, 2: east, 3: west, 4: north
Reading Frequency of reading newspapers or magasines 0: ever, 1: reading
Altitude Cluster altitude in meters 0: ≤2000 m, 1: >2000 m

DHS, Demographic and Health Surveys; WHO, World Health Organisation.

Outcome variable

The outcome variable in this study was stunting status, which was classified according to WHO criteria. The nutritional status of children was separated into two categories based on height for age z-scores, as follows: stunted if standard deviation SD < − 2 was less than the median, and not stunted otherwise. 13

Data preprocessing

Data preprocessing is the activity of preparing (cleaning and arranging) raw data so that it is understandable and useable for analysis. It consists of various stages, including data cleansing, data integration, data transformation, and data reduction. 15 In this study, data cleaning was characterised as the procedure for addressing missing or incomplete data within the dataset. The missing values were addressed through imputation using the K Nearest Neighbours (KNN) imputer, which, when contrasted with the Euclidean distance, might result in a reduction of data similarity. 16 Data transformation here used to describe the process of altering the format or structure of data to make it suitable for analysis, 15 where the researcher encoded categorical data with the map function before converting it to dummy (0 and 1) values with pandas (pd). Obtain dummies that treated variable categories individually, then use the Minimax scaler to normalise the numerical data, which ranges all data values between 0 and 1, code was generated in Python using the popular ML library scikit-learn. Moreover, the Synthetic Minority Over-Sampling Technique (SMOTE) was employed to tackle the class imbalance within the target variable. This technique involves oversampling the minority class by generating synthetic instances along the line segments that connect any or all of the k nearest neighbours within the minority class. 17 The software used during the data preprocessing and analysis was the python Google Collab. 18

Training dataset

A dataset is divided into three subsets: a training set, a validation set, and a test set. The training set is used to train the model, the validation set is used to fine-tune the model’s hyperparameters and avoid overfitting, and the test set is used to assess the model’s final performance on new data. Each subset’s size is determined by the amount of the dataset and the model’s complexity. The dataset, consisting of 3814 observations, was divided into 80% for training (3051 instances) and 20% for testing and validation (763 instances).

 Artificial neural networks model

The ANN was built using 23 inputs to predict stunting in Rwanda. After initialising the neural network, the model employed neurons as features in the input layer and two in the hidden layer. Two hidden layers were used in the ANN, a common choice for optimising performance, and they were tested individually to determine the ideal configuration for achieving the desired results in this proposed model. Because the goal of this study is to identify stunted newborns using training data, the rectifier activation function in the hidden layers and the sigmoid activation function in the output layer are used to set a range (0, 1) of a linear function in ANN. 19 80% and 20% of the data have been used as training and testing data respectively for a model that runs 100 epochs. Each epoch is seen as having one forward and one backward propagation. Finally, the most effective stochastic gradient descent optimiser parameter “Adam” is employed. The batch size is set to 32, which implies there are 10 occurrences in each epoch at any given moment. The loss (binary- crossentropy) function is used to classify the losses. With ANN, the best outcome is offered after computing the loss. 20 , 21

Model training and evaluation

To train the ANN model on the training set using the hyperparameters chosen. During the training phase, the model’s capacity to recognise complicated patterns in the data is constantly refined. We rigorously monitor two critical parameters throughout this training process: loss and accuracy. Loss measures how much our model’s predictions differ from the real values, whereas accuracy measures how frequently the model’s predictions match the actual outcomes. Early stopping is a strategy in which the model’s performance is evaluated on a distinct dataset called the validation set on a frequent basis during training. This collection is unique from the training data and is used to assess the model’s generalisation capabilities. Model evaluation, on the other hand, is used to assess the performance of the trained ANN model on the test set.

The primary metrics used for assessment were accuracy, precision, recall, and the area under the receiver operating characteristic curve (AUC-ROC). This metric quantifies the overall correctness of the model’s predictions by measuring the ratio of correctly predicted stunting to the total children. Precision assesses the accuracy of positive predictions of stunting made by the model. It is calculated as the ratio of true positive predictions to the sum of true positives and false positives. Recall, also known as sensitivity or true positive rate, evaluates the model’s ability to capture all relevant instances. It is calculated as the ratio of true positives to the sum of true positives and false negatives. The AUC-ROC provides a comprehensive evaluation of the model’s ability to discriminate between stunted and no stunted children. A higher AUC-ROC value indicates superior discrimination performance. The analyses were conducted using the TensorFlow and scikit-learn libraries in Python. 22

Features importance

Here are 10 steps that we used for feature selection as seen in Figure 1: Step 1 includes importing the dataset and picking the necessary columns for prediction. In this case, the dataset has 23 input features and the ‘stunting’ variable as the target variable. Step 2 the function LabelEncoder is used to encode categorical variables. LabelEncoder is a scikitlearn (RRID:SCR_002577) utility class that encodes category characteristics as numeric values; Step 3 entails separating the data into input (X) and target (Y) variables. The input characteristics are contained in the X variable, while the target variable is contained in the Y variable; Step 4 entails dividing the data into 80% for training (3051 instances) and 20% for testing and validation (763 instances). The ANN model is trained using the training set, and its performance is evaluated using the testing set; Step 5 using StandardScaler, standardise the input characteristics. The scikit-learn library’s StandardScaler utility class standardises features by eliminating the mean and scaling to unit variance. Step 6: Build an ANN model with two hidden layers and one output layer. The number of neurons in the input layer is equal to the 23 of input features; Step 7 specifically, in our stunting prediction model, we opted for the Adam optimiser and binary cross-entropy loss. The choice of the binary cross-entropy loss function is crucial for binary classification tasks, such as distinguishing between instances of stunting and non-stunting. This loss function quantifies the difference between the predicted and actual outcomes, providing a measure of how well the model is performing in terms of classification accuracy; in Step 8, the model is trained using a dataset comprising a 3814 number of observations. In this case, the training data used for model training involves a determined amount of information. The model is exposed to this dataset over a series of iterations known as epochs. In each epoch, the model refines its weights and biases based on the training data to improve its predictive capabilities. The choice of 100 epochs ensures that the model undergoes a sufficient number of iterations to converge and achieve optimal performance. Additionally, a batch size of 32 is employed, signifying that the model processes 32 instances of data in each epoch before updating its parameters. This batch-wise training approach helps in optimising computational efficiency and contribute to the model’s generalisation ability.

The 10 steps to calculate and print list of feature importance using ANN.

Step 9 to determine feature importance, the eli5 library’s permutation importance is used. Permutation importance was the model-independent strategy used after training an ANN to predict stunting. It measures the decline in model performance when each feature is randomly shuffled across instances to determine the relevance of particular characteristics. The decline in performance, as measured by measures like as accuracy or AUC-ROC, reflects the significance of a feature. Permutation is performed over numerous samples, and the procedure is iterated to ensure consistency. Features that cause a significant decline in performance are thought to be critical for the ANN’s prediction accuracy in stunting situations. This strategy assists in the identification and prioritisation of critical elements that influence the model’s predictions. For scikit-learn and Keras models, the eli5 module implements permutation importance. 21 Step 10 includes printing the features’ importance. The features’ importance is printed in descending order of significance, along with their appropriate weights.

 Results

This section shows the results from ANN model, Figure 2 shows an accuracy of 72%.

Model accuracy.

The ANN model built in this study showed good results in predicting stunting in Rwandan children. The model attained a 72% accuracy and a ROC of 0.84, indicating that it is a very useful tool for detecting children at risk of stunting as shown in Figure 3.

ROC curves of Artificial Neuron Network.

The feature importance analysis of an ANN model for predicting stunting in Rwanda was examined in this article. The model reveals the important elements related to stunting, giving light to both negative and positive value characteristics. The features of importance were computed and displayed using the ANN model as seen in Figure 4. Identifying these positive and negative value characteristics is pivotal for comprehending the intricate dynamics of stunting and devising targeted interventions to mitigate its prevalence.

Feature importance using ANN model. Negative value aspects

Figure 4 shows the realm of stunting prediction, certain factors exhibit a negative correlation with the likelihood of a child experiencing stunting. These include the mother’s height, child’s size at birth, gender of the child, mother’s education, place of residence, reading newspapers, and the mother’s age. These aspects, when present, tend to indicate a reduced risk of stunting.

 Positive value characteristics

There are characteristics associated with a positive correlation, suggesting an increased risk of stunting in children. These positive value features encompass the initiation of breastfeeding, the presence of mother’s anaemia, marital status, province of residence, occurrences of diarrhoea, altitude, birthweight, and the age of the baby as shown in Figure 4.

 Discussion

The ANN model’s results show considerable promise. The model displays its capacity to correctly categorise cases of stunting and non-stunting within the dataset with a significantly high degree of precision, with an accuracy of 72%. This demonstrates the model’s ability to detect critical patterns and variables associated with stunting in Rwandan children as also shown by Uddin et al. 23 This paper also show that ANN is very powerful compared to other models. 24 Furthermore, the model’s ROC score of 0.84 demonstrates its excellent discriminative capabilities. A higher ROC score indicates better distinction between the stunted and non-stunted child. The algorithm excels at reliably rating stunted children above non-stunted children, as indicated by an ROC score of 0.84, which is crucial in identifying individuals at high risk. These findings highlight the ANN model’s potential as a useful tool for predicting and mitigating stunting in Rwanda. 25 This study reveals that it is crucial to note that the ROC curve and AUC-ROC should be assessed in conjunction with other assessment measures like accuracy, precision, and recall to provide a thorough picture of the model’s performance and applicability for practical application in stunting prediction, however, this study used only the ROC. 25

Early identification of children at risk of stunting allows treatments to be targeted to those who need them the most, decreasing the burden of stunting on both the individual and society as a whole. 26 The use of ANN in stunting prediction offers the potential to improve early detection and intervention tactics. Policymakers and healthcare professionals may prioritise targeted treatments and deploy resources more efficiently if the primary factors leading to stunting are identified. 27 The study’s findings suggest that focusing on initiatives to enhance maternal nutrition, promote breastfeeding practices, and improve access to high-quality healthcare services could be of paramount importance in addressing the identified risk factors highlighted in the research as shown in Figure 4. Breastfeeding start appears as an important element, showing that early breastfeeding starting is connected with a decreased chance of stunting. These findings agree with the study conducted with Saberi-Karimian et al. 27 Understanding the factors that contribute to childhood stunting gives vital information to policymakers, healthcare professionals, and communities. 28

The same as the finding of this study suggested that efforts should be directed at improving maternal nutrition and health, supporting exclusive breastfeeding practices, and guaranteeing access to healthcare services that successfully manage maternal anaemia as well as the prevention and treatment of diarrhoea disorders, the mother’s height is important, as a lower height suggests an increased risk of stunting in children. 29 Additionally, customised treatments should be developed for certain provinces with a greater rate of stunting. The awareness among mothers and caregivers about the importance of child proper nutrition, breastfeeding practices, hygiene, and the significance of regular healthcare visits should be increased. 30 The research recommended that regular health checkups, growth monitoring, and testing to detect potential growth and developmental delays should be improved. Early measures, including nutritional supplements, caregiver counselling, and relevant healthcare interventions, can then be administered. 31 The gender of the child is also associated with the outcome of the stunting of the child as seen in Figure 4. The mothers giving birth to boys should pay close attention to the nutrition of their babies, as different studies revealed boys are more stunted than girls. 32

Being at higher altitudes is associated with a high risk of stunting in children in Rwanda as revealed by the study. 11 It is crucial to emphasise that altitude is only one of several factors that contribute to stunting, and its influence varies depending on other contextual factors such as economic status, healthcare facilities, and dietary choices. 33 To alleviate stunting in high-altitude locations, a comprehensive approach is required, which includes increasing access to healthcare, nutrition, sanitation, and education, as well as addressing the underlying socioeconomic determinants of health, 34 this study also confirm the findings shown in the different studies. Finally, encourage collaboration among government agencies, healthcare providers, non-governmental organisations, and community-based organisations in order to execute comprehensive and multi-sectoral stunting elimination strategies. This partnership can ensure a comprehensive strategy to address the recognised stunting feature importance by combining the expertise and resources provided by different stakeholders. 34

However, while the model worked well in this study, it is possible that it may not generalise well to other populations or circumstances. Further study is needed to test the model’s generalisability and to uncover other factors that may improve its performance. Another limitation pertained to the discussion phase, primarily due to the scarcity of existing ANN literature focused on stunting prediction. Consequently, it was challenging to draw meaningful comparisons between this research and prior studies. The contribution of this research is the use of ANN analysis in stunting prediction in Rwanda results in improved identification of significant characteristics, real-time monitoring, targeted interventions, and useful policy decision-making assistance. These contributions increase our understanding of stunting, guide targeted interventions, and may eventually contribute to lowering stunting rates and enhancing children’s well-being in Rwanda.

 Conclusions

The feature significance analysis of the ANN model in predicting stunting in Rwanda demonstrates the intricate interaction of numerous factors in affecting child growth and development. Positive value features stress the relevance of breastfeeding habits, mother health, and socioeconomic variables, whereas negative value features emphasise the importance of maternal qualities, education, and environmental factors. Rwanda may adopt targeted interventions and policies to minimise stunting prevalence and promote healthy growth and development for its children by addressing these major causes. In conclusion, the ANN model developed in this study provides a promising approach to predicting stunting in Rwanda. With further validation and refinement, it has the potential to significantly contribute to efforts aimed at reducing the stunting prevalence and improving child health outcomes in the country.

 Data availability Underlying data

In this study, we analysed datasets that are publicly available in the DHS program repository.

https://www.dhsprogram.com/data/dataset_admin/login_main.cfm. The request was address to DHS program after providing the purpose of the use of data then after we obtained authorisation from the DHS administration to use them. Data access is granted to individuals who create an account and submit a request. The DHS team reviews the request and, upon approval, provides access to the requested data.

 Software availability

Source code available from: https://github.com/Similien1/ANN_stunting/blob/main/Copy_of_Artificial_Neuron_Network.ipynb

Archived source code: https://doi.org/10.5281/zenodo.8424559. 18

Declaration: This manuscript shares the same data as the paper published https://jpmph.org/journal/view.php?doi=10.3961/jpmph.22.388 however the methodology used and result obtained are different.

 Acknowledgments

This research could not have been completed without the contribution of different people who deserve acknowledgement: The African Centre of Excellence in Data Science at the University of Rwanda, World Bank financing (ID: ESC 91) project sponsored this research.

 References WHO: The unicef/who/wb joint child malnutrition estimates (jme) group released new data for 2021. 2021. Osendarp S Akuoku JK Black RE : The covid-19 crisis will exacerbate maternal and child undernutrition and child mortality in low-and middle-income countries. Nature Food. 2021;2(7):476–484. 37117686 10.1038/s43016-021-00319-4 Union, A: Report on the implementation of activities on the roadmap of the au theme of the year for 2021 for the period february–june 2022. 2022. Goel A Goel AK Kumar A : The role of artifcial neural network and machine learning in utilizing spatial information. Spat. Inf. Res. 2022;31:275–285. 10.1007/s41324-022-00494-x Pasini A : Artifcial neural networks for small dataset analysis. J. Thorac. Dis. 2015;7(5):953–960. 26101654 10.3978/j.issn.2072-1439.2015.04.61 Mata J : Interpretation of concrete dam behaviour with artifcial neural network and multiple linear regression models. Eng. Struct. 2011;33(3):903–910. 10.1016/j.engstruct.2010.12.011 Sarker IH : Machine learning: Algorithms, real-world applications and research directions. SN Comput. Sci. 2021;2(3):160. 33778771 10.1007/s42979-021-00592-x PMC7983091 Abiodun OI Jantan A Omolara AE : State-of-the-art in artificial neural network applications: A survey. Heliyon. 2018;4(11):e00938. 10.1016/j.heliyon.2018.e00938 Diahashree G : Transfer learning and the art of using pre-trained models in deep learning. 2017. Baashar Y Alkawsi G Mustafa A : Toward predicting student’s academic performance using artificial neural networks (anns). Appl. Sci. 2022;12(3):1289. 10.3390/app12031289 Ndagijimana S Kabano IH Masabo E : Prediction of stunting among under-5 children in Rwanda using machine learning techniques. J. Prev. Med. Public Health. 2023;56(1):41–49. 36746421 10.3961/jpmph.22.388 PMC9925281 Rwanda MHT : Rwanda Demographic and Health Survey 2019-20. 2021. Kigali: NISR;2021. Organisation, W.H : Guideline: assessing and managing children at primary healthcare facilities to prevent overweight and obesity in the context of the double burden of malnutrition. 2017. UNICEF: Conceptual Framework Child Nutrition. New York, NY 10017, USA;2021. Al-Jabery K Obafemi-Ajayi T Olbricht G : Computational Learning Approaches to Data Analytics in Biomedical Applications. Academic Press;2019. Pujianto U Wibawa AP Akbar MI : K-nearest neighbour (k-nn) based missing data imputation. 2019 5th International Conference on Science in Information Technology (ICSITech). IEEE;2019; pp.83–88. Mansourifar H Shi W : Deep synthetic minority over-sampling technique. arXiv preprint arXiv:2003.09788. 2020. NDAGIJIMANA S : Predicting Stunting in Rwanda Using Artificial Neural Networks: A Demographic Health Survey 2020 Analysis. Zenodo. 2023. 10.5281/zenodo.8424559 Vabalas A Gowen E Poliakoff E : Machine learning algorithm validation with a limited sample size. PLoS One. 2019;14(11):0224365. 10.1371/journal.pone.0224365 Messner W : Hypothesis testing and machine learning: Interpreting variable effects in deep artificial neural networks using Cohen’s f2. arXiv preprint arXiv:2302.01407. 2023. Zhou S-N Jv D-W Meng X-F : Feasibility of machine learning-based modelling and prediction using multiple centers data to assess intrahepatic cholangiocarcinoma outcomes. Ann. Med. 2023;55(1):215–223. 36576390 10.1080/07853890.2022.2160008 PMC9809369 Antunes A Ferreira B Marques N : Hyperparameter optimisation of a convolutional neural network model for pipe burst location in water distribution networks. J. Imaging. 2023;9(3):68. 36976119 10.3390/jimaging9030068 PMC10052689 Uddin S Khan A Hossain ME : Comparing different supervised machine learning algorithms for disease prediction. BMC Med. Inform. Decis. Mak. 2019;19(1):1–16. 10.1186/s12911-019-1004-8 Stamenković LJ : Toxic/Hazardous Substances and Environmental Engineerin Application of ANN and SVM for prediction nutrients in rivers. J. Environ. Sci. Heal. Part A. 2021;56(8):867–873. 34061713 10.1080/10934529.2021.1933325 Hajian-Tilaki K : Receiver operating characteristic (roc) curve analysis for medical diagnostic test evaluation. Caspian J. Intern. Med. 2013;4(2):627–635. 24009950 Hanieh S Braat S Simpson JA : The stunting tool for early prevention: Development and external validation of a novel tool to predict the risk of stunting in children at 3 years of age. BMJ Glob. Health. 2019;4(6):001801. 10.1136/bmjgh-2019-001801 Saberi-Karimian M Khorasanchi Z Ghazizadeh H : Potential value and impact of data mining and machine learning in clinical diagnostics. Crit. Rev. Clin. Lab. Sci. 2021;58(4):275–296. 33739235 10.1080/10408363.2020.1857681 Gracey M King M : Indigenous health part 1: determinants and disease patterns. Lancet. 2009;374(9683):65–75. 10.1016/S0140-6736(09)60914-4 Bryce J Coitinho D Darnton-Hill I : Maternal and child undernutrition: effective action at national level. Lancet. 2008;371(9611):510–526. 18206224 10.1016/S0140-6736(07)61694-8 Ganesan S Jayaraj J Geminiganesan S : A study on parental awareness of feeding practices in children in the age-group 12-24 months. J. Prev. Med. Hyg. 2021;62(4):909. Liu Q Long Q Garner P : Growth monitoring and promotion (gmp) for children in low- and middle-income countries. Cochrane Database Syst. Rev. 2017;2017(1). 10.1002/14651858.CD010102.pub2 Rajabi M Ebrahimi P Aryankhesal A : Collaboration between the government and non-governmental organizations in providing health-care services: A systematic review of challenges. J. Educ. Health Promot. 2021;10. Shively G Schmiess J : Altitude and early child growth in 47 countries. Popul. Environ. 2021;43:257–288. 10.1007/s11111-021-00390-w Thurstans S Opondo C Seal A : Boys are more likely to be under-nourished than girls: a systematic review and meta-analysis of sex differences in undernutrition. BMJ Glob. Health. 2020;5(12):004030. 10.1136/bmjgh-2020-004030 10.5256/f1000research.154905.r274788 Reviewer response for version 1 Aryuni Mediana 1 Referee https://orcid.org/0000-0002-9761-3078 Bina Nusantara University, Jakarta, Indonesia

Competing interests: No competing interests were disclosed.

 12 6 2024 Copyright: © 2024 Aryuni M 2024 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

The paper needs some justifications for these concerns:

 The reason why they only applied 1 Machine learning algorithm (ANN), while their previous research (11) applied 6 algorithms?

The reason why they chose kNN for data imputation and SMOTE for data balancing

Reconsider differentiating validation and testing data.

Feature importance is done before or after learning model development

Figure 1 is not systematic and clear

It will be interesting if they also compare the accuracy with and without SMOTE

They must explain why ANN doesn't perform better than Random forest, SVM, extreme gradient boosting, and gradient boosting in the discussion.

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Partly

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

NA

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

NDAGIJIMANA Similien Primary health care, University of Rwanda, Butare, Southern Province, Rwanda

Competing interests: No competing interests were disclosed.

 31 12 2024

The reason was explained in the background of this paper “Given the crucial importance of addressing the root causes of stunting for effective treatments and policymaking, the researcher chose to conduct this study on the application of ANNs in the specific context of stunting in Rwanda, using the same dataset as the aforementioned publication”

Choosing kNN for data imputation ensures that missing values are filled in a manner that maintains the natural relationships within the data, while SMOTE for data balancing addresses class imbalance, enhancing the model's ability to learn from and predict both stunted and non-stunted cases effectively. These preprocessing steps are crucial for building a robust and reliable ANN model for stunting prediction in Rwanda.

The validation and testing were differentiated in the methodology section 

The feature importance was done after model development

I am sorry I did not get what did you mean by this question is it to refine the diagram? I keep it as it was however the title has been changed 

I did however I get the same before balancing and after balancing 

 While ANNs have shown substantial effectiveness in various predictive tasks, including illness prediction, their performance may not always surpass that of other algorithms like Random Forest, SVM, XGBoost, and Gradient Boosting in specific contexts, such as stunting prediction. The reasons include the higher complexity and risk of overfitting, significant data requirements, sensitivity to data quality, need for extensive feature engineering, and the inherent strengths of other algorithms in handling structured data, computational efficiency, and ease of hyperparameter tuning. Thus, the choice of algorithm should be guided by the specific characteristics of the dataset and the problem at hand.

 10.5256/f1000research.154905.r274789 Reviewer response for version 1 Chowdhury Mashfiqul Huq 1 Referee https://orcid.org/0000-0003-3438-6645 Mawlana Bhashani Science and Technology University, Santosh, Bangladesh

Competing interests: No competing interests were disclosed.

 22 5 2024 Copyright: © 2024 Chowdhury MH 2024 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

This study employs a deep learning model to predict stunting status among children under the age of five in Rwanda. The findings are intriguing. However, several improvements are necessary. The comments are detailed below:

Title can be modified to: Predicting stunting status among under-5 children in Rwanda using neural network model: Evidence from 2020 Rwanda demographic and health survey

In the Abstract, the results section needs improvement. The authors should rewrite the following sentence: Several factors appear as important contributors to the probability of stunting among the negative value aspects.

Introduction section:

a) Discuss the consequences of stunting and the overall situation of stunting in Rwanda among under-5 children.

b) In the second paragraph, the author critically evaluates the linear regression and decision tree models. It is recommended that the author also discuss the logistic regression model.

c) This sentence is not relevant to me: "They can also be parallelised over many processors, increasing computing efficiency.9 ANNs may be used for transfer learning, which involves fine-tuning a pre-trained model for a new task with little data. This is especially important when data is scarce or expensive to collect." 

d) Spelling mistake: Supportive Vector Machine

e) Give a reference to this sentence: ANNs have proven to be highly effective in predicting illnesses.

f) The authors need to provide a more detailed description of the motivation behind this study.

g) The literature review is very limited. I suggest adding more references to existing studies and relevant literature.

h) This sentence is not clear: "Given the crucial importance of addressing the root causes of stunting for effective treatments and policymaking, the researcher chose to conduct this study on the application of ANNs in the specific context of stunting in Rwanda, using the same dataset as the aforementioned publication.

Methods section:

a) This section can be split into the following sub-sections: Data Source and Sampling Design, Study Population, and Variables (Explanatory and Outcome Variables). The authors need to discuss the data file used from the DHS website, including information on any missing variables and whether they were discarded or handled in some other way.

b) In Table 1, the following changes are recommended: replace "Baby's age" with "Child's age," "Antenatal" with "Antenatal care visits," and "Reading" with "Media access." For variables such as source of drinking water, toilet facilities, and place of delivery, define the categories clearly in the variables sub-section. Specify the definitions for unimproved and improved statuses earlier in the text. Also, clarify which places are considered health facilities. Instead of using mother's height, the authors may consider the mother's BMI variable.

c) The variables "size of a child" and "birthweight" are essentially the same. Should both variables be included?

d) The authors need to clarify why the altitude variable is crucial for this analysis.

e) Authors can create a new subsection titled "Experimental Setup" to systematically outline the experimental procedures. This section should be written in a clear and organized manner to ensure ease of understanding for readers.

f) In the section discussing the Artificial Neural Network (ANN) model, it's important to specify the size of the hidden layer and provide justification for this choice, possibly citing relevant references. Additionally, authors should explain how they tuned the hyperparameters of the model. Including a reference for the Adam optimizer would also be beneficial. To improve clarity and prevent redundancy, it's recommended not to repeat the discussion of Scikit-learn, TensorFlow, and Python many times.

g) The authors should clarify the context in which they use both standardization methods, Minimax scaler and Standard scaler, in their paper. It's important to explain why each method is chosen and how they are applied to the data. This clarification will help readers understand the rationale behind using different scaling techniques and their impact on the results.

i) Use observations instead of instances.

j) Figure 1 title needs to be improved.

Results section:

a) In Figure 2, the x-axis and y-axis titles should be clearly labeled. The legends also need to be improved to indicate "Training Loss," "Test Loss," "Training Accuracy," and "Test Accuracy." Additionally, the main title of the figure should be rewritten for clarity.

b) Write a general comment based on Figures 2, 3, 4. Write in 2-3 paragraphs without sub-sections.

c) If feasible, conduct a comparison with a machine learning model like logistic regression. This comparative analysis will elucidate which type of model (ML or DL) is best suited for this type of dataset.

d) This sentence is not clear to me: "This study reveals that it is crucial to note that the ROC curve and AUC-ROC should be assessed in conjunction with other assessment measures like accuracy, precision, and recall to provide a thorough picture of the model’s performance and applicability for practical application in stunting prediction, however, this study used only the ROC".

Discussion section:

Authors can emphasize identifying similarities or consistency with other existing works.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Partly

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Statistics and Data Science

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

NDAGIJIMANA Similien Primary health care, University of Rwanda, Butare, Southern Province, Rwanda

Competing interests: No competing interests were disclosed.

 31 12 2024

For title

The title is modified as suggested

For abstract

I added some results, however, there is a limited number of words

The sentence has been rephrased like this: Factors appear to contribute to stunting among the negative value aspects. 

Introduction section:

The discussion was added

As in the paper I published before and due to the limitation of the article I did not discuss this model here.

The sentence is removed

Supportive is changed into supporting

The reference is given

More details were given

The references are added

I paraphrased this sentence

 Methods section:

The subsections have been added

The variables were changed

The variables "size of a child" and "birthweight " are not almost the same since one measures the height while the other measures the weight.

From the analysis made before we find that altitude contributes to stunting being at high altitude is an associated risk factor for stunting, even in this analysis, ANN shows altitude as the associated factor to stunting.

The section was not added due to the limited number of words however were explained in the methods

The comment was accepted and adjusted as suggested.

The clarification is given

The comments are considered the observation replaced the instances

The title is improved  

 Results section:

a) The title and legends have been added

b) I did it as suggested

c) the comparison made in the previous study

d) The sentence is paraphrased,

“This study highlights the importance of evaluating the ROC curve and AUC-ROC alongside other metrics such as accuracy, precision, and recall to gain a comprehensive understanding of the model's performance and its practical applicability in predicting stunting. However, it is important to note that this study only used the ROC for assessment.

Discussion section:

The emphases on similarities or consistency with other existing works were added

 10.5256/f1000research.154905.r274786 Reviewer response for version 1 Ogwel Billy 1 Referee https://orcid.org/0000-0002-9097-2713 Kenya Medical Research Institute- Center for Global Health Research (KEMRI-CGHR), Kisumu, Kenya

Competing interests: No competing interests were disclosed.

 17 5 2024 Copyright: © 2024 Ogwel B 2024 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

This manuscript addresses a significant public health challenge and provides valuable insights. To further enhance its impact, the authors may consider the following suggestions:

Abstract:

-Add detail on the modelling approach in the methods section.

- Report 95% CI around the estimates

-Be very explicit on the targeted interventions in the conclusion of the abstract.

Introduction

-More recent estimates of stunting are available 148.1 million as at 2022 ( https://www.who.int/data/gho/data/themes/topics/joint-child-malnutrition-estimates-unicef-who-wb)

-The authors can add successful implementations of ANNs in the healthcare domain in the paragraph where they talk about the strengths of ANNs.

-The authors need to add the justification of their study why it is important to use ANNs to the same dataset that was used by authors in reference 11 in the same setting using SVM, NB, RF, LR, and XGBoost algorithms . Do they aim to achieve a higher predictive accuracy?

Methods

The authors say the data was partitioned into a training set, a validation set, and a test set. Yet talk only of a 80%:20% split. They could clarify this.

Results:

-The authors could refer to the TRIPOD checklist on reporting of diagnostic or prognostic prediction studies (Collins GS ,et.al., 2015 [Ref1]) . Specifically, they could : "Describe the flow of participants through the study, including the number of participants with and without the outcome and, if applicable, a summary of the follow-up time. A diagram may be helpful."

-They also need to report performance measures (with CIs) for the prediction model according to this guideline.

Discussion:

The authors could improve the discussion by comparing their results to those of other studies that have used DHS to predict stunting and particularly reference 11 that was done in the same setting using the same data. Did their ANN model improve performance as hypothesized in the background?

the authors could also discuss the potential clinical use of the model, highlighting where and when it can be used in the healthcare system

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Enteric Research, health informatics

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

References : Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Med .2015;13: 10.1186/s12916-014-0241-z 1 25563062 10.1186/s12916-014-0241-z NDAGIJIMANA Similien Primary health care, University of Rwanda, Butare, Southern Province, Rwanda

Competing interests: No competing interests were disclosed.

 31 12 2024

For the abstract

The modelling approach has been added 

- The percentage mentioned in the abstract is derived from the figure; therefore, the 95% confidence intervals have not been provided.

- The target interventions have been explained in detail in the abstract, however, due to the limited number of words more have been given in the conclusion 

For introduction 

- The reference and the estimated number of stunted children were added in the introduction section

-The successful applications of ANN have been added

- The reason was explained in the background of this paper “Given the crucial importance of addressing the root causes of stunting for effective treatments and policymaking, the researcher chose to conduct this study on the application of ANNs in the specific context of stunting in Rwanda, using the same dataset as the aforementioned publication”

For Methodology 

-This explanation clarifies that the validation set is a subset of the training data, not the testing data, which helps to refine and tune the model before the final evaluation of the test set. Hence testing and validation share 20%

For results 

-The proportion of Stunted children was given in the results section

For Discussion 

-The incorporation of Artificial Neural Networks (ANN) within various healthcare applications has been integrated into the discourse; nevertheless, due to the constraints of the paper's length, I have included some additional insights.