A machine learning (ML) algorithm was conducted using Python and analyzed on Google Colab, ^{34, 35} utilizing secondary data from the 2019 Performance Monitoring and Accountability (PMA) Ethiopia cross-sectional household and women’s survey. PMA-Ethiopia is a collaborative five-year initiative (2019–2023) involving Addis Ababa University, Johns Hopkins University, and the Federal Ministry of Health. The project comprises three key components: annual cross-sectional surveys of women aged 15–49, longitudinal studies tracking pregnant women and new mothers, and yearly service delivery point surveys assessing health facilities. ³⁶

The study sourced all reproductive-age women in Ethiopia, with the study population comprising women who participated in the 2019 PMA-Ethiopia cross-sectional survey.

The PMA survey sample is based on a multi-stage cluster design, with urban-rural and primary fields as strata. A nationally representative number of enumeration areas are selected from each region of the country. Then, in each enumeration region, households are identified and mapped before being systematically chosen for participation in the survey round through a random process. The female participant survey is included in each household questionnaire and consists of a series of questions for all women of reproductive age (15-49) residing in that household.

The household and female surveys are carried out by female data collectors, known as resident enumerators (REs) which are typically women over the age of 21 who are from or near the respective enumeration areas and hold at least a high school diploma. Each RE takes about six weeks to collect data from all selected households, eligible women, and service delivery points. The surveys include interviews among females aged 15 to 49 who are consented and screened for eligibility, as well as a random sample of health institutions, pharmacies, and retail stores that provide family planning services in the selected areas.

Women are eligible for the survey if they are regular members of the household, including women staying at their parental home for the delivery and postpartum period, and self-identified as currently pregnant or less than six weeks postpartum. Female respondents are asked questions about their background, birth history, fertility desires, methods of contraception used, and other information that policymakers and program administrators may utilize to promote health and family planning. ³⁶ The analysis sample was weighted to account for nonresponse and differences in selection probabilities. It was further limited to responses from women of reproductive age at the time of the survey, resulting in a weighted sample of 8,837 women.

Dependent variable

In this study, Modern contraceptive use was the dependent variable and was dichotomized into two categories: ‘yes’ and ‘No’. A woman was considered to be using modern contraception if she used any of the following methods; female sterilization, implant, IUD, injectable, pill, emergency contraception, female or male condoms, cycle beads, and LAM. ³⁷

Predictor variables

Various socio-demographic, economic, maternal and health service-related factors were included as predictor variables. Socio-demographic and economic factors included mothers’ current age (categorized in to 15-24 years, 25-34 years and 35-49 years), region (included Gambela, Harari, Southern Nations Nationalities and People (SNNP), Oromia, Somalia, Benishangul-Gumuz, Afar, Amhara, Tigray, and Afar and two cities namely Addis Ababa and Dire Dawa), residence (urban and rural), religion (Christian, Muslim and Other which includes Wakefata and Atheist) and educational level (categorized as no schooling, primary education, and secondary and above), sex of the household head (Male and Female), and relationship (grouped in to head of the household, wife, daughter or daughter-in-law, other relatives and not related), family wealth indices (poorest, middle and richest), household size( grouped in to 1-3 members, 4-5 members, 6 and above members) and marital status(grouped in to never married, Married and widowed/divorced/separated).

Maternal and health service-related factors included number of under five children (grouped in to no under_5 children and 1 and more under_5 children), number of children ever born (grouped in to 1-3 children and 4 and more children), birth in the last 5 years (1-2 birth and 3-5 birth), birth in the last year (no birth and 1 birth), knowledge about reproductive methods (knows about modern methods, knows about traditional methods only, and no knowledge about any method), cesarean delivery (no and yes) and age at first birth (below 15 years, between 16 and 34 years and above 35 years) ( Table 1).

Variable extraction and imputation were conducted using R software version 4.4.3. Using Jupyter Notebook, the Scikit-learn machine learning (ML) algorithms were built in Python 3.11.5. We used six widely accepted machine learning algorithms to predict the determinants of modern contraceptive use in Ethiopia and compared the results of the best algorithm to the results of the traditional logistic regression model. The k-nearest neighbors (KNN) model is chosen based on its ability to detect linear and nonlinear boundaries between groups. The K value represents the number of nearest neighbors and is the core deciding factor in this classifier. The Random Forest (RF) model is used in machine learning situations because it is highly flexible and provides good predictive performance. It produces ensemble predictions that are more accurate than any of the individual predictions. The naive Bayes algorithm is a supervised machine learning algorithm that uses the Bayes theorem for classification and prediction. It has an incremental learning behavior and is not affected by training time. Logistic regression is a statistical model that is used to classify and predict different health parameters, where the target variable is dichotomous and the independent variables are independent of each other. The general framework used in the literature ^{38, 39} based on Yufeng Guo’s 7 machine learning steps was used in this study. The framework describes the seven steps in supervised machine learning, which are as follows: data collection, data preparation, model selection, model training, model evaluation, parameter tuning, and prediction.

Data source/collection

The dataset for this study is available on the PMA Survey website and can be obtained upon formal request. A weighted sample of 8837 reproductive-age women was included in the data. The datasets analyzed in the current study are available in the PMA repository, https://www.pmadata.org/data/available-datasets .

Data preparation: The data preprocessing techniques used in this investigation included data cleaning, data splitting, feature engineering, and dimensionality reduction. After the data were extracted, data cleaning was performed, which included finding and removing outliers from the dataset, as well as resolving missing values and uneven categories in the resulting variable. This study employed the KNN imputation approach to handle the missing values in the dataset related to the independent variables. KNN imputation was chosen over parametric methods as the dataset had a small percentage of missing data (11%), Little’s MCAR test revealed that the missing data mechanism was Missing Completely at Random (MCAR) (p > 0.05), and the relationships between the variables were not strictly linear. ^{33, 40}

The raw data were transformed into features that more accurately depicted the underlying issue for the predictive models, improving the accuracy of the unobserved data. Hence, among other feature engineering techniques, label encoding for coding each category of variables as a number and encoding categorical variables into numeric values for nominal variables were carried out. The process of dimension reduction was used to decrease the number of input variables for the predictive model. Having fewer input variables might lead to a simpler predictive model, which can perform better when generating predictions on new information. To ensure that only the most essential dummy variables were included, we reduced the amount of features using RFECV. Since Principal Component Analysis (PCA) is not ideal for one-hot-encoded categorical features. Using those techniques to assess the link between the independent input components and the output variable and selecting the most important independent variables, feature selection and feature extraction was utilized to forecast the target variable. This technique has frequently been used in earlier public health research to identify the factors and/or predictors of different health outcomes. ^{30, 41}

This study utilized a standard 80/20 split approach, where 80% of the data were used for training and the remaining 20% were used for model testing. The model was trained using the tenfold cross-validation method, which does not waste much data; this approach is highly beneficial when the number of samples is limited. ³⁰ To train the prediction function, K-fold separates all the observations into equal-sized groups of samples termed folds and k-1 folds. The fold that is left out is then used for testing k times repeatedly. ³³ The average of the results calculated in the loop serves as the k-fold cross-validation performance measure.

Model selection: Appropriate models were selected for training after the data were prepared and split into training and testing sets. The task was a binary classifier because the outcome variable was categorical, and suitable classifiers needed to be chosen to make the prediction. Since modern contraceptive use was divided into two mutually exclusive categories (use or not use), the dataset employed in the analysis falls under the category of binary categorization. Hence, in this study, we used six widely used machine learning (ML) algorithms, ³³ logistic regression (LR), random forest (RF), K-nearest neighbors (KNN), extreme gradient boosting (XGBoost), naive Bayes (NB), and support vector machines (SVMs), to predict determinants of modern contraceptive use among reproductive-age women in Ethiopia and compared the results of the traditional logistic regression model to the results of the best algorithm for identifying the new features influencing the outcome of interest.

Model training: The selected classifiers were trained using prepared data after model selection, and their performances were compared via tenfold cross-validation. Following this comparison, the top predictive model was chosen, and it was trained with balanced training data to make the final prediction on hypothetical test data. ³³

Model evaluation: Testing the model’s performance on never-before-seen data that were set aside for this purpose during data splitting can help determine how well the model works after it has been trained. One popular technique for evaluating the effectiveness of a classification model is the confusion matrix, which is a straightforward cross-tabulation of the actual and predicted categories for the outcome variable. The performance criteria, which include the overall accuracy, precision, recall, and F1 score and were employed in this study to evaluate the effectiveness of the selected classifiers, can be calculated using confusion metrics. Additionally, the performance of the ML models was assessed using receiver operating characteristic (ROC) curves, and the value 0.5 = no discrimination. 0.5-0.7 = Poor discrimination. 0.7-0.8 = Acceptable discrimination. 0.8-0.9 = Excellent discrimination. ⁴¹

Hyperparameter tuning: To better understand the possibility of obtaining the optimal values and avoid unnecessary computations for combinations of nonperforming parameters when searching for the optimal parameter settings. Grid search, random search, and Bayesian optimization were used to formulate hyperparameter optimization to improve the speed and quality of the learning process, and we attempted to incrementally adjust the parameters of our model to improve its performance. ⁴²

Making prediction: All the aforementioned activities occur in this stage, which is the last stage in machine learning methodology. By using independent variables as a framework, prediction involves estimating the outcome variable. In this instance, modern contraceptive use was established using crucial factors that were discovered along the route. The best-performing classifier with a specified accuracy was used to predict whether a woman would use modern contraceptive services given various factors. The overall workflow of the methodology is shown below ( Figure 1).

Most of the women (5177, 56.85%) were rural residents, and 5,326 (58.48%) were aged between 26 and 35 years. The majority of participants was poor (32.48%), had a primary education (54.34%) and had no media access (92.1%). Regarding the regional distribution of respondents, the majority of the women were from Oromia (19.44%), followed by the SNNPR (18.15%), and approximately 17.66% were from Amhara. The remaining regions accounted for 44.75% of the total study population ( Table 2).

Among the total respondents, 6,029 (67.55%) had a history of pregnancy, and approximately 4,769 (70.29%) had started sexual intercourse before the age of 18 years. Approximately 2,544 (43.64%) of the women had never delivered at health facilities. Regarding their partner/husband feelings toward FP use, the majority of the women (3,298; 58.41%) had approved the use of FPs by their husband/partner ( Table 3).

Among the study participants, only 24% {95% CI (23.1%, 24.9%)} used modern contraceptive methods. Most (1,204, 13.6%) of the modern contraceptive users had a secondary education ( Figure 2).

Feature selection

Feature selection is crucial for determining the most important predictors of an outcome variable, similar to how p values and t statistics are used in most traditional statistical methods, to determine which variables are significant. Accordingly, when using an ensemble model to predict an outcome, feature importance measures how significant a feature is on average concerning other features.

The most significant predictors of modern contraceptive use according to the extreme gradient boost feature importance results were age at first family planning use (20 years and above), marital status (single), partner/husband feelings about family planning (Approval from partner) and relationship with the head of household (having a wife relationship to the head). In addition, age (36 to 49 years), health care providers advise about the use of FP methods during health facility visit, perception about the FP side effects might disrupt their relationship and household size (5 and above) were other key predictors. The length of the bars on the x-axis, which represents the relative significance of the independent variables in predicting the use of modern contraceptives, shows this. The longer the bar is, the more significant the trait is in determining whether a woman utilizes modern contraception ( Figure 3).

Model selection

After the data were prepared and divided into training and testing datasets, as indicated in Table 4, the appropriate models were chosen for the training dataset. To predict the factors that influence the use of modern contraceptives among reproductive-age women in Ethiopia, we used six commonly used machine learning (ML) algorithms, namely, logistic regression (LR), random forest (RF), K-nearest neighbors (KNN), extreme gradient boosting (XGBoost), naive Bayes (NB), and support vector machines (SVMs). We chose the best model from among these machine learning approaches based on its higher level of accuracy ( Table 4).

A figure based on the mean level of accuracy serves as the representation for the summary models. The extreme gradient boost model therefore had the best model for training the dataset, with a mean accuracy of 80.6% ( Figure 4).

Prediction of modern contraceptive use

Using the remaining test data (predictions from unseen test data), the performance of the predictive models for predicting modern contraceptive use was compared using the mean accuracy and mean area under the curve (AUC) of the ML models in stratified tenfold cross-validation.

After all the models were investigated in this study, extreme gradient boosting was found to have the greatest accuracy (81.97%, 95% CI{(79.06%, 82.7%)} as was the ROC area (76.63%), followed by logistic regression with minor difference (80.52%, 95% CI {(78.6%, 82.3%)} and support vector machines (SVMs) (80.41%), 95% CI{(78.48%, 82.2%)}.

The XGBoost model had relatively low specificity (88.67%), which meant that it performed poorly in identifying predictors of modern contraceptive use in Ethiopia but had high sensitivity (66.6%), which meant that it was more accurate in identifying predictors of modern contraceptive use ( Table 5).

Visualization of the receiver operating characteristic (ROC) curve was performed. Among the six machine learning models employed in this study, the curve of the extreme gradient boost model had the highest percentage of AUC values, indicating that it is the best at classifying the use or nonuse of modern contraceptive methods among reproductive-age women in Ethiopia. Moreover, this best model represented an acceptable range of AUC values (76.63%) ( Figure 5).

✓

A key limitation of this study is the regional imbalance of the sample, with majority of participants being from rural areas, which may limit the generalizability of the findings to more urban populations.

Ethical clearance and consent was not necessary for this study since it was based on publicly available data sources. Permission to use the data was granted by the PMA Ethiopia’s survey project through legal registration.