Navigating Sample Size Determination in Educational Research: A Rapid Review Unveiling Strategies, Challenges, and Recommendations.

Introduction

The term “sample size” describes the number of subjects or observations that make up a study ‘n’ is typically used to represent this number. The size of a sample affects two statistical properties:1) the accuracy of estimates and 2) the study's ability to draw inferences.¹

Surveys, experiments, observational studies, and other types of clinical research studies can all be categorized. Many different factors are involved in excellent research planning. The first step is to define the practical issue. Choosing the relevant participants and controls, as well as the experimental or observational units, was the second stage.

The inclusion and exclusion criteria must be carefully defined and should account for any potential variables that could affect the measurements and units being observed. The study design must be precise, and the procedures must follow the best technique currently available. Based on these considerations, the study's sample size needs to be appropriate for its goals and potential variability. The sample must be “large enough” for the effect to be statistically significant and have the expected size of scientific significance. At the same time, it is crucial that the study sample not be “too big,” where a statistically significant effect of minor scientific import could still be found.² Additionally, the sample size was economically significant. Resources may be wasted in an insufficient study because it may not yield valuable results, whereas an excessively large study consumes more resources than is required. The sample size of a study involving human or animal subjects is a crucial ethical concern, because a poorly planned experiment exposes participants to potentially hazardous procedures without contributing to information.^3,4 Therefore, calculating the power and sample size is crucial in the design of clinical research. Unaccountable studies printed in national and intercontinental journals have found that sample size estimation were incorrectly disclosed or had smaller samples than necessary, which reduced their power.^1,2

There is still much confusion despite the fact that unnumbered studies clarifying the methods of sample size computation have been published in the existing literature. It is crucial to realize that there is no single universal formula for calculating the sample sizes for all study designs. Instead, different study designs require different methods to calculate sample sizes.^3,4 The study was conducted with the objective is to provide insights, address the challenges, and offer recommendation regarding sample size determination in educational research.

Method

To conduct this rapid review, Preferred Reporting Items for Systematic review and meta-analysis (PRISMA-S) criteria were used to guide the search.⁵ The search strategy was aimed to identify relevant articles addressing sample size determination in educational research. Researches searched the following databases such as PubMed, Embase, and the Cochrane Library, additionally; hand searching of reference lists and citation tracking were also conducted to identify additional relevant articles. The search term or keywords for accessing the data from the databases related to sample size (e.g. sample size calculation, sample size determination) and educational research (educational studies, pedagogic studies) were utilized. To perform search effectively and refine search results, the search strings were constructed using BOOLEAN operator (AND, OR). Free full-text, unlocked articles, pertinent terminology and information, and English language usage were considered as inclusion gauge. The exclusion criterion was contemplated and they excluded abstracts, locked articles and journals, no relevance in the data, and languages other than English. Two authors screened the titles and data of the extracted articles to determine the eligibility of the included articles. Data from the extracted articles were documented into standardized format in Microsoft excel sheet. The included articles were assessed by using appropriate criteria such as methodological rigor, relevance to the research question, and transparency of reporting. In the end all the data were synthesized for common themes, relevancy, challenges, and recommendation related to the sample size assessment. The principal investigator carried out the entire review planning process, which was authorized by the other authors. The presentation of the entire search is shown in Figure 1.

Figure 1. The schematic presentation of the data by using updated PRISMA guidelines.

Results

Seven studies were selected from the 9282 articles by using Google Scholar and PubMed with the applicable of stringent inclusion and exclusion criteria. All the information related to the articles were shown in the Table 1.

Discussion

Research in health science education is expanding. Emerging educational research relies on relevant conceptual frameworks, reliable research techniques, and important discoveries.^13,14 Prior reviews have shown that many educational research articles employ small sample sizes, despite the fact that researchers rarely take into account the expected impact size, intend the sample size before, or describe the actual precision in evaluating the results.^15,16 Although authors rarely analyse the anticipated influence size, arrange the sample group in detail, or analyse the results from the perspective of actual precision.⁹

According to the definition of statistical power, it is “the likelihood that the null hypothesis will be rejected in the sample if the observed effect in the population is equal to the effect size”.¹⁷ In other words, the potential that a study will uncover a real, statistically significant effect is known as power. Studies with a higher power are preferable because lower-power studies may miss potentially important connections. A power of 90% is ideal and 80% is typically considered the minimum power. The sample size (the number of observations), the effect size (the value of the effect), and the risk of type I errors all affect power (the likelihood of recognising a “significant” difference when there is none, represented by alpha).^9,18

The study adopted a convenience sampling method for primary research for determination of sample size in education by examination of simulation-based education. First, most research in the sample only had the power to find effects with moderate to large SMD [0.8], while other studies only had the power to find effects with immensely large magnitudes ([2 standard deviations). Most of the negative studies, or those that did not find a statistically relevant difference, had very broad confidence intervals (CI), signifying the probability of large and likely important differences. The first point and discovery were connected. In these trials, the lack of a statistically relevant outcome did not establish superiority or equivalence of the interventions under study.⁹

In one study, the author aimed to present sample size calculations in the context of medical educational interventions and focused on computing sample sizes to compare distinct groups where the result was a continuous (interval or ratio) dependent variable, such as in interventional designs. The criteria for forecasting the sample group, such as the relevance factor, preferred statistical significance, predicted difference in score, and approximate evaluation variation, which may be estimated from previous studies, were discussed in order to determine the number of participants required to assess the effects of an intervention on a specific outcome or the association between variables.^6,19 Interventions in education frequently concentrate on changing latent conceptions, which are theoretical and cannot be readily seen or quantified. This causes the validated scales to vary, changing how the outcome measures are calculated. The educational researcher advocated the use of effect size in determining the sample size. The study design often affects the relationship between larger effect sizes and smaller sample sizes. This resulted in the effect sizes being categorized as “small,” “medium,” and “large,” respectively, for values of 0.20, 0.50, and 0.80.²⁰ Finally, the meta-analysis revealed that the sample size for each group was 24.24 respectively.^21,22

Further, author’s also discussed errors to avoid, including considering sample size estimation as small, medium, or large, which leads to a failure in the accuracy of the evaluation tool and sample characteristics.²³ Second, unless necessary, researchers should avoid creating new institute-specific assessment instruments. This is because they must be validated for accuracy and reliability before use in interventional studies.²⁴ Third, the prospective dropout and attrition rates must be considered.²⁵Finally, the need to avoid equating the effect size with its true significance and employing a confidence interval that offered accuracy in the sample and effect sizes.⁶

The objective of the report was to establish the optimal number of subjects for a study during the planning stage, with sufficient patients to resolve the most clinically important questions and statistical power calculations. The evaluation of the sample size that must be randomised to each arm in order to achieve the standard 80% or 90% power to find a clinically meaningful effect in randomised controlled trials, which frequently use parallel group designs. The need for a control arm, statistical comparability, structural equality, and resemblance of management conditions and observations are among the themes that the author elaborated on as being essential for educational research investigations. If an academic research study exhibits these traits, hence the test arm's success is significantly greater than that of the control arm, and the distinctness cannot be the result of concurrence. The cluster randomization was usually performed for non-therapeutic intervention such as prevention program, healthcare program and training programs. Two to thousands of individuals were found in each cluster. Education research may also consider different cluster sizes.⁸

Minimizing or reducing contamination bias is the fundamental reason for performing cluster-randomized studies. Observations inside clusters are typically more comparable to one another than observations from distinct clusters, creating a unique data structure known as a statistical dependency. The effective sample size of a cluster-randomized study is less than the concrete sample size, which has an impact on sample size computation (i.e., the number of enrolled students). Consequently, it is inappropriate to use typical methods that presume the statistical competence of all observations to rule out the sample size for cluster-randomized investigations.⁸

The purpose of the study, which was conducted by paediatric surgeons, was to disseminate a method for selecting a sample size to identify an effect that would have therapeutic significance through the interpretation and validation of the findings. Using a five-step approach, it is possible to calculate the minimum sample size necessary to ensure sufficient power and accurate interpretation of the study's findings.⁷ The sample size that can be achieved to assess a significant effect on the basis of research or primary data must be justified using the power calculation. The research sample size should have adequate statistical power to identify clinically meaningful effects in scientific investigations.^7,26 The sample size of the prior control group determined the statistical power. To compare the two groups effectively, comparisons must be made with a historical control group that is comparable to the research group, for which data on assessed confounders are available. The suggested 5-step approach can be used with any type of data or study design, although power and sample size primers do not provide examples for every possible research circumstance. The fundamental objective of the primers was to compare the two treatment groups. However, due to multiplicity and multiple testing, there is a higher risk of false-positive results (Type I error) when comparing more than two groups.^27,28

Guo et al. used two different types of hypotheses, taking into account sample size planning factors such superiority/non-inferiority and equivalence of two means. When population variances are unknown, no exact sample can be found through traditional sample size formula and resulting sample size must be suitable enough to meet the required level of significance and probability of correct decision and power. The cost constraint depends on the two experimental goals for given level of αand power 1-β i.e. allocation of having minimal total cost and ratios are a function of unit cost ration and standard deviations.¹²

Historically, three methods have been employed to determine the sample size. The first is an interval strategy, where the confidence interval is high (e.g., 95%) and the sampling error between the true parameter and its estimate is kept to a present modest amount, that is, 3 percent. Since there is no hypothesis testing involved in this method, no threshold of significance is required. The second is a hypothesis-related approach in which both the null and alternative hypotheses must be precisely specified beforehand to detect a significant difference between the parameters under study while simultaneously meeting the required level of significance (Type I error rate) and the desired power (probability of correctly accepting the specified alternative). The third strategy uses a “indifference zone,” where populations that perform better than the others are placed in a zone where they are more likely to be chosen correctly.²⁹

A “priori" literally translates from Latin as “what comes before” and they are a fundamental part of the scientific method since they are created based on assumptions.³⁰ From these assumptions, three hypotheses were inferred. With reference to objective methodologies, the aim of this systematic review was to offer proof for a priori hypotheses and sample size for evaluating the quantity and duration of physical activity in a pediatric population. The results of systematic review suggest that the degree of agreement between subjective and objective measures for determining the intensity and duration of physical exercise should be assumed modest to moderate.¹¹

Currently, there are no data to support an a priori assumption regarding how well the different methods of assessment agree. To select a sample size, attain precision, or have sufficient power to reject a false null hypothesis, a robust a priori hypothesis is necessary. Cost and feasibility, which are frequently the true drivers of the sample size, cannot be disregarded by researchers. Nonetheless, typical power calculations yield only specific sample sizes by making precise assumptions. This study’s results indicate that, for assessing nearly all physical activity, intensity and duration parameters, a sample size of 50–90 subjects offers constant agreement between subjective and objective approaches. The degree of uniformity displayed in each (often non-representative) sample studied, the accuracy of the subjective method created for a target sample, and the inadequacy of the correlation coefficient for detecting agreement issues are all potential explanations for stable agreement in this sample size interval. Additionally, studies with small samples showed higher levels of variability in the range of findings, perhaps as a result of the inferior design of these studies to those with larger samples.³¹

The “vibration of effects” diminishes the reliability of the consensus measures in samples with less than 50 respondents. The study predicts that the basis for the decreased reliability of the agreement measures in studies with samples of 100 or more persons is that the researchers' attempts to ignore the occurrence of an exaggerated effect in a small-sample trial when a finding is made is the primary factor.³² The superiority in methodology systematic evaluations addressing the agreement between subjective and objective measures for assessing physical activity has frequently found low methodological quality in the studies.^32–35

The COSMIN checklist, which was employed in the cited study, identified the absence of an a priori hypothesis and small sample size (n = 50) as the primary factors affecting the methodological standard of the redeemed studies. These factors were obeyed by a lack of data regarding subjects who were missing and the way in which missing data were handled. The author disapproved with questionnaires, diaries, and/or logs those received low ratings in methodological quality evaluations are ineffective tools for gathering subjective data.³²

The sample size was depended upon the degree of heterogeneity, if the analysis was performed by multiple investigators and teams. Moreover, studies with limited data showed higher levels of variability in the range of findings, perhaps as a result of the inferior design of these studies to those with larger samples.^33,34

A statistician is essential to rule out the number of subjects and analyse the final results of the entire investigation. To perform a suitable well-defined study that produces rational and trustworthy implications that can be applied to the sample population, it is crucial for the investigator to understand the fundamentals of analytical methods. Clinicians can use statistics to extract crucial information from empirical data, which improves patient care. Statistical notions must be considered from the initial planning stage to the final reporting phase. In general, there are two sorts of sample size estimation problems: sample size for (a) an estimating study and (b) to tests a hypothesis, or a comparison study.¹⁰

When performing an estimation study, the researcher was interested in estimating the quantity of one or more parameters, including, among other parameters, the mean haemoglobin level or arthritis prevalence. Researchers were interested in comparing population characteristics at one or more time points or characteristics of two or more populations in studies that test hypotheses. For instance, they might compare the prevalence of arthritis between two populations before and after the administration of an intervention. A researcher should select a large number of people if they wanted the estimation in their study to be more precise, because as the accuracy (or margin of error) grows (or lowers), the minimum sample size necessary increases. For instance, for a sample size greater than that preferred for a 95% confidence level, that estimate of a parameter is required. The computation of sample size in studies testing hypotheses aims to obtain the appealed power for disclosing a difference that is therapeutically or experimental significant at a predetermined significance level.³⁵

According to the statistics, there are various methods, test and formula for estimation of the sample size required to perform the research and other relevant studies. But the lack of research regarding the appropriate and whole number needed for performing any research is not established yet, like pilot study confirms the 12 participants for each group enrolling for the particular trial.³⁶

This review highlights the diverse approaches and procedures used in many studies to determine the appropriate sample sizes for educational research. It also emphasized the difficulties in defining the main outcome, choosing the right statistical tests, and taking effect size and statistical power into consideration. This article fosters a critical comprehension of the results and their relevance in various research contexts. Along with the recommendations, key considerations include defining research objectives, selecting appropriate study designs, and ensuring adequate statistical power. Through adherence to PRISMA-S guidelines for rapid reviews, the article emphasizes the importance of transparency and rigor in the review process. This commitment to methodological rigor enhances the credibility and trustworthiness of the insights presented.

Conclusion

The review suggested that the sample size should be considered as soon as possible throughout the research phase to gather more insightful background that will fundamentally have a stronger influence on pedagogic application. All types of research investigations require the determination of sample size, and selecting the appropriate formula is essential. According to the study's main goal, outcome variable, study plan, intended statistical investigation, study groups, and assorting procedure to be utilized, a suitable sample size formula was chosen. The sample population needed for a study is determined by a variety of variables, including the feasibility of the study, its power, the accuracy of the calculated value, its analytical relevance and confidence level, its ability to detect a clinically significant difference, and other factors, such as financial support, workforce, subject availability, and time. Studies involving cluster randomization require a larger sample size and a complex method for calculations. The sample size for conducting any new method basically required 24.24 members in each group. The median sample size for the simulation-based educational research was 30. Further, more research is needed for the appropriate sample size and universal single formula based on every study design.