Reviewer # 1:

Abstract

Background    -   Redundant phrasing: "two observersers" is misspelled and repetitive.

We corrected the typographical error and improved the phrasing for clarity in the Background section. The revised sentence now reads:

“In basic motor skills evaluation, two observers may rate the same child’s performance differently, introducing variability in the assessment”

Results             -   Tracking of "10 other publications" is unclear and vague.

We revised the sentence in the Results section to provide a more precise description of the process. The updated text now reads:

“From the 7 technology development studies, we examined their citation networks using Google Scholar and identified 10 subsequent peer-reviewed publications that either enhanced the original technologies or applied them in new research contexts. ”

                          -   Overuse of technical language without explanation for general readers (e.g., "engineering criteria").

We revised the sentence to reduce technical jargon and make it more accessible to general readers. In the corresponding section, the updated sentence now reads:

“The validation of these algorithms was based on engineering standards, focusing on their accuracy and technical performance, but without integrating medical and psychological knowledge about children's motor development.”

Keywords       -   Missing potential keywords like "assessment tools," “functional abilities,” and    "gross motor” based on the ‘title and the contents of the study

We have updated the list of keywords to better reflect the title and content of the study. The following terms have been added:

“assessment tools,” “functional abilities,” and “gross motor.”

Introduction: suggestions

Overly Technical Language:

While the introduction is generally informative, it becomes very technical in certain sections, especially when describing the AI process (e.g., "Fast Fourier Transformation," "principal components analysis," "linear discriminant analysis"). This might be difficult for readers unfamiliar with these terms or the AI field.

Suggestion: Simplify the explanation or provide brief definitions or context for these terms.

We have revised the third paragraph of the Introduction to provide brief definitions and clearer context for the mentioned techniques. Specifically, we now explain the role of Fast Fourier Transformation, principal components analysis, and linear discriminant analysis in simpler terms to aid reader understanding. The revised sentence reads as follows:

“Then, these data undergo pre-processing to reduce noise and enhance relevant features. This step often involves filtering techniques, such as Fast Fourier Transformation (which helps separate important movement signals from background noise) or wavelet transforms. Additionally, to simplify complex data and highlight key movement patterns, methods like principal components analysis (which reduces data dimensions while preserving essential information) or linear discriminant analysis (which enhances the distinction between movement categories) are applied (17).”

2. Lack of Clear Connection Between BMS and AI:

While the introduction discusses BMS, AI technologies, and their potential to address observer bias, the connection between these topics could be more explicit. The introduction seems to jump between BMS, traditional measurement tools, observer bias, and AI without a smooth flow that ties everything together.

Suggestion: Strengthen the connection between the problems with current BMS assessment and how AI could specifically address them. A clearer explanation of how AI can solve observer bias and improve accuracy in BMS assessment would help make the argument more compelling.

We agree that the connection between BMS, traditional measurement tools, observer bias, and artificial intelligence (AI)-related technology could be more explicit. In response to your suggestion, we have made adjustments to the third paragraph of the Introduction to improve the flow and strengthen the connection between these topics.

Specifically, we added a clearer explanation of how AI can directly address observer bias and improve accuracy in BMS assessment. We now describe how AI, by automating the motion classification process, reduces human subjectivity and enables more objective and accurate evaluations.

In this way, we aim to reinforce the argument that AI-related technology not only offers an alternative to traditional methods but effectively addresses the limitations of current BMS assessment approaches.

3. Limited Emphasis on the Scope of the Problem:

The issue of observer bias and its impact on BMS assessment is raised, but it’s not explored in-depth. The extent of the problem (how often it happens, how significant the impact is) isn’t fully explained.

Suggestion: Provide more concrete examples or data to highlight the real-world implications of observer bias in BMS assessments. This would help emphasize the need for better solutions like AI.

We have strengthened the second paragraph of the Introduction by incorporating concrete data that illustrate the prevalence and real-world consequences of this issue. Specifically, we now include evidence from two reviews: one indicating that, among 960 behavioral studies, only 3.2% reported interobserver reliability measures and only 1.9% met rigorous criteria for minimizing bias (12); and another highlighting that in child development research, poor reporting and variability in assessor performance may obscure children’s true developmental status, potentially compromising clinical decisions (13).

These additions aim to clarify the extent and significance of observer bias, reinforcing the necessity of adopting more objective tools—such as AI-based technologies—for accurate and reliable BMS assessment.

4. Vague Description of "AI-Related Technologies":

The term “AI-related technologies” is introduced, but it remains somewhat vague. While the steps of motion recognition are detailed, it’s not entirely clear what specific AI tools or algorithms are most effective in this context or how these methods directly translate to motor skill assessment in children.

Suggestion: Clarify the specific AI tools used or might be used in BMS assessment. This could make the introduction feel more grounded in practical applications.

We have revised the beginning of the third paragraph of the Introduction to explicitly define “AI-related technologies” and specify the tools most relevant for BMS assessment. The revised text reads as follows:

“AI-related technologies (i.e., computational systems that use artificial intelligence to analyze, learn from, and interpret data) offer a promising alternative to minimize observer bias in BMS assessment (14). For example, for motion capture and analysis, computer vision tools such as OpenPose, MediaPipe and DeepLabCut enable pose estimation and tracking of key points of the human body with high accuracy (15). In addition, deep learning techniques, such as Convolutional Neural Networks (CNN) and vision-specialized Transformer Models (ViT), have proven to be effective in classifying motion sequences in videos (16). In that sense, these AI-related technologies for recognizing and classifying human motion patterns consist of several steps (Figure 1) (17).”

5. Inconsistent Flow and Structure:

The flow between sections could be smoother. For example, the transition from talking about BMS instruments (TGMD-2, BOT-2) to observer bias is a bit abrupt. Similarly, after discussing the issue of observer bias, the leap to the technicalities of AI feels disconnected.

Suggestion: Reorganize the content so that each section builds more naturally on the previous one. Consider adding a paragraph that ties observer bias to the introduction of AI as a solution before jumping into the technical aspects.

We appreciate your feedback on the flow and structure of the manuscript. Our team has improved the text to facilitate smooth transitions between sections.

Specifically, the first part of the paragraph now reads: " Typically, BMS assessment relies on trained professionals who observe, record, and score children's performance on specific motor tasks (8,9). However, a major challenge in this approach is observer bias. Even when raters receive standardized training, small differences in scoring can introduce variability in BMS measurements. This variability reduces the accuracy of the assessment and can lead to misinterpretations. For example, two children with similar motor skills may receive different scores depending on the assessor, resulting in inconsistent results. When these inconsistencies follow a systematic pattern, they contribute to observer bias, a well-documented source of measurement error (10,11).".

We have also added a transitional paragraph at the end of the second paragraph, linking the problem of observer bias to the introduction of AI technologies as a solution. This paragraph highlights how observer bias can affect BMS assessment results, and the following paragraph addresses how AI-related technologies can mitigate this problem, offering a more objective and accurate alternative. The text reads: "In fact, one review reported that of 960 behavioral studies, only 3.2% reported measures of interobserver reliability, and only 1.9% met rigorous criteria for minimizing bias (12). Similarly, another review on child development found that the quality of reporting on the use of assessors in these studies was poor and that variability in assessor performance may obscure the true developmental status of children, compromising complex and costly clinical decisions (13)”.

6. Missing Emphasis on the Target Age Group (3-6 Years):

The introduction discusses BMS in general terms but doesn’t emphasize the importance of assessing motor skills in children aged 3 to 6 years. Since this is a specific target group, more focus on why this age range is critical for BMS development would strengthen the relevance of the study.

Suggestion: Provide a brief explanation of why this age group is the focus and what makes BMS development particularly crucial at this stage.

We have added a brief explanation in the first paragraph of the introduction that highlights why this age group is crucial for BMS development. Specifically, we have pointed out that during this period of rapid physical and motor development, children refine fundamental skills that are essential for later complex activities, which directly impacts their cognitive, emotional, and social development. In addition, we underscored the relevance of early assessment to identify potential delays and facilitate timely interventions.

Now the text says the following: "The development of basic motor skills (BMS) in children aged 3 to 6 years is critical, as this is a period of rapid motor growth, where children acquire physical skills that allow them to participate in a variety of activities (1,2). At this age, children experience significant improvements in gross motor control, allowing them to perform movements such as running, jumping, and manipulating objects with greater precision (3,4). The acquisition of these motor skills is essential for physical, cognitive and emotional development, as BMS are strongly linked to general well-being, self-esteem and social integration (5)."

7. Lack of Citations for Claims About AI in Other Fields:

There are a few claims about AI applications in other fields (e.g., assessing physical activity in adults, recognizing walking alterations), but not all of these are fully supported by citations. The mention of “a recent review” and other general statements could be more precise.

Suggestion: Provide specific references or clarify that the claims about AI in other fields are based on the literature review cited earlier.

We have carefully reviewed the claims about AI applications and ensured that each is supported by an appropriate reference. In addition, we have revised the general claims to improve their accuracy with the cited literature.

Specifically, we have updated citations 4, 5, 6, and 7 with their respective references:

4. Jiang G-P, Jiao X-B, Wu S-K, Ji Z-Q, Liu W-T, Chen X, et al. Balance, proprioception, and gross motor development of Chinese children aged 3 to 6 years. J Mot Behav. 2018;50(3):343–52. Available at: http://dx.doi.org/10.1080/00222895.2017.1363694

5. Gandotra A, Kotyuk E, Bizonics R, Khan I, Petánszki M, Kiss L, et al. An exploratory study of the relationship between motor skills and indicators of cognitive and socio-emotional development in preschoolers. Eur J Dev Psychol. 2023;20(1):50–65. Available at: http://dx.doi.org/10.1080/17405629.2022.2028617

6. Bremer E, Cairney J. Fundamental Movement Skills and Health-Related Outcomes: A Narrative Review of Longitudinal and Intervention Studies Targeting Typically Developing Children. Am J Lifestyle Med. 2018;12(2):148-59. https://doi.org/10.1177/1559827616640196

7. Eddy LH, Wood ML, Shire KA, Bingham DD, Bonnick E, Creaser A, et al. A systematic review of randomized and case-controlled trials investigating the effectiveness of school-based motor skill interventions in 3- to 12-year-old children. Child Care Health Dev. 2019;45(6):773-90. https://doi.org/10.1111/cch.12712

8. No Clear Statement of the Research Gap:

While the introduction implies a gap in the literature (i.e., a lack of formal review of AI in BMS assessment for children), this could be more explicitly stated and emphasized. It could be clearer why this review is necessary and what new insights it might offer.

Suggestion: Make the research gap more prominent by rephrasing the last couple of sentences to more forcefully explain why this scoping review is essential.

We have revised the last two final sentences of the introduction to highlight the research gap in the use of AI to accurately assess BMS in children and the need for this scoping review. Specifically, we now emphasize the lack of a comprehensive review of AI applications for assessing basic motor skills (BMS) in preschool children, as well as the need to systematize the scope, limitations, and validity of these technologies.

Specifically, we have added the following: "However, despite the increasing use of AI in motor performance assessment, there is no comprehensive review examining its specific application in the assessment of BMS in preschool children, being a crucial stage for early detection and intervention. Furthermore, the scope, limitations and validity of AI-based technologies in this context are not yet clearly systematized. Therefore, it is required to synthesize existing knowledge and guide the development of more accurate and accessible assessment tools"

9. Repetitive Mention of AI:

The introduction repeatedly mentions “AI-related technologies,” which could be streamlined. The term is used several times in close proximity without adding new information each time, which can feel redundant.

Suggestion: Try to vary the phrasing (e.g., "machine learning tools," "AI classifiers") to keep the introduction engaging and avoid repetition.

Our team has made sure to review and update the text to avoid redundancy of the term “AI-related technologies”. We have also decided to keep the term “AI-related technologies” consistently throughout the introduction to ensure clarity and consistency throughout the manuscript. In fact, in the third paragraph, we have included a clarification referring to the term “AI-related technologies”.

Specifically, in the first three lines of the third paragraph in the introduction, we have added: "AI-related technologies (i.e., computational systems that use artificial intelligence to analyze, learn from, and interpret data) offer a promising alternative to minimize observer bias in BMS assessment (14)"

10. Unfinished Sentence in Objective Statement:

The final sentence of the introduction (before the objectives) seems cut off. This makes the last thought feel incomplete and leaves the reader hanging.

Suggestion: Complete the sentence and ensure that all points are concluded before introducing the research objectives.

We have reviewed and revised this section to ensure the idea is fully expressed and concluded before presenting the research objectives. Now, in the final lines of the fourth paragraph of the introduction, we have added the following:

“However, despite the increasing use of AI in motor performance assessment, there is no comprehensive review examining its specific application in the assessment of BMS in preschool children, being a crucial stage for early detection and intervention. Moreover, the scope, limitations and validity of AI-based technologies in this context are not yet clearly systematized. Therefore, it is required to synthesize existing knowledge and guide the development of more accurate and accessible assessment tools”

Methods and discussion:

1. Lack of Detail in Study Selection Criteria

While the article clearly defines the types of studies it targeted (engineering, substantive validation, and use of AI-related technologies), it doesn’t provide much detail on what specific inclusion and exclusion criteria were applied beyond these categories. For example, were studies excluded for reasons like sample size, study design, or methodological rigor?

Suggestion: Provide more specific inclusion and exclusion criteria. Were only randomized controlled trials considered? Were there any restrictions based on the publication type (e.g., peer-reviewed articles only)?

We have revised the manuscript and added more detailed inclusion and exclusion criteria. Specifically, in the third paragraph of methods, we added: "We also defined the following criteria for the search: 1) studies in preschool-aged children (3 to 6 years), 2) studies in which the motor ability (motor or play skills) of the child was assessed using AI-related technologies for motion detection, and 3) studies in which at least one of the basic motor skills described in the literature (running, jumping, kicking, throwing, or catching a ball) was measured. In addition, we excluded 1) studies that did not clearly describe the AI-related technology used or developed, 2) opinion articles, editorials, or narrative reviews without empirical data and 3) gray literature (e.g. theses, dissertations, or non-peer-reviewed reports)."

2. Vague Explanation of Search Strategy

The search strategy mentions specific databases but does not explain the rationale for selecting these particular sources over others. Are there other databases relevant to the field that might have been excluded? How were the keywords selected, and were any synonyms or related terms considered?

Suggestion: Provide a clearer justification for the selection of these particular databases and search terms. Did you consider any grey literature (e.g., theses, dissertations, reports from non-peer-reviewed sources) to broaden the search?

We've added clearer justification for our selection of databases and search terms. Specifically, we added in the methods section, subsection Search strategy, the following two paragraphs:

" We searched for studies published before January 30, 2023 in the target publications in Medline (SCR_002185), Web of Science (SCR_022706), IEEE (SCR_008314), and EBSCO (SCR_022707). These databases were selected because they specialize in biomedical, engineering, and multidisciplinary research, ensuring that we captured relevant studies in health sciences, AI applications, and motion analysis.

Search terms included keyword combinations such as “child,” “preschool,” “basic motor skills,” “artificial intelligence,” “motion sensing,” and “calibration,” along with related terms and synonyms identified through a preliminary literature review (keywords) and controlled vocabulary (MeSH terms). The full search strategy and complete list of search terms are available here (42)."

Likewise, we cite the complete search strategy as number 42.

Details on the Rayyan Platform

The mention of the Rayyan platform for managing studies is helpful, but the text doesn't clarify whether Rayyan was used for the full review process or just the initial screening. The review process appears to be manual in nature, but it could benefit from some details on how disagreements were handled between reviewers (e.g., was there a consensus meeting or did one reviewer have final say on the study’s inclusion?).

Our team has detailed how Rayyan was used during the review process. Specifically, in the methods section, subsection Search strategy, in the third, fourth and fifth paragraph, we add:

"The search formulas were applied to the databases and all the files were exported in RIS format. Then, to ensure an objective selection process, these identified files were uploaded to the Rayyan platform which facilitated blind selection by the reviewers and expedited the identification of duplicates.

The selection process consisted of two phases. In the first phase, titles and abstracts were reviewed by two independent groups (each consisting of two previously trained medical students). To minimize selection bias, the Rayyan blinding function was used, which prevented reviewers from identifying the decisions of the other reviewers until the final selection phase. In addition, allocation of studies to reviewers was randomized within each group to further reduce potential bias. In case of disagreement, a consensus discussion was held among the reviewers. If consensus could not be reached, the principal investigator made the final inclusion decision.

In the second phase, a full-text review was performed following the same procedure, ensuring consistency and methodological rigor. The final set of studies was determined after resolving all discrepancies through consensus discussions and the intervention of the principal investigator."

Potential Bias in Study Selection

Suggestion: Mention strategies to reduce bias in study selection. For example, were studies randomly assigned to reviewers, and was there a protocol to ensure that preconceived notions didn’t influence the selection process?

Our team has added information about the process to minimize bias during study selection. Specifically, in the methods section, Search Strategy subsection, in the fourth and fifth paragraphs, we add:

"The selection process consisted of two phases. In the first phase, titles and abstracts were reviewed by two independent groups (each consisting of two previously trained medical students). To minimize selection bias, the Rayyan blinding function was used, which prevented reviewers from identifying the decisions of the other reviewers until the final selection phase. In addition, allocation of studies to reviewers was randomized within each group to further reduce potential bias. In case of disagreement, a consensus discussion was held among the reviewers. If consensus could not be reached, the principal investigator made the final inclusion decision.

In the second phase, a full-text review was performed following the same procedure, ensuring consistency and methodological rigor. The final set of studies was determined after resolving all discrepancies through consensus discussions and the intervention of the principal investigator."

5. Limited Details on Data Extraction Process

While the data extraction form is outlined with categories such as general information, engineering, substantive validation, and use, there is little detail on how the data were extracted from the studies. For example, was any kind of reliability check conducted between reviewers, or were discrepancies resolved through discussion?

Suggestion: Provide more details on the data extraction process, such as whether two independent reviewers performed the extraction and how discrepancies were resolved. Additionally, were any statistical methods used to assess inter-rater reliability in the extraction process?

We have updated the paragraph by adding information in the methods section, Data extraction subsection. Now the text says the following:

"Data extraction was performed in a structured manner using a pre-designed form (43). To reduce errors and improve the accuracy of the extracted data, one peer reviewer performed the initial extraction and a second peer independently verified the information. Any discrepancies in the extraction were reviewed jointly and/or, with the intervention of the principal investigator. Cross-checks were implemented to ensure the consistency of the information collected."

Lack of Clarity on Data Analysis

The description of data analysis mentions the use of descriptive statistics (frequencies and percentages) and the COSMIN standards for evaluating psychometric properties. However, there is no clear explanation of how the data were analyzed or how the results were synthesized. Were any meta-analysis or qualitative synthesis methods considered? How were the studies compared and summarized?

Suggestion: Provide more clarity on how the data were synthesized. Was any quantitative or qualitative analysis beyond simple descriptive statistics performed, such as meta-analysis or thematic analysis? Did you perform a subgroup analysis for specific types of AI technologies or specific study characteristics?

Since our study was a scoping review, we did not conduct a meta-analysis. Instead, we opted for a narrative synthesis of the results, presented in tables of frequencies and percentages. We also used the COSMIN standards, whose scoring and rating process is described in their original report, which we also reference in our study.

Specifically, our text, in the methods section, subsection "Data analysis," in its first paragraph, states the following: "All data collected were summarized as categorical variables, organized and presented in tables, using descriptive statistics such as simple frequencies and percentages. Since this was a scoping review, a narrative synthesis was used to summarize the findings of the studies, focusing on the characteristics and psychometric properties evaluated according to COSMIN standards."

Lack of Discussion on Potential Confounding Variables

There is a brief mention of the use of psychometric standards (COSMIN) to evaluate AI technologies, but the article doesn't discuss how confounding factors (e.g., differences in sample size, age groups, or types of technology used) might impact the validity of the conclusions. Were these factors taken into consideration in the evaluation process?

Suggestion: Acknowledge potential confounding variables and how they might affect the validity of the studies reviewed. How did the authors control for these variables, if at all, in the synthesis process?

We would like to clarify that, since our study is a scoping review focused on the mapping and characterization of AI-based technologies used to assess motor skills in children, we did not aim to establish causal or correlational relationships between variables. Therefore, the methodological approach does not involve controlling or statistically analyzing confounding variables, as would be expected in primary empirical studies. The purpose of this review was to identify and describe the technologies developed, their applications, and the reported psychometric dimensions.

Nonetheless, we have incorporated a critical reflection on the differences in sample sizes, age ranges (expressed in both years and months), and the types of technologies used, acknowledging these as limitations that could influence the validity and generalizability of the findings reported by the included studies.

Specifically, the last lines of the limitations section say the following: "This review did not aim to analyze associations between variables; however, variability in sample sizes, age ranges, and types of AI-based technologies used across studies may affect the comparability and generalizability of the findings. These differences should be considered when interpreting the results and highlight the need for more standardized approaches in future research."

8. Unclear Rationale for Data Collection Methods

In the section about data extraction, the mention of "feasibility and usability" as part of the data form is important but lacks further context. Was there a specific framework used to evaluate these factors? Were the usability and feasibility criteria standardized across the included studies?

Suggestion: Provide more information on the frameworks or criteria used to assess usability and feasibility. Were these evaluations subjective or based on standardized measures? Clarifying this would help readers understand how the data extraction process assessed these aspects.

Discussion   -    Discuss usability, cost-effectiveness, and accessibility challenges. -Emphasize the need for interdisciplinary collaboration in future validation studies. -Propose specific psychometric properties (e.g., construct validity) to prioritize  future AI validation research. - Add practical recommendations for improving AI tools (e.g., integrating psychometric standards, improving usability for non-specialists).

Since no standardized framework was used to assess feasibility and usability, this is an aspect that limits the interpretation and comparison of the results among the included studies. We state this information in the limitations.

Specifically, we added to the beginning of the limitations:: "Also, although this review was based on COSMIN standards to assess the psychometric quality of AI-related technologies, due to the heterogeneity observed in the included studies, no specific adjustments were made to control for possible confounding variables. Therefore, the conclusions need to be interpreted with caution. It is recommended that future research address these factors and use control methods to provide more generalizable conclusions. Furthermore, feasibility and usability were extracted only if the reviewed studies explicitly reported having done so in their analysis of AI-related technologies. Therefore, further studies should evaluate these analyzes using a standardized framework."

In response to comments on discussion, we have expanded this section on implications. Specifically, we added the following paragraphs in implications:

"To facilitate use, developers could conduct studies that evaluate the acceptance, ease of use, cost-effectiveness, and accessibility of these technologies. For example, most technologies rely on sensors and monitors that, while accurate, can be costly, require specialized training, and can be difficult to implement in real-world settings for physicians, teachers, therapists, or practitioners unfamiliar with these tools. In addition, disparities in access to advanced technologies may limit their adoption, particularly in low-resource settings.

Also, these types of technologies may be closer to more universal and cost-effective devices, such as video cameras, smartphones, and tablets, that can assess and report motor skills in real time. However, addressing these challenges requires a collaborative and interdisciplinary approach. Future validation studies should involve experts from multiple fields, including engineers, healthcare professionals, educators and policy makers, to ensure that these technologies are not only accurate, but also practical, scalable and accessible to diverse populations.

New validation studies of these technologies should include validation standards for BMS tests, prioritizing key psychometric properties such as construct validity, criterion validity, reliability, measurement error, among others. To make this possible, engineering teams could incorporate specialists in psychometrics, developmental therapy and medicine to work collaboratively. This multidisciplinary approach will facilitate the integration of medical knowledge and psychometric standards into future software releases, improving both measurement accuracy and practical usability. Finally, developers should consider providing open source code or detailed methodological documentation, which will allow for further refinement, replication, and clinical adaptation of these technologies in future research and real-world applications."