Convergent validity and reliability of a novel repeated agility protocol in junior rugby league players

Anthony Nicholls; Anthony Leicht; Jonathan Connor; Aaron Halliday; Kenji Doma

doi:10.12688/f1000research.23129.3

Home Browse Convergent validity and reliability of a novel repeated agility protocol...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Revised

Convergent validity and reliability of a novel repeated agility protocol in junior rugby league players

[version 3; peer review: 2 approved]

Anthony Nicholls¹, Anthony Leicht¹, Jonathan Connor¹, Aaron Halliday², Kenji Doma ¹

Anthony Nicholls¹, Anthony Leicht¹, [...] Jonathan Connor¹, Aaron Halliday², Kenji Doma ¹

PUBLISHED 22 Nov 2021

Author details Author details

¹ Sport & Exercise Science, James Cook University, Douglas, Queensland, 4814, Australia
² Physical Education, Kirwan State Highschool, Kirwan, Queensland, 4817, Australia

Anthony Nicholls
Roles: Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Anthony Leicht
Roles: Conceptualization, Formal Analysis, Methodology, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Jonathan Connor
Roles: Methodology, Project Administration, Supervision, Writing – Review & Editing

Aaron Halliday
Roles: Methodology, Project Administration, Supervision, Writing – Review & Editing

Kenji Doma
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Software, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: : Rugby league involves repeated, complex, and high intensity change-of-direction (COD) movements with no existing test protocols that specifically assesses these multiple physical fitness components simultaneously. Thus, the current study examined the convergent validity of a repeated Illinois Agility (RIA) protocol with the repeated T-agility protocol, and the repeatability of the RIA protocol in adolescent Rugby League players. Furthermore, aerobic capacity and anaerobic and COD performance were assessed to determine whether these physical qualities were important contributors to the RIA protocol.
Methods: Twenty-two junior Rugby League players completed 4 sessions with each separated by 7 days. Initially, physical fitness characteristics at baseline (i.e., Multi-stage Shuttle test, countermovement jump, 30-m sprint, single-effort COD and repeated sprint ability [RSA]) were assessed. The second session involved a familiarisation of RIA and repeated T-agility test (RTT) protocols. During the third and fourth sessions, participants completed the RIA and RTT protocols in a randomised, counterbalanced design to examine the validity and test-retest reliability of these protocols.
Results: For convergent validity, significant correlations were identified between RIA and RTT performances (r= >0.80; p<0.05). For contributors to RIA performance, significant correlations were identified between all baseline fitness characteristics and RIA (r = >0.71; p < 0.05). Reliability of the RIA protocol was near perfect with excellent intra-class correlation coefficient (0.87-0.97), good ratio limits of agreement (×/÷ 1.05-1.06) and low coefficient of variations (1.8-2.0%).
Conclusions: The current study has demonstrated the RIA to be a simple, valid and reliable field test for RL athletes that can provide coaches with information about their team’s ability to sustain high intensity, multi-directional running efforts.

Keywords

Change of direction, anaerobic power, sprint, speed, recovery rate

Corresponding author: Kenji Doma

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2021 Nicholls A et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Nicholls A, Leicht A, Connor J et al. Convergent validity and reliability of a novel repeated agility protocol in junior rugby league players [version 3; peer review: 2 approved]. F1000Research 2021, 9:624 (https://doi.org/10.12688/f1000research.23129.3) First published: 17 Jun 2020, 9:624 (https://doi.org/10.12688/f1000research.23129.1) Latest published: 22 Nov 2021, 9:624 (https://doi.org/10.12688/f1000research.23129.3)

Revised Amendments from Version 2

Included "multi-state shuttle test" in the Abstract and changed the sub-headings from "agility" to "change-of-direction" in the Methods section.

See the authors' detailed response to the review by James Dugdale
See the authors' detailed response to the review by Aaron T. Scanlan

Introduction

Rugby League (RL) is an intermittent, invasion type game that requires players to complete repetitive bursts of sprinting and change-of-direction (COD) movements in response to the dynamic constraints of the game¹. Traditionally, the physical component of COD has been assessed using protocols with a single bout approach for the COD performance measure considered a strong determinant of match-performance in team sports^2–5. However, team sports, such as RL, encounter repeated bursts of COD movements to defend or evade defenders during a game⁶. Consequently, performance of repeated-COD activities with brief periods of rest may be an important performance component necessary for RL athletes.

As a monitoring tool, the reliability of repeated-COD protocols have been explored in a variety of sports^7–9. Results from a study examining a Repeated T-Test (RTT) agility protocol in soccer players significantly correlated with anaerobic measures of power, speed and repeat-sprint ability (RSA), with excellent test-retest reliability⁹. While a good indicator of COD performance, the Agility T-Test consists of a linear sprint, lateral shuffles and a backwards run, which are movements that are sporadic in RL⁹. In fact, RL players change direction frequently and utilise evading movements¹⁰ that are not replicated by the Agility T-Test. Therefore, the Illinois Agility test may be more reflective of the evading activities undertaken in RL, as the protocol includes vigorous changes in direction by weaving in and out of cones¹¹. Furthermore, the majority of studies examining the reliability of repeated-COD protocols have been conducted in adult athletes, despite a previous study reporting lower reliability in younger athletes^12,13. Thus, research is warranted examining the reliability of repeated COD-protocols in adolescent athletes.

In addition to reliability, there has been limited investigations into validating repeated-COD protocols. Indeed, the RTT protocol was significantly correlated with anaerobic measures of power, speed and repeat-sprint ability (RSA)⁹, suggesting that these physical qualities were pertinent for repeated-COD performance. However, separate repeated-COD protocols have yet to be compared, which is essential as each COD protocol exhibits distinct movement demands that may be suitable for specific sports. To date, no studies have examined the validity and reliability of a repeated Illinois Agility (RIA) protocol. Reporting these properties would be essential for widespread usability in RL⁴.

The aims of this study were three-fold: 1) to examine the convergent validity of a novel RIA test with the repeated Agility T-test protocol (i.e. RTT); 2) to identify contributors of RIA performance by correlating its measures to speed, anaerobic capacity and aerobic capacity; and 3) to determine the test-retest reliability of the RIA protocol. It was hypothesised that the RIA would demonstrate acceptable convergent validity and reliability as a repeated-COD test, with relationships identified between results of the RIA and the RTT, aerobic capacity, speed, and anaerobic capacity protocols. Examining the convergent validity of the RIA protocol will determine whether this novel assessment exhibits similar attributes to a standardised COD protocol (i.e., RTT). In addition, the reliability of the RIA will determine whether this test can be reliably adopted in practice by accounting for the inherent error of the test across repeated measurements. The quality of these psychometric properties will provide coaches with a tool to assist in monitoring and training RL athletes as well as in talent development and identification.

Methods

Research design

The current study was a randomised, counter-balanced study conducted across five sessions from June, 2018 to August, 2018 (Figure 1). During the first session, the participants completed a Multistage Shuttle test to determine predicted maximal aerobic capacity (VO_2max)¹⁴. The second session was utilised to obtain baseline assessments of speed (30-metre sprint), COD (Illinois Agility test, Agility T-Test) and repeat-sprint ability (RSA). The third session familiarised participants with the RTT and RIA tests. During the fourth and fifth sessions, participants undertook both the RIA and RTT, in randomised order, with at least 15-minutes of recovery between each protocol.

Figure 1. Schematic of the research design consisting of four sessions, including best-effort agility, speed, repeat sprint ability (RSA), repeated T-test agility (RTT) and repeated Illinois agility (RIA) measures.

At the start of each session, muscle soreness rating was collected prior to performing a standardised warm up, using a 1–10 visual analogue scale, with 1 and 10 indicating ‘no soreness’ and ‘very, very sore’, respectively¹⁵. Participants then performed a standardised warm-up consisting of jogging for 3–5 minutes and 15-metre sprints at 50%, 70% and 100% of maximal effort. A countermovement jump (CMJ) test (Yard Stick, Swift Performance, Queensland, Australia) was then conducted to assess leg power¹⁶, which was also repeated before the second COD test to confirm recovery between the repeated-COD tests.

Participants

In total, 22 adolescent, male, RL players (age 16.2 ± 0.8 yrs; body mass 80.7 ± 16.3 kg; height 1.77 ± 0.7 m) were recruited via word of mouth, flyers and liaison with sporting teams. The participants were part of the School of Athletic Excellence program, which selects and prepares students to compete at state and national competitions. The participants were injury-free with at least 2 years of RL experience. According to an a priori calculation¹⁷, a sample size of 22 was sufficient to identify significant differences in repeated-COD performance (power of 80%, alpha level of 0.05). Participants were instructed to avoid strenuous physical activity and caffeine for up to 12 hours before each testing session. All protocols were approved by the Institutional Human Research Ethics Committee and written informed consent was received from the participants and their parent/guardian prior to partaking in this study (Approval number H7248).

Multistage shuttle test

For the Multistage Shuttle test, participants ran back and forth in time with a series of audio signals on a 20-m indoor court in time with a series of audio signals¹⁴. The time between audio signals progressively decreased during the test resulting in an increased effort and running speed for athletes each minute. Predicted VO_2max was estimated based on the level completed, using a previously developed regression equation¹⁴.

Countermovement jump test

The countermovement jump protocol was measured with a vertical jump apparatus, based on 1-cm increments, with the units of measure reported in cm (Yard Stick, Swift Performance, Queensland, Australia). To ensure standardisation of the countermovement jump test, participants were instructed to draw their arms backwards upon the eccentric phase, then swing the arms forward during the concentric phase to gain momentum and maximise the stretch-shortening cycle mechanics¹⁸. The participants attempted three countermovement jumps, with approximately 30–60 seconds of rest in-between, and the highest jump reported.

30-m Sprint and Change-of-direction protocols

Assessment of speed was achieved by completing 30-m maximal sprints. The Agility T-test protocol was set up within a 10-m × 10-m figure-T course (Figure 2A)¹⁹. The Illinois Agility protocol consisted of a 10-m × 5-m course (Figure 2B)^4,20. To ensure protocol familiarity, the participants completed three trials at sub-maximal effort followed by one final maximal trial, with each trial interspersed by two minutes of recovery. Trial completion times were recorded using an electronic timing gate system (Speedlight Timing Gates, Swift Performance, Australia) positioned at the start/finishing line with the same height and distances between the gates for each test and reported in seconds. The fastest time was used for later analysis.

Figure 2.

Schematic of A) T-Test Agility and B) Illinois Agility protocols.

Repeat Sprint and Change-of-direction Protocols

The RSA, RTT and RIA protocols were completed by repeating the previously described protocols (i.e. 30-m sprint, T-test and Illinois Agility, respectively) across 6 repetitions with varying recovery periods in-between each repetition. Specifically, each repetition within the RSA, RTT and RIA was separated by 20-, 35- and 60-second recovery, respectively, with work-to-rest ratios of approximately 1:3^8,9. The participant’s instantaneous heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions²¹. The following parameters were also calculated for each repeated agility protocol: total time (TT) of 6 cycles, best cycle time (BT), the average cycle time (AT) and fatigue index (FI)⁸. FI was calculated as follows⁹:

F a t i g u e I n d e x = ((\frac{T T}{B T \times 6}) \times 100) - 100

Statistical analysis

Data was analysed using a statistical software (IBM SPSS version 25, Chicago, Illinois) and reported as mean ± standard deviation. Normality of the data was assessed using the Kolmogorov-Smirnov statistic. Construct validity of the repeated-COD protocols was identified via Pearson’s product moment correlation coefficients for RTT and RIA measures (i.e., TT, BT, AT and FI) and construct validity with aerobic capacity, leg power, speed and COD variables (i.e., VO_2max, CMJ, 30-m sprint time, T-Test and Illinois Agility, respectively) were assessed as contributors to the RIA protocol. The cut-off for acceptable convergent validity and contributors to the RIA protocol was established when the association was statistically significant with an r-value of ≥ 0.70^22,23. Reliability of the repeated-COD measures was determined via a paired T-test, intraclass correlation coefficients (ICC, SPSS 2-way mixed, 95% confidence intervals), coefficient of variation (CV, 95% confidence intervals) and systematic bias/ratio with 95% limits of agreement (LOA)²⁴. Where significant relationships existed between the mean difference and average of test-retest values (i.e. heteroscedastic errors), variables were transformed (natural logarithm) prior to the calculation of measurement bias/ratio × / ÷ ratio LOA²⁵. The level of significance for all analyses was set at 0.05. Finally, effect size (Cohen’s d) with 95% CI was used to calculate the magnitude of differences in muscle soreness and CMJ measures between RIA and RTT protocols to determine whether the recovery periods were appropriate. The ES classifications were set as small, moderate and large with values of 0.2, 0.5 and 0.8, respectively (Cohen, 1988).

Results

For convergent validity, significant correlations were identified between RIA and most RTT variables (Table 1²⁶). For contributors to RIA performance, significant correlations were identified with RSA, 30-m sprint time, best effort agility measures, aerobic capacity and CMJ height (Table 2²⁶).

Table 1. Relationship between performance measures of the repeated Illinois agility test (RIA), repeated T-agility test (RTT) and repeated sprint ability (RSA).

	RIA				RTT
	TT (s)	BT (s)	AT (s)	FI (%)	TT (s)	BT (s)	AT (s)	FI (%)
RSA
TT (s)	0.80***	0.73***	0.80***	0.55***	0.70**	0.51*	0.70**	0.55**
BT (s)	0.63***	0.51***	0.63***	0.56***	0.71**	0.54*	0.71**	0.50*
AT (s)	0.80***	0.73***	0.80***	0.55***	0.70**	0.51*	0.70**	0.55**
FI (%)	0.48**	0.61***	0.49**	0.28	0.28	0.13	0.28	0.37
RTT
TT (s)	0.84***	0.81***	0.84***	0.51***	–	–	–	–
BT (s)	0.81***	0.81***	0.81***	0.44***	–	–	–	–
AT (s)	0.84***	0.80***	0.84***	0.51***	–	–	–	–
FI (%)	0.43**	0.32	0.43**	0.48**	–	–	–	–

TT = total time; BT = best time; AT = average time; FI = fatigue index

*P<0.05 **P<0.01 ***P<0.001

Table 2. Pearson correlation coefficients between repeated performances, perceptual and physiological indices (repeated Illinois agility [RIA], repeated T-agility test [RTT]) with aerobic capacity, leg power, speed, and agility test performance measures.

	VO_2max (mL·kg^-1·min^-1)	CMJ (cm)	Sprint 30m (sec)	IA (sec)	TTA (sec)
RIA
TT (s)	-0.73**	-0.85**	0.89**	0.87**	0.72**
BT (s)	-0.71**	-0.79**	0.81**	0.86**	0.71**
AT (s)	-0.73**	-0.85**	0.89**	0.87**	0.72**
FI (%)	-0.43*	-0.57**	0.61**	0.49*	0.40
HR_Avg	-0.43	-0.09	0.34	0.24	0.38
HR_Max	-0.20	-0.03	0.12	0.04	0.21
RPE_Avg	0.17	0.17	-0.21	-0.16	-0.21
RPE_Max	-0.04	-0.23	0.27	0.35	0.21
RTT
TT (s)	-0.68**	-0.76**	0.80**	0.84**	0.80**
BT (s)	-0.65**	-0.74**	0.80**	0.86**	0.85**
AT (s)	-0.68**	-0.76**	0.80**	0.84**	0.80**
FI (%)	-0.41	-0.37	0.34	0.28	0.14
HR_Avg	-0.41	-0.05	0.26	0.13	0.23
HR_Max	-0.33	-0.68	0.21	0.06	0.24
RPE_Avg	0.00	0.02	-0.03	0.02	-0.07
RPE_Max	0.02	0.04	-0.03	0.03	-0.03

CMJ = countermovement jump; TT= total time; BT = best time; AT = average time; FI = fatigue index ; Sprint 30m = 30 metre sprint; IA = Illinois Agility test; TTA = T-test agility; VO_2max = maximal aerobic capacity , RPE_Avg= Average Rate of Perceived Exertion, RPE_Max = Maximum Rate of Perceived Exertion, HR_Avg= Heart rate average, HR_Max = Maximum heart rate *P<0.05 **P<0.01 ***P<0.001

Most RIA measures exhibited excellent test-retest reliability (ICC = 0.92-0.97), good levels of agreement (ratio LOA = 1.05-1.06) and low measurement error (CV = 2.17 - 2.68%) (Table 3) between Sessions 4 and 5. However, FI and average RPE demonstrated moderate test-retest reliability (ICC = 0.87 and 0.76, respectively), poorer levels of agreement (ratio LOA = 2.57 and 2.23, respectively) and higher measurement error (CV = 25.3 and 15.8%, respectively, Table 3).

For the RTT, excellent test-retest reliability (ICC = 0.91), good levels of agreement (ratio LOA = 1.08) and low measurement error (CV = 2.17-2.68%) were identified for TT, BT and AT (Table 3) between Sessions 4 and 5, although maximum RPE demonstrated higher levels of measurement error (CV = 12.3%) (Table 3). In addition, FI, average RPE and maximum HR displayed moderate to large reliability (ICC = 0.69 – 0.89), poorer agreement (ratio LOA = 1.10 – 2.59) and higher measurement error (CV = 2.38 – 27.6%) compared to the RIA protocol (Table 3).

Table 3. Test-retest results, intra-class correlation coefficients (ICC, 95% confidence interval (CI)), measurement bias/ratio (log-transformed data) (×/÷ 95% ratio limits of agreement (ratio-LOA)) and within-subject coefficient of variation (95 % CI) of the repeated Illinois Agility (RIA) and T-test (RTT) protocol.

	Test (s)	Retest (s)	p	ICC (95% CI)	CV% (95% CI)	Bias ratio-LOA
RIA
TT (s)	108.22 ± 9.14	107.38 ± 8.39	0.23	0.97 (0.92 - 0.99)***	1.97 (0.91-2.16)	1.01 ×/ 1.06
BT (s)	17.00 ± 1.03	17.05 ± 1.05	0.60	0.96 (0.90 - 0.98)***	1.77 (0.98-1.83)	1.00 ×/ 1.05
AT (s)	18.04 ± 1.52	17.90 ± 1.40	0.23	0.97 (0.92 - 0.99)***	1.97 (0.91-2.16)	1.01 ×/ 1.06
FI (%)	6.02 ± 3.50	4.91 ± 3.18	0.03†	0.87 (0.68 - 0.95)***	25.3 (22.9-40.1)	1.32 ×/ 2.57
RPE_Avg	4.9 ± 1.2	4.3 ± 1.7	0.07	0.76 (0.41-0.90)**	15.8 (6.1-25.6)	1.20 ×/ 2.23
RPE_Max	6.5 ± 1.6	6.1 ± 1.7	0.04†	0.93 (0.83-0.97)***	8.1 (4.8-11.7)	1.08 ×/ 1.34
HR_Avg (bpm)	183.8 ± 8.5	180.2 ± 10.2	0.09	0.92 (0.89-0.97)***	2.10 (1.48-2.72)	1.02 ×/ 1.06
HR_Max (bpm)	189.0 ± 8.3	188.3 ± 9.6	0.53	0.94 (0.83-0.98)***	1.31 (0.78-1.84)	1.00 ×/ 1.05
RTT
TT (s)	68.69 ± 4.79	69.01 ± 5.15	0.61	0.91 (0.79 - 0.96)***	2.68 (1.91-1.39)	1.00 ×/ 1.08
BT (s)	11.01 ± 0.7	11.06 ± 0.74	0.58	0.91 (0.78 - 0.96)***	2.17 (1.55-2.80)	1.00 ×/ 1.08
AT (s)	11.45 ± 0.80	11.50 ± 0.86	0.61	0.91 (0.79 - 0.96)***	2.68 (1.91-3.14)	1.00 ×/ 1.08
FI (%)	3.98 ± 1.68	3.97 ± 1.89	0.99	0.69 (0.25 - 0.87)**	27.6 (15.3-33.7)	1.03 ×/ 2.59
RPE_Avg	3.2 ± 1.2	3.6 ± 1.4	0.10	0.89 (0.73-0.95)***	15.3 (8.4-22.2)	0.91 ×/ 1.85
RPE_Max	4.4 ± 1.7	4.9 ± 1.8	0.02†	0.93 (0.84-0.97)***	12.3 (4.7-19.8)	0.90 ×/ 1.78
HR_Avg (bpm)	176.2 ± 7.5	174.0 ± 13.0	0.35	0.79 (0.45-0.92)**	2.88 (1.70-4.04)	1.02 ×/ 1.11
HR_Max (bpm)	186.3 ± 8.0	183.7 ± 10.0	0.20	0.68 (0.14-0.88)*	2.38 (1.06-3.69)	1.02 ×/ 1.10

TT= total time; BT = best time; AT = average time; FI = fatigue index, RPE_Avg = Average Rate of Perceived Exertion, RPE_Max = Maximum Rate of Perceived Exertion, HR_Avg = Heart rate average, HR_Max = Maximum heart rate P<0.05 **P<0.01 *** P<0.001

† Significantly different (p<0.05)

No significant differences were found for muscle soreness (p = 0.10) and CMJ performance (p = 0.80) between the testing sessions.

Discussion

This study showed that the RIA and RTT protocols were strongly correlated with each other, particularly with respect to the time-derived measures (BT, TT and AT). In addition, strong correlations were identified between the time-derived measures of RIA with VO2max, CMJ and 30-m sprint performance. Excellent test-retest reliability was evident for the time-derived, perceptual and physiological measures of the RIA protocol, although FI was questionable. The current findings support RIA is a reliable and valid assessment of COD and fitness in young RL players.

The strong correlations of the time-derived measures (BT, TT and AT) between the RIA and RTT protocols, highlighted the RIA protocol was a valid assessment of a repeated-COD, but with movement demands more representative of RL. In addition, the TT and BT of the RIA was strongly associated with the TT and BT of the RSA, indicating that the ability to maintain linear speed would result in superior performances in the RIA protocol, possibly due to similar metabolic demands⁷. Comparable findings were reported by Fessi, Makni⁹, with strong correlations identified between the BT and TT of their repeated agility protocol and RSA protocols in 45 team-sport athletes. The comparable measures between RIA, RTT and RSA suggests that anaerobic fitness, in conjunction with efficient recovery dynamics during short periods of rest in-between explosive activities, are essential qualities for optimal performance in an RIA protocol. Collectively, our results and others^7,9, suggest that performance of repeated-COD relies heavily upon the anaerobic system, a metabolic pathway predominant in RL²⁷.

The current study also identified strong test-retest reliability for time-derived measures (i.e., BT, TT and AT) of the RIA, with minimal measurement error. However, the measurement error was substantially higher for FI, confirming previous studies that reported substantially stronger reliability measures for BT, TT and AT compared to that of FI from various repeated-COD protocols^7,8,28. It has been suggested that FI may exhibit weaker reproducibility as the measure is multifactorial and dependent on the stability of other variables (i.e., TT and BT)^7,29. Subsequently, we, and others^7,8,28,29, recommend that time-derived measures be primarily evaluated during repeated-COD protocols.

Another novelty of the current study was the reliability of the psychophysiological responses during both RIA and RTT protocols. The test-retest reliability values for HR and RPE ranged between questionable-to-excellent classifications according to ICC scores for both RIA and RTT. However, distinctly greater measurement error and bias was observed for RPE when compared to HR measures for both RIA and RTT. These findings were similar to previous studies with poorer reliability for RPE than HR measures during various running protocols^30–32. It has been postulated that HR has better stability across days given that it is an objective measure, compared to the highly subjective RPE³³. It has also been reported that participant’s prior knowledge of the number of sprints during repeated sprint-type protocols may affect results due to pacing³⁴. Accordingly, HR measures may be a better physiological indicator for monitoring exercise-induced stress during repeated-COD protocols.

An additional, yet essential finding of this study was the relationship between baseline characteristics and performances measures from the repeated-COD tests. Measures of CMJ, best-effort speed and best-effort COD performance correlated significantly with the time-derived variables of the RIA. These relationships indicated that lower limb power, linear speed and COD capabilities were contributing factors to successful repeated-COD performances. Our findings aligned with those of Haj-Sassi, Dardouri⁸, who reported strong correlations between measures of jump performance and repeated-COD performance with an Agility T-test protocol. Similar findings were also reported by previous studies with muscular strength, and linear and COD speed considered strong contributors of COD performance^35,36. The significance of this finding attests to lower limb power production being a critical component of repeated-COD performance, especially within the RIA.

Finally, the current study identified significant correlations between VO_2max and RIA performance measures. These findings are similar to previous studies using various repeated-COD protocols^28,29 as well as RSA protocols^37–39. Measures of VO_2max has been considered essential for repeated-sprint type protocols, due to muscular reoxygenation rate^8,40, optimal capacity to remove and buffer hydrogen ions within working muscles⁴¹ and efficiently replenish phosphagen stores⁴². The findings of the present study suggest that aerobic capacity is a strong contributor to superior repeated-COD efforts, further highlighting the need to optimise recovery capacities between high-intensity bouts for RL athletes.

In conclusion, the RIA protocol exhibited moderate-to-excellent test-retest reliability and low measurement error for the majority of time-derived measures and psychophysiological measures, and questionable reliability for FI. Further, the RIA protocol showed strong correlations with the RTT protocol, demonstrating that the RIA protocol provided a valid measure of repeated COD performance. Finally, this study has clearly demonstrated that repeated agility performances rely upon contributions from both anaerobic and aerobic systems with the RIA, demonstrating that the qualities required for optimal RIA performance may be representative of the physical demands in RL. The RIA protocol may provide practitioners with a simple, yet effective monitoring tool to quantify athlete’s ability to generate and sustain multi-directional efforts, and their ability to recover during intermittent activities.

Data availability

Underlying data

James Cook University Research Data: Convergent validity and reliability of a novel repeated agility protocol in junior rugby league players. https://doi.org/10.25903/5eb0f568fad20²⁶

This project contains the following underlying data:

- Raw_data_De-identified.xlsx (Agility protocol data in excel format)
- Raw_data_De-identified.ods (Agility protocol data in ods format)

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Faculty Opinions recommended

References

1. Austin DJ, Gabbett T, Jenkins DJ: Repeated high-intensity exercise in a professional rugby league. J Strength Cond Res. 2011; 25(7): 1898–1904. PubMed Abstract | Publisher Full Text
2. Dos'Santos T, Thomas C, Jones PA, et al.: Mechanical Determinants of Faster Change of Direction Speed Performance in Male Athletes. J Strength Cond Res. 2017; 31(3): 696–705. PubMed Abstract | Publisher Full Text
3. Gabbett TJ, Kelly JN, Sheppard JM: Speed, change of direction speed, and reactive agility of rugby league players. J Strength Cond Res. 2008; 22(1): 174–181. PubMed Abstract | Publisher Full Text
4. Raya MA, Gailey RS, Gaunaurd IA, et al.: Comparison of three agility tests with male servicemembers: Edgren Side Step Test, T-Test, and Illinois Agility Test. J Rehabil Res Dev. 2013; 50(7): 951–960. PubMed Abstract | Publisher Full Text
5. Nimphius S, Callaghan SJ, Bezodis NE, et al.: Change of direction and agility tests: Challenging our current measures of performance. Strength Cond J. 2018; 40(1): 26–38. Publisher Full Text
6. King T, Jenkins D, Gabbett T: A time-motion analysis of professional rugby league match-play. J Sports Sci. 2009; 27(3): 213–219. PubMed Abstract | Publisher Full Text
7. Wong DP, Chan GS, Smith AW: Repeated-sprint and change-of-direction abilities in physically active individuals and soccer players: Training and testing implications. J Strength Cond Res. 2012; 26(9): 2324–2330. PubMed Abstract | Publisher Full Text
8. Haj-Sassi R, Dardouri W, Gharbi Z, et al.: Reliability and validity of a new repeated agility test as a measure of anaerobic and explosive power. J Strength Cond Res. 2011; 25(2): 472–480. PubMed Abstract | Publisher Full Text
9. Fessi MS, Makni E, Elloumi M, et al.: Reliability and criterion-related validity of a new repeated agility test. Biol Sport. 2016; 33(2): 159–164. PubMed Abstract | Publisher Full Text | Free Full Text
10. Wheeler K: Agility skill execution in rugby union. In: Faculty of Science, Health and Education. University of Sunshine Coast. 2010. Reference Source
11. Durandt J, du Toit S, Borresen J, et al.: Fitness and body composition profiling of elite junior South African rugby players. South Afr J Sports Med. 2006; 18(2): 38–45. Publisher Full Text
12. Dugdale JH, Arthur CA, Sanders D, et al.: Reliability and validity of field-based fitness tests in youth soccer players. Eur J Sport Sci. 2019; 19(6): 745–756. PubMed Abstract | Publisher Full Text
13. Dugdale JH, Sanders D, Hunter AM: Reliability of Change of Direction and Agility Assessments in Youth Soccer Players. Sports (Basel). 2020; 8(4): 51. PubMed Abstract | Publisher Full Text | Free Full Text
14. Ramsbottom R, Brewer J, Williams C: A progressive shuttle run test to estimate maximal oxygen uptake. Br J Sports Med. 1988; 22(4): 141–144. PubMed Abstract | Publisher Full Text | Free Full Text
15. Doma K, Nicholls A, Gahreman D, et al.: The effect of a resistance training session on physiological and thermoregulatory measures of sub-maximal running performance in the heat in heat-acclimatized men. Sports Med Open. 2019; 5(1): 21. PubMed Abstract | Publisher Full Text | Free Full Text
16. Doma K, Schumann M, Leicht AS, et al.: The repeated bout effect of traditional resistance exercises on running performance across 3 bouts. Appl Physiol Nutr Metab. 2017; 42(9): 978–985. PubMed Abstract | Publisher Full Text
17. Ramírez-Campillo R, Meylan C, Alvarez C, et al.: Effects of in-season low-volume high-intensity plyometric training on explosive actions and endurance of young soccer players. J Strength Cond Res. 2014; 28(5): 1335–42. PubMed Abstract | Publisher Full Text
18. Doma K, Schumann M, Sinclair WH, et al.: The repeated bout effect of typical lower body strength training sessions on sub-maximal running performance and hormonal response. Eur J Appl Physiol. 2015; 115(8): 1789–99. PubMed Abstract | Publisher Full Text
19. Semenick D: Tests and measurements: The t-test. Strength & Conditioning Journal. 1990; 12(1): 36–37. Reference Source
20. Cureton TK: Illinois agility run. Champaign, Ill: Stipes Publishing Company. 1970. Reference Source
21. Zamunér AR, Moreno MA, Camargo TM, et al.: Assessment of subjective perceived exertion at the anaerobic threshold with the borg CR-10 scale. J Sports Sci Med. 2011; 10(1): 130–6. PubMed Abstract | Free Full Text
22. Abma IL, Rovers M, van der Wees PJ: Appraising convergent validity of patient-reported outcome measures in systematic reviews: constructing hypotheses and interpreting outcomes. BMC Res Notes. 2016; 9: 226. PubMed Abstract | Publisher Full Text | Free Full Text
23. Cordier R, Chen YW, Speyer R, et al.: Child-report measures of occupational performance: a systematic review. PLoS One. 2016; 11(1): e0147751. PubMed Abstract | Publisher Full Text | Free Full Text
24. Leicht AS, Sealey RM, Sinclair WH: The reliability of VO_2(peak) determination in healthy females during an incremental arm ergometry test. Int J Sports Med. 2009; 30(7): 509–15. PubMed Abstract | Publisher Full Text
25. Nevill AM, Atkinson G: Assessing agreement between measurements recorded on a ratio scale in sports medicine and sports science. Br J Sports Med. 1997; 31(4): 314–8. PubMed Abstract | Publisher Full Text | Free Full Text
26. Doma K: Convergent validity and reliability of a novel repeated agility protocol in junior rugby league players. James Cook University. (dataset). 2020; 89b303814f7501cfee8dd987fb606ad5.
27. Gabbett TJ: Physiological and anthropometric characteristics of junior rugby league players over a competitive season. J Strength Cond Res. 2005; 19(4): 764–771. PubMed Abstract
28. Zagatto AM, Ardigò LP, Barbieri FA, et al.: Performance and metabolic demand of a new repeated-sprint ability test in basketball players: does the number of changes of direction matter? J Strength Cond Res. 2017; 31(9): 2438–2446. PubMed Abstract | Publisher Full Text
29. Daneshfar A, Gahreman DE, Koozehchian MS, et al.: Multi directional repeated sprint is a valid and reliable test for assessment of junior handball players. Front Physiol. 2018; 9(1): 317. PubMed Abstract | Publisher Full Text | Free Full Text
30. Doma K, Deakin GB, Leicht AS, et al.: The reliability of running economy among trained distance runners and field-based players. J Exerc Sci Phys Fit. 2012; 10(2): 90–96. Publisher Full Text
31. Lamb KL, Eston RG, Corns D: Reliability of ratings of perceived exertion during progressive treadmill exercise. Br J Sports Med. 1999; 33(5): 336–9. PubMed Abstract | Publisher Full Text | Free Full Text
32. Scott TJ, Black CR, Quinn J, et al.: Validity and reliability of the session-RPE method for quantifying training in Australian football: a comparison of the CR10 and CR100 scales. J Strength Cond Res. 2013; 27(1): 270–6. PubMed Abstract | Publisher Full Text
33. Vera J, Perales JC, Jiménez R, et al.: A test-retest assessment of the effects of mental load on ratings of affect, arousal and perceived exertion during submaximal cycling. J Sports Sci. 2018; 36(22): 2521–2530. PubMed Abstract | Publisher Full Text
34. Billaut F, Bishop DJ, Schaerz S, et al.: Influence of knowledge of sprint number on pacing during repeated-sprint exercise. Med Sci Sport Exerc. 2011; 43(4): 665–672. PubMed Abstract | Publisher Full Text
35. Sheppard JM, Young WB, Doyle TL, et al.: An evaluation of a new test of reactive agility and its relationship to sprint speed and change of direction speed. J Sci Med Sport. 2006; 9(4): 342–349. PubMed Abstract | Publisher Full Text
36. Spiteri T, Nimphius S, Hart NH, et al.: Contribution of strength characteristics to change of direction and agility performance in female basketball athletes. J Strength Cond Res. 2014; 28(9): 2415–2423. PubMed Abstract | Publisher Full Text
37. Bishop D, Edge J: Determinants of repeated-sprint ability in females matched for single-sprint performance. Eur J Appl Physiol. 2006; 97(4): 373–9. PubMed Abstract | Publisher Full Text
38. Gharbi Z, Dardouri W, Haj-Sassi R, et al.: Aerobic and anaerobic determinants of repeated sprint ability in team sports athletes. Biol Sport. 2015; 32(3): 207–12. PubMed Abstract | Publisher Full Text | Free Full Text
39. Glaister M: Multiple sprint work: physiological responses, mechanisms of fatigue and the influence of aerobic fitness. Sports Med. 2005; 35(9): 757–77. PubMed Abstract | Publisher Full Text
40. Buchheit M, Ufland P: Effect of endurance training on performance and muscle reoxygenation rate during repeated-sprint running. Eur J Appl Physiol. 2011; 111(2): 293–301. PubMed Abstract | Publisher Full Text
41. Sahlin K, Henriksson J: Buffer capacity and lactate accumulation in skeletal muscle of trained and untrained men. Acta Physiol Scand. 1984; 122(3): 331–9. PubMed Abstract | Publisher Full Text
42. Bogdanis GC, Nevill ME, Boobis LH, et al.: Contribution of phosphocreatine and aerobic metabolism to energy supply during repeated sprint exercise. J Appl Physiol (1985). 1996; 80(3): 876–84. PubMed Abstract | Publisher Full Text

Comments on this article Comments (0)

Version 3

VERSION 3 PUBLISHED 17 Jun 2020

Author details Author details

¹ Sport & Exercise Science, James Cook University, Douglas, Queensland, 4814, Australia
² Physical Education, Kirwan State Highschool, Kirwan, Queensland, 4817, Australia

Anthony Nicholls
Roles: Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Anthony Leicht
Roles: Conceptualization, Formal Analysis, Methodology, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Jonathan Connor
Roles: Methodology, Project Administration, Supervision, Writing – Review & Editing

Aaron Halliday
Roles: Methodology, Project Administration, Supervision, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (3)

version 3

Revised

Published: 22 Nov 2021, 9:624

https://doi.org/10.12688/f1000research.23129.3

version 2

Revised

Published: 08 Oct 2021, 9:624

https://doi.org/10.12688/f1000research.23129.2

version 1

Published: 17 Jun 2020, 9:624

https://doi.org/10.12688/f1000research.23129.1

© 2021 Nicholls A et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Nicholls A, Leicht A, Connor J et al. Convergent validity and reliability of a novel repeated agility protocol in junior rugby league players [version 3; peer review: 2 approved]. F1000Research 2021, 9:624 (https://doi.org/10.12688/f1000research.23129.3)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 3

VERSION 3

PUBLISHED 22 Nov 2021

Revised

Views

Reviewer Report 22 Nov 2021

Aaron T. Scanlan, Human Exercise and Training Laboratory, School of Health, Medical and Applied Sciences, Central Queensland University, Rockhampton, Qld, Australia

Approved

https://doi.org/10.5256/f1000research.79608.r100796

Thank you for making the requested changes, ... Continue reading

CITE

Report a concern

Respond or Comment

Version 2

VERSION 2

PUBLISHED 08 Oct 2021

Revised

Views

Reviewer Report 08 Nov 2021

James Dugdale, School of Applied Sciences, University of Stirling, Stirling, UK; Edinburgh Napier University, Edinburgh, UK

Approved

https://doi.org/10.5256/f1000research.78004.r96566

Thank you to the authors for their time and efforts taken to revise this manuscript. I am of the opinion ... Continue reading

CITE

Report a concern

Author Response 22 Nov 2021

Kenji Doma, Sport & Exercise Science, James Cook University, Douglas, 4814, Australia

22 Nov 2021

Author Response

Thank you very much for your comments.
Competing Interests: No competing interests were disclosed.
Thank you very much for your comments.
Thank you very much for your comments.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 22 Nov 2021

Kenji Doma, Sport & Exercise Science, James Cook University, Douglas, 4814, Australia

22 Nov 2021

Author Response

Thank you very much for your comments.
Competing Interests: No competing interests were disclosed.
Thank you very much for your comments.
Thank you very much for your comments.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 11 Oct 2021

Aaron T. Scanlan, Human Exercise and Training Laboratory, School of Health, Medical and Applied Sciences, Central Queensland University, Rockhampton, Qld, Australia

Approved

https://doi.org/10.5256/f1000research.78004.r96565

Thank you for the opportunity to further review this submission. I thank the authors for giving such extensive consideration to the previous reviewer suggestions and making relevant changes where applicable. The manuscript has been strengthened, and I only have some ... Continue reading

Abstract - thank you for adding "Beep test", but for consistency with the article text, consider changing this to "Multistage Shuttle test".
Introduction, paragraph 4, second aim - is RSA meant to be aerobic capacity here? In either case, aerobic capacity needs to be included in this aim specifically alongside the other fitness attributes.
Methods, 30-m sprint and agility protocols - be sure to check headings as well given previous suggestions on use of the term "agility". Also in this section, please indicate if the position of the timing gates were kept consistent across all participants with mention of the height (above ground) and distance apart available.
Statistical analysis - still use of "construct validity" on the fourth line in this paragraph as per previous revisions.
Discussion - there is a random equals sign on the last line of the fifth paragraph.

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Exercise and sport science

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 22 Nov 2021

Kenji Doma, Sport & Exercise Science, James Cook University, Douglas, 4814, Australia

22 Nov 2021

Author Response

Reviewer’s comment: Thank you for the opportunity to further review this submission. I thank the authors for giving such extensive consideration to the previous reviewer suggestions and making relevant changes ... Continue reading Reviewer’s comment: Thank you for the opportunity to further review this submission. I thank the authors for giving such extensive consideration to the previous reviewer suggestions and making relevant changes where applicable. The manuscript has been strengthened, and I only have some further minor comments for consideration:
Author’s response: Thank you very much for your comments, we believe your input has strengthened the quality of our work.

Reviewer’s comment: Abstract - thank you for adding "Beep test", but for consistency with the article text, consider changing this to "Multistage Shuttle test".
Author’s response: Amended as requested.

Reviewer’s comment: Introduction, paragraph 4, second aim - is RSA meant to be aerobic capacity here? In either case, aerobic capacity needs to be included in this aim specifically alongside the other fitness attributes.
Author’s response: This is a great pickup, now amended to aerobic capacity.

Reviewer’s comment: Methods, 30-m sprint and agility protocols - be sure to check headings as well given previous suggestions on use of the term "agility". Also in this section, please indicate if the position of the timing gates were kept consistent across all participants with mention of the height (above ground) and distance apart available.
Author’s response: The headings have now been changed to change-of-direction, and the height and position of the gates were described as being consistent for each test.

Reviewer’s comment: Statistical analysis - still use of "construct validity" on the fourth line in this paragraph as per previous revisions.
Author’s response: Changed to construct validity.

Reviewer’s comment: Discussion - there is a random equals sign on the last line of the fifth paragraph.
Author’s response: Thank you, this is now amended.
Reviewer’s comment: Thank you for the opportunity to further review this submission. I thank the authors for giving such extensive consideration to the previous reviewer suggestions and making relevant changes where applicable. The manuscript has been strengthened, and I only have some further minor comments for consideration:
Author’s response: Thank you very much for your comments, we believe your input has strengthened the quality of our work.

Reviewer’s comment: Abstract - thank you for adding "Beep test", but for consistency with the article text, consider changing this to "Multistage Shuttle test".
Author’s response: Amended as requested.

Reviewer’s comment: Introduction, paragraph 4, second aim - is RSA meant to be aerobic capacity here? In either case, aerobic capacity needs to be included in this aim specifically alongside the other fitness attributes.
Author’s response: This is a great pickup, now amended to aerobic capacity.

Reviewer’s comment: Methods, 30-m sprint and agility protocols - be sure to check headings as well given previous suggestions on use of the term "agility". Also in this section, please indicate if the position of the timing gates were kept consistent across all participants with mention of the height (above ground) and distance apart available.
Author’s response: The headings have now been changed to change-of-direction, and the height and position of the gates were described as being consistent for each test.

Reviewer’s comment: Statistical analysis - still use of "construct validity" on the fourth line in this paragraph as per previous revisions.
Author’s response: Changed to construct validity.

Reviewer’s comment: Discussion - there is a random equals sign on the last line of the fifth paragraph.
Author’s response: Thank you, this is now amended.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 22 Nov 2021

Kenji Doma, Sport & Exercise Science, James Cook University, Douglas, 4814, Australia

22 Nov 2021

Author Response

Reviewer’s comment: Thank you for the opportunity to further review this submission. I thank the authors for giving such extensive consideration to the previous reviewer suggestions and making relevant changes ... Continue reading Reviewer’s comment: Thank you for the opportunity to further review this submission. I thank the authors for giving such extensive consideration to the previous reviewer suggestions and making relevant changes where applicable. The manuscript has been strengthened, and I only have some further minor comments for consideration:
Author’s response: Thank you very much for your comments, we believe your input has strengthened the quality of our work.

Reviewer’s comment: Abstract - thank you for adding "Beep test", but for consistency with the article text, consider changing this to "Multistage Shuttle test".
Author’s response: Amended as requested.

Reviewer’s comment: Introduction, paragraph 4, second aim - is RSA meant to be aerobic capacity here? In either case, aerobic capacity needs to be included in this aim specifically alongside the other fitness attributes.
Author’s response: This is a great pickup, now amended to aerobic capacity.

Reviewer’s comment: Methods, 30-m sprint and agility protocols - be sure to check headings as well given previous suggestions on use of the term "agility". Also in this section, please indicate if the position of the timing gates were kept consistent across all participants with mention of the height (above ground) and distance apart available.
Author’s response: The headings have now been changed to change-of-direction, and the height and position of the gates were described as being consistent for each test.

Reviewer’s comment: Statistical analysis - still use of "construct validity" on the fourth line in this paragraph as per previous revisions.
Author’s response: Changed to construct validity.

Reviewer’s comment: Discussion - there is a random equals sign on the last line of the fifth paragraph.
Author’s response: Thank you, this is now amended.
Reviewer’s comment: Thank you for the opportunity to further review this submission. I thank the authors for giving such extensive consideration to the previous reviewer suggestions and making relevant changes where applicable. The manuscript has been strengthened, and I only have some further minor comments for consideration:
Author’s response: Thank you very much for your comments, we believe your input has strengthened the quality of our work.

Reviewer’s comment: Abstract - thank you for adding "Beep test", but for consistency with the article text, consider changing this to "Multistage Shuttle test".
Author’s response: Amended as requested.

Reviewer’s comment: Introduction, paragraph 4, second aim - is RSA meant to be aerobic capacity here? In either case, aerobic capacity needs to be included in this aim specifically alongside the other fitness attributes.
Author’s response: This is a great pickup, now amended to aerobic capacity.

Reviewer’s comment: Methods, 30-m sprint and agility protocols - be sure to check headings as well given previous suggestions on use of the term "agility". Also in this section, please indicate if the position of the timing gates were kept consistent across all participants with mention of the height (above ground) and distance apart available.
Author’s response: The headings have now been changed to change-of-direction, and the height and position of the gates were described as being consistent for each test.

Reviewer’s comment: Statistical analysis - still use of "construct validity" on the fourth line in this paragraph as per previous revisions.
Author’s response: Changed to construct validity.

Reviewer’s comment: Discussion - there is a random equals sign on the last line of the fifth paragraph.
Author’s response: Thank you, this is now amended.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Version 1

VERSION 1

PUBLISHED 17 Jun 2020

Views

Reviewer Report 03 Aug 2021

Aaron T. Scanlan, Human Exercise and Training Laboratory, School of Health, Medical and Applied Sciences, Central Queensland University, Rockhampton, Qld, Australia

Approved with Reservations

https://doi.org/10.5256/f1000research.25532.r90678

Thank you for the opportunity to review this manuscript. A logical study with potential practical impact was presented exploring the validity and reliability of a repeated change-of-direction speed test suited to rugby league. This aspect of fitness testing is lacking in the literature and of use for end-users, so this manuscript has merit for publication. I do however have some suggestions and queries listed below that will help strengthen some aspects of the work that should be addressed by the authors:

Abstract, Methods – indicate whether you are referring to the multi-stage fitness test (i.e., Beep test) here or if just an assessment of multi-stage fitness was performed.
Abstract – some justification of using the repeated T-agility test as the other standard repeated change-of-direction performance to assess convergent agility of the repeated Illinois Agility test is needed in the Abstract.
Abstract – the focus of exploring the contribution of different fitness attributes to repeated Illinois Agility test performance is not made in the Abstract. Consider including this as a secondary objective and/or making it clear why this analysis is needed.
Abstract, Conclusions – make it clear that this test is useful for Rugby League coaches specifically, and change “athlete” to plural form or indicate “team” here instead.
Introduction, opening paragraph – in this paragraph you identify “agility” and hint that it involves physical and cognitive components. So in essence, you are examining change-of-direction speed (the physical component) rather than “agility” per se. Consider making this clear and using the term “change-of-direction speed” or “change-of-direction performance” thereafter when referring to tests and attributes that are purely physical without the cognitive component.
Introduction, opening paragraph – stronger rationale is needed justifying the inclusion of the Agility T-test and Illinois Agility test in rugby league. Can you add a sentence or two outlining why these tests are suited to the sport and therefore the focus of your study?
Introduction, end of second paragraph – here indicate whether the validity and reliability of this test has not been investigated at all or just specifically in Rugby League athletes. Also, at the end of this sentence make it clear that you are referring to the usability of this test.
Introduction, aims – for the second aim, you are not comparing the RIA measures to other measures, but instead correlating them, so please change this aim accordingly. Also, consider including a little rationale around this aim in the previous paragraph as it is unclear as to why this is important.
Introduction, aims – it might pay to include a sentence stating why examining convergent validity (e.g. to show that you are assessing similar attributes with a new test that is more practical and specific to Rugby League when compared to a standard, generic test routinely used) and retest reliability (e.g. to detect the inherent error in the test and ascertain whether it can be reliably adopted in practice to assess repeated measurements in athletes) are needed for practical uptake of the test, which would strengthen the rationale of the first and third aims as well in the Introduction section.
Introduction, hypotheses – you only mention speed and anaerobic capacity as your fitness attributes here, but you also included aerobic fitness (Multistage Fitness test)?
Methods, Research design – why was the order flipped between session 4 and session 5? It seems like you would want to the athletes to complete the same exact session across both when assessing retest reliability as it is introducing a confounding factor? Also, no hyphen needed between “15” and “minutes” here.
Methods, Participants – can you provide some further indication as to the specific playing level of the athletes? The name of the program is great, but this is not exactly clear for all readers.
Figure 1 – consider changing the session numbers, as in text you identify the Multistage Fitness test as session 1, but here you indicate the other fitness testing as session 1. Consistency needed.
Methods, Participants – add “in” after “partaking” and make parents/guardians singular rather than plural.
Methods, Multistage Shuttle test – stay consistent with capitalising the test names like this one as it is done inconsistently throughout. Also, hyphenate “20m” here.
Methods, Countermovement jump test – hyphenate “1 cm” here. Also, try to indicate what units the key outcomes from each test were reported in (e.g. mL/kg/min, cm, s).
Methods, 30-m Sprint and Agility protocols – separate statements on linear and change-of-direction speed here. Also, hyphenate “10m” and “5m” for the Illinois Agility test.
Methods, repeat protocols – to calculate average HR and RPE, was HR measured from when the test started to when the test finished in 1-second intervals? And was RPE taken after each effort or just after all efforts for each specific test? These are not quite clear.
Methods, Statistical analysis – you mention “construct” validity for the first time here. If this is a key aim and aspect of the study (i.e., correlating performance during the test with fitness attributes), then this needs to be established earlier (i.e., introduction and aims).
Methods, Statistical analysis – an r value of 0.5 seems quite low to establish convergent validity (only 25% shared variance)? In this regard, what was the cut-off for construct validity?
Results, first paragraph – clarify whether this is CMJ height specifically.
Results, second paragraph – make it clear that you are referring to the third and fourth sessions here (and make sure this is consistent as you identify these as the fourth and fifth sessions earlier).
Results, third paragraph – you do not indicate the maximum RPE was taken as an outcome in the methods anywhere (only average), yet it is listed here? Same for maximum HR, which appears later in this section also.
Table 1 – shouldn’t RIA and RTT be the two key tests going from left to right rather than RIA and RSA? In fact, it is not clear why RSA is included here given you identify RIA vs. RTT for convergent validity assessment throughout the manuscript. Please adjust or rework earlier sections.
Discussion, first paragraph – some minor errors, but change “was” to “were” in the 7^th line.
Discussion, first paragraph – at the end of this paragraph, some explanation as to how yours (and the previous studies) support that anaerobic fitness is predominantly stressed in the repeated COD tests. I am assuming the strong correlations with mostly anaerobic fitness attributes, but this is an assumption and should be explained further to clarify this statement.
Discussion, fourth paragraph – be careful with your statement suggesting they are “key attributes for RL athletes”. You did not show this but state it, so rework or remove this part to focus specifically on what your data show.
Discussion, concluding paragraph – here you highlight how anaerobic and aerobic fitness underpin test performance, but previously you focus on anaerobic fitness in other sections and in this section. Please make sure the message is consistent throughout.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Exercise and sport science

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 08 Oct 2021

Kenji Doma, Sport & Exercise Science, James Cook University, Douglas, 4814, Australia

08 Oct 2021

Author Response

Reviewer’s comment: Thank you for the opportunity to review this manuscript. A logical study with potential practical impact was presented exploring the validity and reliability of a repeated change-of-direction speed ... Continue reading Reviewer’s comment: Thank you for the opportunity to review this manuscript. A logical study with potential practical impact was presented exploring the validity and reliability of a repeated change-of-direction speed test suited to rugby league. This aspect of fitness testing is lacking in the literature and of use for end-users, so this manuscript has merit for publication. I do however have some suggestions and queries listed below that will help strengthen some aspects of the work that should be addressed by the authors:
Author’s response: Dear Reviewer, thank you very much for your comments. We believe your feedback has improved the quality of our work.

Reviewer’s comment: Abstract, Methods – indicate whether you are referring to the multi-stage fitness test (i.e., Beep test) here or if just an assessment of multi-stage fitness was performed.
Author’s response: A Beep test was performed, which is now included in the Abstract.

Reviewer’s comment: Abstract – some justification of using the repeated T-agility test as the other standard repeated change-of-direction performance to assess convergent agility of the repeated Illinois Agility test is needed in the Abstract.
Author’s response: This is a valid point. The Background now includes, “Thus, the current study examined the convergent validity of a repeated Illinois Agility (RIA) protocol with the repeated T-agility protocol, and the repeatability of the RIA protocol in adolescent Rugby League players.”

Reviewer’s comment: Abstract – the focus of exploring the contribution of different fitness attributes to repeated Illinois Agility test performance is not made in the Abstract. Consider including this as a secondary objective and/or making it clear why this analysis is needed.
Author’s response: Thank you for the suggestion. We have now included, “Furthermore, aerobic capacity and anaerobic and COD performance were assessed to determine whether these physical qualities were important contributors to the RIA protocol.”

Reviewer’s comment: Abstract, Conclusions – make it clear that this test is useful for Rugby League coaches specifically, and change “athlete” to plural form or indicate “team” here instead.
Author’s response: The conclusion has been rewritten to, “The current study has demonstrated the RIA to be a simple, valid and reliable field test for RL athletes that can provide coaches with information about their team’s ability to sustain high intensity, multi-directional running efforts.”

Reviewer’s comment: Introduction, opening paragraph – in this paragraph you identify “agility” and hint that it involves physical and cognitive components. So in essence, you are examining change-of-direction speed (the physical component) rather than “agility” per se. Consider making this clear and using the term “change-of-direction speed” or “change-of-direction performance” thereafter when referring to tests and attributes that are purely physical without the cognitive component.
Author’s response: All the terms with ‘agility’ has been replaced with ‘change-of-direction’ performance or protocols throughout the text.

Reviewer’s comment: Introduction, opening paragraph – stronger rationale is needed justifying the inclusion of the Agility T-test and Illinois Agility test in rugby league. Can you add a sentence or two outlining why these tests are suited to the sport and therefore the focus of your study?
Author’s response: We have removed Agility T-test and Illinois Agility test from the first paragraph, and focused the justification of the Illinois Agility test for RL in the second paragraph (see lines 41-47).

Reviewer’s comment: Introduction, end of second paragraph – here indicate whether the validity and reliability of this test has not been investigated at all or just specifically in Rugby League athletes. Also, at the end of this sentence make it clear that you are referring to the usability of this test.
Author’s response: We separated the second paragraph into two paragraphs. Thus, we highlighted that no studies have examined the validity and reliability of the RIA protocol at the end of the fourth paragraph.

Reviewer’s comment: Introduction, aims – for the second aim, you are not comparing the RIA measures to other measures, but instead correlating them, so please change this aim accordingly. Also, consider including a little rationale around this aim in the previous paragraph as it is unclear as to why this is important.
Author’s response: Changed to ‘correlating’ as requested. The previous paragraph expanded on the need to examine the validity of the RIA protocol, which we now hope provides further justification of our aims.

Reviewer’s comment: Introduction, aims – it might pay to include a sentence stating why examining convergent validity (e.g. to show that you are assessing similar attributes with a new test that is more practical and specific to Rugby League when compared to a standard, generic test routinely used) and retest reliability (e.g. to detect the inherent error in the test and ascertain whether it can be reliably adopted in practice to assess repeated measurements in athletes) are needed for practical uptake of the test, which would strengthen the rationale of the first and third aims as well in the Introduction section.
Author’s response: This is a great suggestion. We have now worded the end of the Introduction to, “Examining the convergent validity of the RIA protocol will determine whether this novel assessment exhibits similar attributes to a standardised COD protocol (i.e., RTT). In addition, the reliability of the RIA will determine whether this test can be reliably adopted in practice by accounting for the inherent error of the test across repeated measurements. The quality of these psychometric properties will provide coaches with a tool to assist in monitoring and training RL athletes as well as in talent development and identification.”

Reviewer’s comment: Introduction, hypotheses – you only mention speed and anaerobic capacity as your fitness attributes here, but you also included aerobic fitness (Multistage Fitness test)?
Author’s response: Included aerobic capacity in the hypotheses.

Reviewer’s comment: Methods, Research design – why was the order flipped between session 4 and session 5? It seems like you would want to the athletes to complete the same exact session across both when assessing retest reliability as it is introducing a confounding factor? Also, no hyphen needed between “15” and “minutes” here.
Author’s response: This is a good question. We employed a cross-over design for sessions 4 and 5 to limit the confounder of any learning effects (i.e., were participants still able to exhibit reproducible measures when the protocols were swapped around, as opposed to the participants becoming accustomed to the order effects between sessions). We have also included the hyphen.

Reviewer’s comment: Methods, Participants – can you provide some further indication as to the specific playing level of the athletes? The name of the program is great, but this is not exactly clear for all readers.
Author’s response: Further description of the excellence program has been provided.

Reviewer’s comment: Figure 1 – consider changing the session numbers, as in text you identify the Multistage Fitness test as session 1, but here you indicate the other fitness testing as session 1. Consistency needed.
Author’s response: Session 1 included in the Multistage Fitness test as requested (see updated Figure 1).

Reviewer’s comment: Methods, Participants – add “in” after “partaking” and make parents/guardians singular rather than plural.
Author’s response: Amended as requested.

Reviewer’s comment: Methods, Multistage Shuttle test – stay consistent with capitalising the test names like this one as it is done inconsistently throughout. Also, hyphenate “20m” here.
Author’s response: Amended the test name to upper case as requested and hyphenated to 20-m.

Reviewer’s comment: Methods, Countermovement jump test – hyphenate “1 cm” here. Also, try to indicate what units the key outcomes from each test were reported in (e.g. mL/kg/min, cm, s).
Author’s response: Hyphenated to 1-cm and reported units of measure as requested for each test.

Reviewer’s comment: Methods, 30-m Sprint and Agility protocols – separate statements on linear and change-of-direction speed here. Also, hyphenate “10m” and “5m” for the Illinois Agility test.
Author’s response: Statements have been separated and hyphenated as requested.

Reviewer’s comment: Methods, repeat protocols – to calculate average HR and RPE, was HR measured from when the test started to when the test finished in 1-second intervals? And was RPE taken after each effort or just after all efforts for each specific test? These are not quite clear.
Author’s response: We have included “ “The participant’s instantaneous heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions. – see lines 138-141.

Reviewer’s comment: Methods, Statistical analysis – you mention “construct” validity for the first time here. If this is a key aim and aspect of the study (i.e., correlating performance during the test with fitness attributes), then this needs to be established earlier (i.e., introduction and aims).
Author’s response: We have taken out “construct validity” and mentioned aerobic capacity, power and COD as “contributors to the RIA protocol”.

Reviewer’s comment: Methods, Statistical analysis – an r value of 0.5 seems quite low to establish convergent validity (only 25% shared variance)? In this regard, what was the cut-off for construct validity?
Author’s response: This has been re-established to a minimum r-value of 0.7 for both convergent validity and contributors to the RIA protocol.

Reviewer’s comment: Results, first paragraph – clarify whether this is CMJ height specifically.
Author’s response: Height included as requested.

Reviewer’s comment: Results, second paragraph – make it clear that you are referring to the third and fourth sessions here (and make sure this is consistent as you identify these as the fourth and fifth sessions earlier).
Author’s response: Sessions numbers included as requested, and we also applied this approach in the next paragraph for the RTT protocol to ensure clarity.

Reviewer’s comment: Results, third paragraph – you do not indicate the maximum RPE was taken as an outcome in the methods anywhere (only average), yet it is listed here? Same for maximum HR, which appears later in this section also.
Author’s response: This is a great pick up. In the methods section, we have now included, “The participant’s heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions.”

Reviewer’s comment: Table 1 – shouldn’t RIA and RTT be the two key tests going from left to right rather than RIA and RSA? In fact, it is not clear why RSA is included here given you identify RIA vs. RTT for convergent validity assessment throughout the manuscript. Please adjust or rework earlier sections.
Author’s response: Table 1 has now been amended as requested. RSA was included more to determine the importance of the athlete’s ability to sustain speed and effort over repeated bouts, which has now been included in the second aim.

Reviewer’s comment: Discussion, first paragraph – some minor errors, but change “was” to “were” in the 7th line.
Author’s response: Amended as requested.

Reviewer’s comment: Discussion, first paragraph – at the end of this paragraph, some explanation as to how yours (and the previous studies) support that anaerobic fitness is predominantly stressed in the repeated COD tests. I am assuming the strong correlations with mostly anaerobic fitness attributes, but this is an assumption and should be explained further to clarify this statement.
Author’s response: Closer to the end of this paragraph, we have included a sentence that reads, “The comparable measures between RIA, RTT and RSA suggests that anaerobic fitness, in conjunction with efficient recovery dynamics during short periods of rest in-between explosive activities, are essential qualities for optimal performance in an RIA protocol.”

Reviewer’s comment: Discussion, fourth paragraph – be careful with your statement suggesting they are “key attributes for RL athletes”. You did not show this but state it, so rework or remove this part to focus specifically on what your data show.
Author’s response: Removed as requested.

Reviewer’s comment: Discussion, concluding paragraph – here you highlight how anaerobic and aerobic fitness underpin test performance, but previously you focus on anaerobic fitness in other sections and in this section. Please make sure the message is consistent throughout.
Author’s response: We have now included aerobic capacity as possible important contributors to the RIA protocol in the Abstract, Introduction (aim and hypothesis) and earlier sections in the Discussion.
Reviewer’s comment: Thank you for the opportunity to review this manuscript. A logical study with potential practical impact was presented exploring the validity and reliability of a repeated change-of-direction speed test suited to rugby league. This aspect of fitness testing is lacking in the literature and of use for end-users, so this manuscript has merit for publication. I do however have some suggestions and queries listed below that will help strengthen some aspects of the work that should be addressed by the authors:
Author’s response: Dear Reviewer, thank you very much for your comments. We believe your feedback has improved the quality of our work.

Reviewer’s comment: Abstract, Methods – indicate whether you are referring to the multi-stage fitness test (i.e., Beep test) here or if just an assessment of multi-stage fitness was performed.
Author’s response: A Beep test was performed, which is now included in the Abstract.

Reviewer’s comment: Abstract – some justification of using the repeated T-agility test as the other standard repeated change-of-direction performance to assess convergent agility of the repeated Illinois Agility test is needed in the Abstract.
Author’s response: This is a valid point. The Background now includes, “Thus, the current study examined the convergent validity of a repeated Illinois Agility (RIA) protocol with the repeated T-agility protocol, and the repeatability of the RIA protocol in adolescent Rugby League players.”

Reviewer’s comment: Abstract – the focus of exploring the contribution of different fitness attributes to repeated Illinois Agility test performance is not made in the Abstract. Consider including this as a secondary objective and/or making it clear why this analysis is needed.
Author’s response: Thank you for the suggestion. We have now included, “Furthermore, aerobic capacity and anaerobic and COD performance were assessed to determine whether these physical qualities were important contributors to the RIA protocol.”

Reviewer’s comment: Abstract, Conclusions – make it clear that this test is useful for Rugby League coaches specifically, and change “athlete” to plural form or indicate “team” here instead.
Author’s response: The conclusion has been rewritten to, “The current study has demonstrated the RIA to be a simple, valid and reliable field test for RL athletes that can provide coaches with information about their team’s ability to sustain high intensity, multi-directional running efforts.”

Reviewer’s comment: Introduction, opening paragraph – in this paragraph you identify “agility” and hint that it involves physical and cognitive components. So in essence, you are examining change-of-direction speed (the physical component) rather than “agility” per se. Consider making this clear and using the term “change-of-direction speed” or “change-of-direction performance” thereafter when referring to tests and attributes that are purely physical without the cognitive component.
Author’s response: All the terms with ‘agility’ has been replaced with ‘change-of-direction’ performance or protocols throughout the text.

Reviewer’s comment: Introduction, opening paragraph – stronger rationale is needed justifying the inclusion of the Agility T-test and Illinois Agility test in rugby league. Can you add a sentence or two outlining why these tests are suited to the sport and therefore the focus of your study?
Author’s response: We have removed Agility T-test and Illinois Agility test from the first paragraph, and focused the justification of the Illinois Agility test for RL in the second paragraph (see lines 41-47).

Reviewer’s comment: Introduction, end of second paragraph – here indicate whether the validity and reliability of this test has not been investigated at all or just specifically in Rugby League athletes. Also, at the end of this sentence make it clear that you are referring to the usability of this test.
Author’s response: We separated the second paragraph into two paragraphs. Thus, we highlighted that no studies have examined the validity and reliability of the RIA protocol at the end of the fourth paragraph.

Reviewer’s comment: Introduction, aims – for the second aim, you are not comparing the RIA measures to other measures, but instead correlating them, so please change this aim accordingly. Also, consider including a little rationale around this aim in the previous paragraph as it is unclear as to why this is important.
Author’s response: Changed to ‘correlating’ as requested. The previous paragraph expanded on the need to examine the validity of the RIA protocol, which we now hope provides further justification of our aims.

Reviewer’s comment: Introduction, aims – it might pay to include a sentence stating why examining convergent validity (e.g. to show that you are assessing similar attributes with a new test that is more practical and specific to Rugby League when compared to a standard, generic test routinely used) and retest reliability (e.g. to detect the inherent error in the test and ascertain whether it can be reliably adopted in practice to assess repeated measurements in athletes) are needed for practical uptake of the test, which would strengthen the rationale of the first and third aims as well in the Introduction section.
Author’s response: This is a great suggestion. We have now worded the end of the Introduction to, “Examining the convergent validity of the RIA protocol will determine whether this novel assessment exhibits similar attributes to a standardised COD protocol (i.e., RTT). In addition, the reliability of the RIA will determine whether this test can be reliably adopted in practice by accounting for the inherent error of the test across repeated measurements. The quality of these psychometric properties will provide coaches with a tool to assist in monitoring and training RL athletes as well as in talent development and identification.”

Reviewer’s comment: Introduction, hypotheses – you only mention speed and anaerobic capacity as your fitness attributes here, but you also included aerobic fitness (Multistage Fitness test)?
Author’s response: Included aerobic capacity in the hypotheses.

Reviewer’s comment: Methods, Research design – why was the order flipped between session 4 and session 5? It seems like you would want to the athletes to complete the same exact session across both when assessing retest reliability as it is introducing a confounding factor? Also, no hyphen needed between “15” and “minutes” here.
Author’s response: This is a good question. We employed a cross-over design for sessions 4 and 5 to limit the confounder of any learning effects (i.e., were participants still able to exhibit reproducible measures when the protocols were swapped around, as opposed to the participants becoming accustomed to the order effects between sessions). We have also included the hyphen.

Reviewer’s comment: Methods, Participants – can you provide some further indication as to the specific playing level of the athletes? The name of the program is great, but this is not exactly clear for all readers.
Author’s response: Further description of the excellence program has been provided.

Reviewer’s comment: Figure 1 – consider changing the session numbers, as in text you identify the Multistage Fitness test as session 1, but here you indicate the other fitness testing as session 1. Consistency needed.
Author’s response: Session 1 included in the Multistage Fitness test as requested (see updated Figure 1).

Reviewer’s comment: Methods, Participants – add “in” after “partaking” and make parents/guardians singular rather than plural.
Author’s response: Amended as requested.

Reviewer’s comment: Methods, Multistage Shuttle test – stay consistent with capitalising the test names like this one as it is done inconsistently throughout. Also, hyphenate “20m” here.
Author’s response: Amended the test name to upper case as requested and hyphenated to 20-m.

Reviewer’s comment: Methods, Countermovement jump test – hyphenate “1 cm” here. Also, try to indicate what units the key outcomes from each test were reported in (e.g. mL/kg/min, cm, s).
Author’s response: Hyphenated to 1-cm and reported units of measure as requested for each test.

Reviewer’s comment: Methods, 30-m Sprint and Agility protocols – separate statements on linear and change-of-direction speed here. Also, hyphenate “10m” and “5m” for the Illinois Agility test.
Author’s response: Statements have been separated and hyphenated as requested.

Reviewer’s comment: Methods, repeat protocols – to calculate average HR and RPE, was HR measured from when the test started to when the test finished in 1-second intervals? And was RPE taken after each effort or just after all efforts for each specific test? These are not quite clear.
Author’s response: We have included “ “The participant’s instantaneous heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions. – see lines 138-141.

Reviewer’s comment: Methods, Statistical analysis – you mention “construct” validity for the first time here. If this is a key aim and aspect of the study (i.e., correlating performance during the test with fitness attributes), then this needs to be established earlier (i.e., introduction and aims).
Author’s response: We have taken out “construct validity” and mentioned aerobic capacity, power and COD as “contributors to the RIA protocol”.

Reviewer’s comment: Methods, Statistical analysis – an r value of 0.5 seems quite low to establish convergent validity (only 25% shared variance)? In this regard, what was the cut-off for construct validity?
Author’s response: This has been re-established to a minimum r-value of 0.7 for both convergent validity and contributors to the RIA protocol.

Reviewer’s comment: Results, first paragraph – clarify whether this is CMJ height specifically.
Author’s response: Height included as requested.

Reviewer’s comment: Results, second paragraph – make it clear that you are referring to the third and fourth sessions here (and make sure this is consistent as you identify these as the fourth and fifth sessions earlier).
Author’s response: Sessions numbers included as requested, and we also applied this approach in the next paragraph for the RTT protocol to ensure clarity.

Reviewer’s comment: Results, third paragraph – you do not indicate the maximum RPE was taken as an outcome in the methods anywhere (only average), yet it is listed here? Same for maximum HR, which appears later in this section also.
Author’s response: This is a great pick up. In the methods section, we have now included, “The participant’s heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions.”

Reviewer’s comment: Table 1 – shouldn’t RIA and RTT be the two key tests going from left to right rather than RIA and RSA? In fact, it is not clear why RSA is included here given you identify RIA vs. RTT for convergent validity assessment throughout the manuscript. Please adjust or rework earlier sections.
Author’s response: Table 1 has now been amended as requested. RSA was included more to determine the importance of the athlete’s ability to sustain speed and effort over repeated bouts, which has now been included in the second aim.

Reviewer’s comment: Discussion, first paragraph – some minor errors, but change “was” to “were” in the 7th line.
Author’s response: Amended as requested.

Reviewer’s comment: Discussion, first paragraph – at the end of this paragraph, some explanation as to how yours (and the previous studies) support that anaerobic fitness is predominantly stressed in the repeated COD tests. I am assuming the strong correlations with mostly anaerobic fitness attributes, but this is an assumption and should be explained further to clarify this statement.
Author’s response: Closer to the end of this paragraph, we have included a sentence that reads, “The comparable measures between RIA, RTT and RSA suggests that anaerobic fitness, in conjunction with efficient recovery dynamics during short periods of rest in-between explosive activities, are essential qualities for optimal performance in an RIA protocol.”

Reviewer’s comment: Discussion, fourth paragraph – be careful with your statement suggesting they are “key attributes for RL athletes”. You did not show this but state it, so rework or remove this part to focus specifically on what your data show.
Author’s response: Removed as requested.

Reviewer’s comment: Discussion, concluding paragraph – here you highlight how anaerobic and aerobic fitness underpin test performance, but previously you focus on anaerobic fitness in other sections and in this section. Please make sure the message is consistent throughout.
Author’s response: We have now included aerobic capacity as possible important contributors to the RIA protocol in the Abstract, Introduction (aim and hypothesis) and earlier sections in the Discussion.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 08 Oct 2021

Kenji Doma, Sport & Exercise Science, James Cook University, Douglas, 4814, Australia

08 Oct 2021

Author Response

Reviewer’s comment: Thank you for the opportunity to review this manuscript. A logical study with potential practical impact was presented exploring the validity and reliability of a repeated change-of-direction speed ... Continue reading Reviewer’s comment: Thank you for the opportunity to review this manuscript. A logical study with potential practical impact was presented exploring the validity and reliability of a repeated change-of-direction speed test suited to rugby league. This aspect of fitness testing is lacking in the literature and of use for end-users, so this manuscript has merit for publication. I do however have some suggestions and queries listed below that will help strengthen some aspects of the work that should be addressed by the authors:
Author’s response: Dear Reviewer, thank you very much for your comments. We believe your feedback has improved the quality of our work.

Reviewer’s comment: Abstract, Methods – indicate whether you are referring to the multi-stage fitness test (i.e., Beep test) here or if just an assessment of multi-stage fitness was performed.
Author’s response: A Beep test was performed, which is now included in the Abstract.

Reviewer’s comment: Abstract – some justification of using the repeated T-agility test as the other standard repeated change-of-direction performance to assess convergent agility of the repeated Illinois Agility test is needed in the Abstract.
Author’s response: This is a valid point. The Background now includes, “Thus, the current study examined the convergent validity of a repeated Illinois Agility (RIA) protocol with the repeated T-agility protocol, and the repeatability of the RIA protocol in adolescent Rugby League players.”

Reviewer’s comment: Abstract – the focus of exploring the contribution of different fitness attributes to repeated Illinois Agility test performance is not made in the Abstract. Consider including this as a secondary objective and/or making it clear why this analysis is needed.
Author’s response: Thank you for the suggestion. We have now included, “Furthermore, aerobic capacity and anaerobic and COD performance were assessed to determine whether these physical qualities were important contributors to the RIA protocol.”

Reviewer’s comment: Abstract, Conclusions – make it clear that this test is useful for Rugby League coaches specifically, and change “athlete” to plural form or indicate “team” here instead.
Author’s response: The conclusion has been rewritten to, “The current study has demonstrated the RIA to be a simple, valid and reliable field test for RL athletes that can provide coaches with information about their team’s ability to sustain high intensity, multi-directional running efforts.”

Reviewer’s comment: Introduction, opening paragraph – in this paragraph you identify “agility” and hint that it involves physical and cognitive components. So in essence, you are examining change-of-direction speed (the physical component) rather than “agility” per se. Consider making this clear and using the term “change-of-direction speed” or “change-of-direction performance” thereafter when referring to tests and attributes that are purely physical without the cognitive component.
Author’s response: All the terms with ‘agility’ has been replaced with ‘change-of-direction’ performance or protocols throughout the text.

Reviewer’s comment: Introduction, opening paragraph – stronger rationale is needed justifying the inclusion of the Agility T-test and Illinois Agility test in rugby league. Can you add a sentence or two outlining why these tests are suited to the sport and therefore the focus of your study?
Author’s response: We have removed Agility T-test and Illinois Agility test from the first paragraph, and focused the justification of the Illinois Agility test for RL in the second paragraph (see lines 41-47).

Reviewer’s comment: Introduction, end of second paragraph – here indicate whether the validity and reliability of this test has not been investigated at all or just specifically in Rugby League athletes. Also, at the end of this sentence make it clear that you are referring to the usability of this test.
Author’s response: We separated the second paragraph into two paragraphs. Thus, we highlighted that no studies have examined the validity and reliability of the RIA protocol at the end of the fourth paragraph.

Reviewer’s comment: Introduction, aims – for the second aim, you are not comparing the RIA measures to other measures, but instead correlating them, so please change this aim accordingly. Also, consider including a little rationale around this aim in the previous paragraph as it is unclear as to why this is important.
Author’s response: Changed to ‘correlating’ as requested. The previous paragraph expanded on the need to examine the validity of the RIA protocol, which we now hope provides further justification of our aims.

Reviewer’s comment: Introduction, aims – it might pay to include a sentence stating why examining convergent validity (e.g. to show that you are assessing similar attributes with a new test that is more practical and specific to Rugby League when compared to a standard, generic test routinely used) and retest reliability (e.g. to detect the inherent error in the test and ascertain whether it can be reliably adopted in practice to assess repeated measurements in athletes) are needed for practical uptake of the test, which would strengthen the rationale of the first and third aims as well in the Introduction section.
Author’s response: This is a great suggestion. We have now worded the end of the Introduction to, “Examining the convergent validity of the RIA protocol will determine whether this novel assessment exhibits similar attributes to a standardised COD protocol (i.e., RTT). In addition, the reliability of the RIA will determine whether this test can be reliably adopted in practice by accounting for the inherent error of the test across repeated measurements. The quality of these psychometric properties will provide coaches with a tool to assist in monitoring and training RL athletes as well as in talent development and identification.”

Reviewer’s comment: Introduction, hypotheses – you only mention speed and anaerobic capacity as your fitness attributes here, but you also included aerobic fitness (Multistage Fitness test)?
Author’s response: Included aerobic capacity in the hypotheses.

Reviewer’s comment: Methods, Research design – why was the order flipped between session 4 and session 5? It seems like you would want to the athletes to complete the same exact session across both when assessing retest reliability as it is introducing a confounding factor? Also, no hyphen needed between “15” and “minutes” here.
Author’s response: This is a good question. We employed a cross-over design for sessions 4 and 5 to limit the confounder of any learning effects (i.e., were participants still able to exhibit reproducible measures when the protocols were swapped around, as opposed to the participants becoming accustomed to the order effects between sessions). We have also included the hyphen.

Reviewer’s comment: Methods, Participants – can you provide some further indication as to the specific playing level of the athletes? The name of the program is great, but this is not exactly clear for all readers.
Author’s response: Further description of the excellence program has been provided.

Reviewer’s comment: Figure 1 – consider changing the session numbers, as in text you identify the Multistage Fitness test as session 1, but here you indicate the other fitness testing as session 1. Consistency needed.
Author’s response: Session 1 included in the Multistage Fitness test as requested (see updated Figure 1).

Reviewer’s comment: Methods, Participants – add “in” after “partaking” and make parents/guardians singular rather than plural.
Author’s response: Amended as requested.

Reviewer’s comment: Methods, Multistage Shuttle test – stay consistent with capitalising the test names like this one as it is done inconsistently throughout. Also, hyphenate “20m” here.
Author’s response: Amended the test name to upper case as requested and hyphenated to 20-m.

Reviewer’s comment: Methods, Countermovement jump test – hyphenate “1 cm” here. Also, try to indicate what units the key outcomes from each test were reported in (e.g. mL/kg/min, cm, s).
Author’s response: Hyphenated to 1-cm and reported units of measure as requested for each test.

Reviewer’s comment: Methods, 30-m Sprint and Agility protocols – separate statements on linear and change-of-direction speed here. Also, hyphenate “10m” and “5m” for the Illinois Agility test.
Author’s response: Statements have been separated and hyphenated as requested.

Reviewer’s comment: Methods, repeat protocols – to calculate average HR and RPE, was HR measured from when the test started to when the test finished in 1-second intervals? And was RPE taken after each effort or just after all efforts for each specific test? These are not quite clear.
Author’s response: We have included “ “The participant’s instantaneous heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions. – see lines 138-141.

Reviewer’s comment: Methods, Statistical analysis – you mention “construct” validity for the first time here. If this is a key aim and aspect of the study (i.e., correlating performance during the test with fitness attributes), then this needs to be established earlier (i.e., introduction and aims).
Author’s response: We have taken out “construct validity” and mentioned aerobic capacity, power and COD as “contributors to the RIA protocol”.

Reviewer’s comment: Methods, Statistical analysis – an r value of 0.5 seems quite low to establish convergent validity (only 25% shared variance)? In this regard, what was the cut-off for construct validity?
Author’s response: This has been re-established to a minimum r-value of 0.7 for both convergent validity and contributors to the RIA protocol.

Reviewer’s comment: Results, first paragraph – clarify whether this is CMJ height specifically.
Author’s response: Height included as requested.

Reviewer’s comment: Results, second paragraph – make it clear that you are referring to the third and fourth sessions here (and make sure this is consistent as you identify these as the fourth and fifth sessions earlier).
Author’s response: Sessions numbers included as requested, and we also applied this approach in the next paragraph for the RTT protocol to ensure clarity.

Reviewer’s comment: Results, third paragraph – you do not indicate the maximum RPE was taken as an outcome in the methods anywhere (only average), yet it is listed here? Same for maximum HR, which appears later in this section also.
Author’s response: This is a great pick up. In the methods section, we have now included, “The participant’s heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions.”

Reviewer’s comment: Table 1 – shouldn’t RIA and RTT be the two key tests going from left to right rather than RIA and RSA? In fact, it is not clear why RSA is included here given you identify RIA vs. RTT for convergent validity assessment throughout the manuscript. Please adjust or rework earlier sections.
Author’s response: Table 1 has now been amended as requested. RSA was included more to determine the importance of the athlete’s ability to sustain speed and effort over repeated bouts, which has now been included in the second aim.

Reviewer’s comment: Discussion, first paragraph – some minor errors, but change “was” to “were” in the 7th line.
Author’s response: Amended as requested.

Reviewer’s comment: Discussion, first paragraph – at the end of this paragraph, some explanation as to how yours (and the previous studies) support that anaerobic fitness is predominantly stressed in the repeated COD tests. I am assuming the strong correlations with mostly anaerobic fitness attributes, but this is an assumption and should be explained further to clarify this statement.
Author’s response: Closer to the end of this paragraph, we have included a sentence that reads, “The comparable measures between RIA, RTT and RSA suggests that anaerobic fitness, in conjunction with efficient recovery dynamics during short periods of rest in-between explosive activities, are essential qualities for optimal performance in an RIA protocol.”

Reviewer’s comment: Discussion, fourth paragraph – be careful with your statement suggesting they are “key attributes for RL athletes”. You did not show this but state it, so rework or remove this part to focus specifically on what your data show.
Author’s response: Removed as requested.

Reviewer’s comment: Discussion, concluding paragraph – here you highlight how anaerobic and aerobic fitness underpin test performance, but previously you focus on anaerobic fitness in other sections and in this section. Please make sure the message is consistent throughout.
Author’s response: We have now included aerobic capacity as possible important contributors to the RIA protocol in the Abstract, Introduction (aim and hypothesis) and earlier sections in the Discussion.
Reviewer’s comment: Thank you for the opportunity to review this manuscript. A logical study with potential practical impact was presented exploring the validity and reliability of a repeated change-of-direction speed test suited to rugby league. This aspect of fitness testing is lacking in the literature and of use for end-users, so this manuscript has merit for publication. I do however have some suggestions and queries listed below that will help strengthen some aspects of the work that should be addressed by the authors:
Author’s response: Dear Reviewer, thank you very much for your comments. We believe your feedback has improved the quality of our work.

Reviewer’s comment: Abstract, Methods – indicate whether you are referring to the multi-stage fitness test (i.e., Beep test) here or if just an assessment of multi-stage fitness was performed.
Author’s response: A Beep test was performed, which is now included in the Abstract.

Reviewer’s comment: Abstract – some justification of using the repeated T-agility test as the other standard repeated change-of-direction performance to assess convergent agility of the repeated Illinois Agility test is needed in the Abstract.
Author’s response: This is a valid point. The Background now includes, “Thus, the current study examined the convergent validity of a repeated Illinois Agility (RIA) protocol with the repeated T-agility protocol, and the repeatability of the RIA protocol in adolescent Rugby League players.”

Reviewer’s comment: Abstract – the focus of exploring the contribution of different fitness attributes to repeated Illinois Agility test performance is not made in the Abstract. Consider including this as a secondary objective and/or making it clear why this analysis is needed.
Author’s response: Thank you for the suggestion. We have now included, “Furthermore, aerobic capacity and anaerobic and COD performance were assessed to determine whether these physical qualities were important contributors to the RIA protocol.”

Reviewer’s comment: Abstract, Conclusions – make it clear that this test is useful for Rugby League coaches specifically, and change “athlete” to plural form or indicate “team” here instead.
Author’s response: The conclusion has been rewritten to, “The current study has demonstrated the RIA to be a simple, valid and reliable field test for RL athletes that can provide coaches with information about their team’s ability to sustain high intensity, multi-directional running efforts.”

Reviewer’s comment: Introduction, opening paragraph – in this paragraph you identify “agility” and hint that it involves physical and cognitive components. So in essence, you are examining change-of-direction speed (the physical component) rather than “agility” per se. Consider making this clear and using the term “change-of-direction speed” or “change-of-direction performance” thereafter when referring to tests and attributes that are purely physical without the cognitive component.
Author’s response: All the terms with ‘agility’ has been replaced with ‘change-of-direction’ performance or protocols throughout the text.

Reviewer’s comment: Introduction, opening paragraph – stronger rationale is needed justifying the inclusion of the Agility T-test and Illinois Agility test in rugby league. Can you add a sentence or two outlining why these tests are suited to the sport and therefore the focus of your study?
Author’s response: We have removed Agility T-test and Illinois Agility test from the first paragraph, and focused the justification of the Illinois Agility test for RL in the second paragraph (see lines 41-47).

Reviewer’s comment: Introduction, end of second paragraph – here indicate whether the validity and reliability of this test has not been investigated at all or just specifically in Rugby League athletes. Also, at the end of this sentence make it clear that you are referring to the usability of this test.
Author’s response: We separated the second paragraph into two paragraphs. Thus, we highlighted that no studies have examined the validity and reliability of the RIA protocol at the end of the fourth paragraph.

Reviewer’s comment: Introduction, aims – for the second aim, you are not comparing the RIA measures to other measures, but instead correlating them, so please change this aim accordingly. Also, consider including a little rationale around this aim in the previous paragraph as it is unclear as to why this is important.
Author’s response: Changed to ‘correlating’ as requested. The previous paragraph expanded on the need to examine the validity of the RIA protocol, which we now hope provides further justification of our aims.

Reviewer’s comment: Introduction, aims – it might pay to include a sentence stating why examining convergent validity (e.g. to show that you are assessing similar attributes with a new test that is more practical and specific to Rugby League when compared to a standard, generic test routinely used) and retest reliability (e.g. to detect the inherent error in the test and ascertain whether it can be reliably adopted in practice to assess repeated measurements in athletes) are needed for practical uptake of the test, which would strengthen the rationale of the first and third aims as well in the Introduction section.
Author’s response: This is a great suggestion. We have now worded the end of the Introduction to, “Examining the convergent validity of the RIA protocol will determine whether this novel assessment exhibits similar attributes to a standardised COD protocol (i.e., RTT). In addition, the reliability of the RIA will determine whether this test can be reliably adopted in practice by accounting for the inherent error of the test across repeated measurements. The quality of these psychometric properties will provide coaches with a tool to assist in monitoring and training RL athletes as well as in talent development and identification.”

Reviewer’s comment: Introduction, hypotheses – you only mention speed and anaerobic capacity as your fitness attributes here, but you also included aerobic fitness (Multistage Fitness test)?
Author’s response: Included aerobic capacity in the hypotheses.

Reviewer’s comment: Methods, Research design – why was the order flipped between session 4 and session 5? It seems like you would want to the athletes to complete the same exact session across both when assessing retest reliability as it is introducing a confounding factor? Also, no hyphen needed between “15” and “minutes” here.
Author’s response: This is a good question. We employed a cross-over design for sessions 4 and 5 to limit the confounder of any learning effects (i.e., were participants still able to exhibit reproducible measures when the protocols were swapped around, as opposed to the participants becoming accustomed to the order effects between sessions). We have also included the hyphen.

Reviewer’s comment: Methods, Participants – can you provide some further indication as to the specific playing level of the athletes? The name of the program is great, but this is not exactly clear for all readers.
Author’s response: Further description of the excellence program has been provided.

Reviewer’s comment: Figure 1 – consider changing the session numbers, as in text you identify the Multistage Fitness test as session 1, but here you indicate the other fitness testing as session 1. Consistency needed.
Author’s response: Session 1 included in the Multistage Fitness test as requested (see updated Figure 1).

Reviewer’s comment: Methods, Participants – add “in” after “partaking” and make parents/guardians singular rather than plural.
Author’s response: Amended as requested.

Reviewer’s comment: Methods, Multistage Shuttle test – stay consistent with capitalising the test names like this one as it is done inconsistently throughout. Also, hyphenate “20m” here.
Author’s response: Amended the test name to upper case as requested and hyphenated to 20-m.

Reviewer’s comment: Methods, Countermovement jump test – hyphenate “1 cm” here. Also, try to indicate what units the key outcomes from each test were reported in (e.g. mL/kg/min, cm, s).
Author’s response: Hyphenated to 1-cm and reported units of measure as requested for each test.

Reviewer’s comment: Methods, 30-m Sprint and Agility protocols – separate statements on linear and change-of-direction speed here. Also, hyphenate “10m” and “5m” for the Illinois Agility test.
Author’s response: Statements have been separated and hyphenated as requested.

Reviewer’s comment: Methods, repeat protocols – to calculate average HR and RPE, was HR measured from when the test started to when the test finished in 1-second intervals? And was RPE taken after each effort or just after all efforts for each specific test? These are not quite clear.
Author’s response: We have included “ “The participant’s instantaneous heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions. – see lines 138-141.

Reviewer’s comment: Methods, Statistical analysis – you mention “construct” validity for the first time here. If this is a key aim and aspect of the study (i.e., correlating performance during the test with fitness attributes), then this needs to be established earlier (i.e., introduction and aims).
Author’s response: We have taken out “construct validity” and mentioned aerobic capacity, power and COD as “contributors to the RIA protocol”.

Reviewer’s comment: Methods, Statistical analysis – an r value of 0.5 seems quite low to establish convergent validity (only 25% shared variance)? In this regard, what was the cut-off for construct validity?
Author’s response: This has been re-established to a minimum r-value of 0.7 for both convergent validity and contributors to the RIA protocol.

Reviewer’s comment: Results, first paragraph – clarify whether this is CMJ height specifically.
Author’s response: Height included as requested.

Reviewer’s comment: Results, second paragraph – make it clear that you are referring to the third and fourth sessions here (and make sure this is consistent as you identify these as the fourth and fifth sessions earlier).
Author’s response: Sessions numbers included as requested, and we also applied this approach in the next paragraph for the RTT protocol to ensure clarity.

Reviewer’s comment: Results, third paragraph – you do not indicate the maximum RPE was taken as an outcome in the methods anywhere (only average), yet it is listed here? Same for maximum HR, which appears later in this section also.
Author’s response: This is a great pick up. In the methods section, we have now included, “The participant’s heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions.”

Reviewer’s comment: Table 1 – shouldn’t RIA and RTT be the two key tests going from left to right rather than RIA and RSA? In fact, it is not clear why RSA is included here given you identify RIA vs. RTT for convergent validity assessment throughout the manuscript. Please adjust or rework earlier sections.
Author’s response: Table 1 has now been amended as requested. RSA was included more to determine the importance of the athlete’s ability to sustain speed and effort over repeated bouts, which has now been included in the second aim.

Reviewer’s comment: Discussion, first paragraph – some minor errors, but change “was” to “were” in the 7th line.
Author’s response: Amended as requested.

Reviewer’s comment: Discussion, first paragraph – at the end of this paragraph, some explanation as to how yours (and the previous studies) support that anaerobic fitness is predominantly stressed in the repeated COD tests. I am assuming the strong correlations with mostly anaerobic fitness attributes, but this is an assumption and should be explained further to clarify this statement.
Author’s response: Closer to the end of this paragraph, we have included a sentence that reads, “The comparable measures between RIA, RTT and RSA suggests that anaerobic fitness, in conjunction with efficient recovery dynamics during short periods of rest in-between explosive activities, are essential qualities for optimal performance in an RIA protocol.”

Reviewer’s comment: Discussion, fourth paragraph – be careful with your statement suggesting they are “key attributes for RL athletes”. You did not show this but state it, so rework or remove this part to focus specifically on what your data show.
Author’s response: Removed as requested.

Reviewer’s comment: Discussion, concluding paragraph – here you highlight how anaerobic and aerobic fitness underpin test performance, but previously you focus on anaerobic fitness in other sections and in this section. Please make sure the message is consistent throughout.
Author’s response: We have now included aerobic capacity as possible important contributors to the RIA protocol in the Abstract, Introduction (aim and hypothesis) and earlier sections in the Discussion.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 09 Nov 2020

James Dugdale, School of Applied Sciences, University of Stirling, Stirling, UK; Edinburgh Napier University, Edinburgh, UK

Approved with Reservations

https://doi.org/10.5256/f1000research.25532.r74005

The present study aimed to explore the validity and reliability of repeated “agility” tests in a sample of youth rugby players. The study provides interesting data for an applied audience and is methodologically sound and accurate in the presentation of findings. This being said, I have identified a range of areas that should be improved prior to potential resubmissions of this manuscript. For example, the introduction (and study in its entirety) is inaccurate in its use of the term “agility” (opposed to COD which the authors actually assess). Furthermore, the introduction needs to be further developed to support the scope of the study. The authors need to create more of a rationale for running such a large number of correlations – exploring correlational analyses needs to be warranted based upon an assumption of a relationship (or lack thereof) between variables. In its current form, I am unsure whether this manuscript provides such rationale. The methods and results are accurate, but with a large number of small errors. Finally, the discussion addresses key discussion points formulated from the findings of the study, but similar to the introduction, lacks some critical depth.

I do think that the authors have a publishable study here, however, a number of revisions will be required before I would be happy to approve this manuscript for indexing.
I hope that my comments are received as I intend them and are useful in further strengthening this manuscript. I hope to receive the opportunity to review this manuscript again in its revised form.

Abstract
The abstract is concise and accurate to the details of the study. I have just a couple of minor comments (below):
The initial sentence is accurate and outlines the requirement of the study. However, I would encourage the authors to insert a word such as ‘simultaneously’ or ‘congruently’, or attempt to capture the lack of a suitable test to capture the performance of these abilities in combination.
Abstract Methods – the inclusion of the repeated T-agility test appears suddenly with no prior reference. If this is a key concentration of your study, I would encourage to insert earlier in the abstract along with the RIA.
Abstract Results – please specify exact p values rather than just <0.05. CV% can be presented to 1dp for brevity.

Introduction
The introduction is short and concise. However, I feel some key information that would further support the rationale for your study has been omitted. I would encourage the authors to further develop this section. A suggestion would be to expand the second paragraph into two: the first discussing the validity aspects, the second the reliability aspects. I do not expect these changes to be too onerous, but believe it would strengthen this section of your manuscript if you were to implement these changes. I have some further specific comments below.
The opening sentence would read better if it were split into two separate sentences.
The authors should attempt to clarify during opening paragraph that they are discussing change of direction (COD) performance, rather than agility. Although COD performance is a key component of agility performance, the lack of an external stimuli prevents classification of agility. Although the tests identified by the authors in this opening paragraph have titled themselves ‘agility’ tests, by definition they are not. The authors should be careful in their stance here as the literature has evolved substantially since the inception of tests such as these, and appropriate classification and terminology is essential for the present study.
Young, W.; Dawson, B.; Henry, G. Agility and Change-of-Direction Speed are Independent Skills: Implications for Agility in Invasion Sports. Int. J. Sports Sci. Coach. 2015, 10, 159–169.¹
Nimphius, S.; Callaghan, S.J.; Bezodis, N.E.; Lockie, R.G. Change of Direction and Agility Tests: Challenging Our Current Measures of Performance. Strength Cond. J. 2018, 40, 26–38. ²
Sheppard, J.M.; Young, W.B. Agility literature review: Classifications, training and testing. J. Sports Sci. 2006, 24, 919–932.³
Regarding the reliability aspect of the study, two recent papers have suggested that reliability of COD and agility may be lower in adolescent and youth athletes (as per your sample). Although these authors observed this during maximal COD/agility performance, opposed to repeated as in the present study, I believe this would be worth including and would further warrant the exploration of your research question and study.
Taylor, J.M.; Cunningham, L.; Hood, P.; Thorne, B.; Irvin, G.; Weston, M. The reliability of a modified 505 test and change-of-direction deficit time in elite youth football players. Sci. Med. Footb. 2019, 3, 157–162.⁴
Dugdale JH, Sanders D, Hunter AM. Reliability of Change of Direction and Agility Assessments in Youth Soccer Players. Sports (Basel). 2020 Apr 18; 8 (4): 51. doi: 10.3390/sports8040051.⁵
James H. Dugdale, Calum A. Arthur, Dajo Sanders & Angus M. Hunter (2019) Reliability and validity of field-based fitness tests in youth soccer players, European Journal of Sport Science, 19:6, 745-756, DOI: 10.1080/17461391.2018.1556739 ⁶

Methods
The methods report the appropriate details relating to the study design, protocols, participants, and analyses performed. However, further clarity is required in a few places throughout this section (specific comments below). Further, there are a number of oversights throughout this section that require careful attention. None of these are major points, and should take little time to amend. Yet, need to be addressed prior to potential resubmissions of this manuscript.
Figure 1: I am unsure how much this figure aids the interpretation of the study design. I personally do not feel the visual nature of this schematic clarifies the description of the trials from the text, as the study is relatively simplistic. I do not insist that the authors remove this figure, however, please consider my suggestion that it may not add value to the manuscript and may just be a “figure for the sake of a figure”. Secondly, between the trials, the authors have stated that washout periods of > 7 days were implemented. The authors should consider including the range of days (e.g. 7-14 days), as this would provide further information to the reader and allow for more accurate interpretation of the results.
Can a citation or elaboration be provided to rationalise the >15 min rest interval between attempts?
The term ‘respectively’ needs to be included after the explanation of the 1-10 muscle rating scale. Also, a brief sentence as to why this was collected should accompany this methodological point.
The authors state that the warm up was the same prior to each session. I would encourage using the term ‘standardised’ when describing this in their methods to add certainty around their controls. Also, a citation needs to accompany the statement that the CMJ test was implemented to assess recovery. If this is what was intended by the insertion of citation 12, I would encourage moving this citation to the end of the sentence.
Participant details – participant height needs to be reported to 2dp. Can the authors further elaborate on what the “School of Athletic Excellence” is? For example, what standard is this, is it affiliated to a specific school, or region performance programme? This may not be clear to readers.
Multistage shuttle test – a brief sentence stating how performance or a score was derived is necessary (i.e. maximum number of shuttles completed, level, etc.). This information is currently not evident.
CMJ – please state the unit of measurement for this test, and to what degree (e.g. recorded in cm to the nearest 0.1cm).
30-m sprint and agility protocols – please separate each test into a separate sentence. At present, they appear too cramped together and it is difficult to read (particularly with reference to figures and citations within). Also, please provide citations for the original studies for the T-drill and Illinois agility tests. These need to be credited within the manuscript.
Repeated sprint and agility protocols – were these tests performed one after the other (what I interpret from the term ‘cycles’) or were they performed independently? Clarity is required to ensure the reader understands which of these interpretations is correct.
The sentence starting “Immediately after each repeated agility cycle…” does not make sense. Please rephrase this.
Statistical analysis – Reference 16 seems a weak reference for this point. I encourage the authors to consider a more suitable citation for selecting this acceptable correlation cut off point.

Results
The section of the results dedicated to the muscle soreness and CMJ differences (or lack thereof) should be condensed to a singular sentence and placed at the foot of the results section. This is a methodological control and not a key finding from your study, therefore, it should not feature so early and heavily within your results section. Stating that there were no differences, providing ES, and significance values would suffice.
Table 3 – I am interested in your rationale for not providing ES to demonstrate differences (or lack thereof) for your test-retest data? This would add value in my opinion. I would also encourage the authors to present CV% data to 1dp both within this table and throughout the manuscript.
I notice that the authors use terminology such as “excellent” or “moderate” test-retest reliability, yet they have not included these threshold within the statistical analysis section of the methods. Please include this detail so that the reader understands your criteria.
I also note that the authors say “most RIA performance measures exhibited excellent test-retest reliability…”. Could the authors rephrase this to say “all RIA performance measures except… measures exhibited excellent test-retest reliability”? This would be more accurate and less ambiguous for the reader. This comment also applies later in this section where the authors refer to “a few variables…”.

Discussion
I found the discussion well-written and thought the authors accurately interpreted and attempted to explain their findings. However, similar to the introduction, I felt this section lacked depth when discussing the main findings of the study. I encourage the authors to discuss their findings in greater depth, utilising some of the reading attached to this review.
I encourage the authors to utilise the first paragraph of the discussion to summarise all of their main findings, and then progress to discussing each point in turn.
The third paragraph begins “another novelty of this study”, yet this appears to be the first time novelty is addressed in the discussion?
The fourth paragraph can draw upon a wealth of correlational studies to further support the findings observed here.
Young WB, Dawson B, Henry GJ. Agility and Change-of-Direction Speed are Independent Skills: Implications for Training for Agility in Invasion Sports. International Journal of Sports Science & Coaching. 2015;10(1):159-169. doi:10.1260/1747-9541.10.1.159 ¹
Spiteri T, Nimphius S, Hart NH, Specos C, Sheppard JM, Newton RU. Contribution of strength characteristics to change of direction and agility performance in female basketball athletes. J Strength Cond Res. 2014 Sep;28(9):2415-23. doi: 10.1519/JSC.0000000000000547. PMID: 24875426.⁷
Gabbett, T. J., Kelly, J. N., & Sheppard, J. M. (2008). Speed, Change of Direction Speed, and Reactive Agility of Rugby League Players. The Journal of Strength & Conditioning Research, 22, 174-181. http://dx.doi.org/10.1519/JSC.0b013e31815ef700 ⁸
Sheppard, J.M.; Young, W.B.; Doyle, T.L.A.; Sheppard, T.A.; Newton, R.U. An evaluation of a new test of reactive agility and its relationship to sprint speed and change of direction speed. J. Sci Med. Sport 2006, 9, 342–349.⁹
The conclusion should be reframed around the key objectives of the study (i.e. that it is reliable and valid). The mention of aerobic and anaerobic energy systems seems misplaced here.

References
While a variety of key texts are identified and listed within the bibliography and cited throughout the manuscript, a number of key texts are omitted. I encourage the authors to further familiarise themselves with relevant studies assessing reliability and validity of COD and agility in team sport athletes, as well as the texts identified earlier within this review.
Dos’Santos, T.; Thomas, C.; Jones, P.A.; Comfort, P. Mechanical Determinants of Faster Change of Direction Speed Performance in Male Athletes. J. Strength Cond. Res. 2017, 31, 696–705.¹⁰
Gabbett, T.J.; Kelly, J.N.; Sheppard, J.M. Speed, change of direction speed, and reactive agility of rugby league players. J. Strength Cond. Res. 2008, 22, 174–181.⁸
Oliver, J.L.; Meyers, R.W. Reliability and generality of measures of acceleration, planned agility, and reactive agility. Int. J. Sports Physiol. Perform. 2009, 4, 345–354.¹¹

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

References

1. Young W, Dawson B, Henry G: Agility and Change-of-Direction Speed are Independent Skills: Implications for Training for Agility in Invasion Sports. International Journal of Sports Science & Coaching. 2015; 10 (1): 159-169 Publisher Full Text
2. Nimphius S, Callaghan S, Bezodis N, Lockie R: Change of Direction and Agility Tests. Strength and Conditioning Journal. 2018; 40 (1): 26-38 Publisher Full Text
3. Sheppard JM, Young WB: Agility literature review: classifications, training and testing.J Sports Sci. 2006; 24 (9): 919-32 PubMed Abstract | Publisher Full Text
4. Taylor J, Cunningham L, Hood P, Thorne B, et al.: The reliability of a modified 505 test and change-of-direction deficit time in elite youth football players. Science and Medicine in Football. 2019; 3 (2): 157-162 Publisher Full Text
5. Dugdale JH, Sanders D, Hunter AM: Reliability of Change of Direction and Agility Assessments in Youth Soccer Players.Sports (Basel). 2020; 8 (4). PubMed Abstract | Publisher Full Text
6. Dugdale JH, Arthur CA, Sanders D, Hunter AM: Reliability and validity of field-based fitness tests in youth soccer players.Eur J Sport Sci. 2019; 19 (6): 745-756 PubMed Abstract | Publisher Full Text
7. Spiteri T, Nimphius S, Hart NH, Specos C, et al.: Contribution of strength characteristics to change of direction and agility performance in female basketball athletes.J Strength Cond Res. 2014; 28 (9): 2415-23 PubMed Abstract | Publisher Full Text
8. Gabbett TJ, Kelly JN, Sheppard JM: Speed, change of direction speed, and reactive agility of rugby league players.J Strength Cond Res. 2008; 22 (1): 174-81 PubMed Abstract | Publisher Full Text
9. Sheppard JM, Young WB, Doyle TL, Sheppard TA, et al.: An evaluation of a new test of reactive agility and its relationship to sprint speed and change of direction speed.J Sci Med Sport. 2006; 9 (4): 342-9 PubMed Abstract | Publisher Full Text
10. DosʼSantos T, Thomas C, Jones PA, Comfort P: Mechanical Determinants of Faster Change of Direction Speed Performance in Male Athletes.J Strength Cond Res. 2017; 31 (3): 696-705 PubMed Abstract | Publisher Full Text
11. Oliver J, Meyers R: Reliability and Generality of Measures of Acceleration, Planned Agility, and Reactive Agility. International Journal of Sports Physiology and Performance. 2009; 4 (3): 345-354 Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Fitness testing; youth athletes; talent identification; talent development; physical performance; strength and conditioning

CITE

Report a concern

Author Response 08 Oct 2021

Kenji Doma, Sport & Exercise Science, James Cook University, Douglas, 4814, Australia

08 Oct 2021

Author Response

Reviewer’s comment: The present study aimed to explore the validity and reliability of repeated “agility” tests in a sample of youth rugby players. The study provides interesting data for an ... Continue reading Reviewer’s comment: The present study aimed to explore the validity and reliability of repeated “agility” tests in a sample of youth rugby players. The study provides interesting data for an applied audience and is methodologically sound and accurate in the presentation of findings. This being said, I have identified a range of areas that should be improved prior to potential resubmissions of this manuscript. For example, the introduction (and study in its entirety) is inaccurate in its use of the term “agility” (opposed to COD which the authors actually assess). Furthermore, the introduction needs to be further developed to support the scope of the study. The authors need to create more of a rationale for running such a large number of correlations – exploring correlational analyses needs to be warranted based upon an assumption of a relationship (or lack thereof) between variables. In its current form, I am unsure whether this manuscript provides such rationale. The methods and results are accurate, but with a large number of small errors. Finally, the discussion addresses key discussion points formulated from the findings of the study, but similar to the introduction, lacks some critical depth. I do think that the authors have a publishable study here, however, a number of revisions will be required before I would be happy to approve this manuscript for indexing. I hope that my comments are received as I intend them and are useful in further strengthening this manuscript. I hope to receive the opportunity to review this manuscript again in its revised form.
Author’s response: Dear reviewer, thank you for your comments on the paper. We agree that your comments have improved the quality of our work.

Abstract
Reviewer’s comment: The abstract is concise and accurate to the details of the study. I have just a couple of minor comments (below):
The initial sentence is accurate and outlines the requirement of the study. However, I would encourage the authors to insert a word such as ‘simultaneously’ or ‘congruently’, or attempt to capture the lack of a suitable test to capture the performance of these abilities in combination.
Author’s response: The word ‘simultaneously’ has been included.

Reviewer’s comment: Abstract Methods – the inclusion of the repeated T-agility test appears suddenly with no prior reference. If this is a key concentration of your study, I would encourage to insert earlier in the abstract along with the RIA.
Author’s response: Repeated T-agility protocol now mentioned in the Background.

Reviewer’s comment: Abstract Results – please specify exact p values rather than just <0.05. CV% can be presented to 1dp for brevity.
Author’s response: We were unable to provide exact p-values, as several outcome measures from the repeated COD protocols, such as average time, best time, total time etc were presented as “performances”. This same approach was used for the r-values as well to ensure the word limit did not exceed journal guidelines. However, we did present the CV% to 1dp as requested.

Introduction
Reviewer’s comment: The introduction is short and concise. However, I feel some key information that would further support the rationale for your study has been omitted. I would encourage the authors to further develop this section. A suggestion would be to expand the second paragraph into two: the first discussing the validity aspects, the second the reliability aspects. I do not expect these changes to be too onerous, but believe it would strengthen this section of your manuscript if you were to implement these changes. I have some further specific comments below.
Author’s response: The second paragraph has been further developed into two as requested, with the first focusing on reliability and the second on validity.

Reviewer’s comment: The opening sentence would read better if it were split into two separate sentences.
Author’s response: We have simplified the first sentence by removing the context of ‘agility’, as we have converted all ‘agility’ terms to ‘change-of-direction’.

Reviewer’s comment: The authors should attempt to clarify during opening paragraph that they are discussing change of direction (COD) performance, rather than agility. Although COD performance is a key component of agility performance, the lack of an external stimuli prevents classification of agility. Although the tests identified by the authors in this opening paragraph have titled themselves ‘agility’ tests, by definition they are not. The authors should be careful in their stance here as the literature has evolved substantially since the inception of tests such as these, and appropriate classification and terminology is essential for the present study.
Young, W.; Dawson, B.; Henry, G. Agility and Change-of-Direction Speed are Independent Skills: Implications for Agility in Invasion Sports. Int. J. Sports Sci. Coach. 2015, 10, 159–169.1
Nimphius, S.; Callaghan, S.J.; Bezodis, N.E.; Lockie, R.G. Change of Direction and Agility Tests: Challenging Our Current Measures of Performance. Strength Cond. J. 2018, 40, 26–38. 2
Sheppard, J.M.; Young, W.B. Agility literature review: Classifications, training and testing. J. Sports Sci. 2006, 24, 919–932.3
Author’s response: Thank you for your suggestion. As mentioned earlier, we have changed all ‘agility’ terms into ‘change-of-direction’.

Reviewer’s comment: Regarding the reliability aspect of the study, two recent papers have suggested that reliability of COD and agility may be lower in adolescent and youth athletes (as per your sample). Although these authors observed this during maximal COD/agility performance, opposed to repeated as in the present study, I believe this would be worth including and would further warrant the exploration of your research question and study.
Taylor, J.M.; Cunningham, L.; Hood, P.; Thorne, B.; Irvin, G.; Weston, M. The reliability of a modified 505 test and change-of-direction deficit time in elite youth football players. Sci. Med. Footb. 2019, 3, 157–162.4
Dugdale JH, Sanders D, Hunter AM. Reliability of Change of Direction and Agility Assessments in Youth Soccer Players. Sports (Basel). 2020 Apr 18; 8 (4): 51. doi: 10.3390/sports8040051.5
James H. Dugdale, Calum A. Arthur, Dajo Sanders & Angus M. Hunter (2019) Reliability and validity of field-based fitness tests in youth soccer players, European Journal of Sport Science, 19:6, 745-756, DOI: 10.1080/17461391.2018.15567396
Author’s response: Thank you for your suggestion, we have included the references above in the paragraph explaining studies on the reliability of COD protocols.

Methods
Reviewer’s comment: The methods report the appropriate details relating to the study design, protocols, participants, and analyses performed. However, further clarity is required in a few places throughout this section (specific comments below). Further, there are a number of oversights throughout this section that require careful attention. None of these are major points, and should take little time to amend. Yet, need to be addressed prior to potential resubmissions of this manuscript.
Author’s response: Thank you for your comments, we have addressed these below.

Reviewer’s comment: Figure 1: I am unsure how much this figure aids the interpretation of the study design. I personally do not feel the visual nature of this schematic clarifies the description of the trials from the text, as the study is relatively simplistic. I do not insist that the authors remove this figure, however, please consider my suggestion that it may not add value to the manuscript and may just be a “figure for the sake of a figure”. Secondly, between the trials, the authors have stated that washout periods of > 7 days were implemented. The authors should consider including the range of days (e.g. 7-14 days), as this would provide further information to the reader and allow for more accurate interpretation of the results.
Author’s response: Thank you for your comment. We have decided to keep the figure to ensure clarity as we used a cross-over randomized design, which is unusual for a study examining the reliability and validity of protocols. We have included 7-14 days as a washout period as suggested in the figure.

Reviewer’s comment: Can a citation or elaboration be provided to rationalise the >15 min rest interval between attempts?
Author’s response: A citation has been included with a study that used a 15-minute recovery period in-between repeated sprint and COD protocols in the one testing session.

Reviewer’s comment: The term ‘respectively’ needs to be included after the explanation of the 1-10 muscle rating scale. Also, a brief sentence as to why this was collected should accompany this methodological point.
Author’s response: Included as requested.

Reviewer’s comment: The authors state that the warm up was the same prior to each session. I would encourage using the term ‘standardised’ when describing this in their methods to add certainty around their controls. Also, a citation needs to accompany the statement that the CMJ test was implemented to assess recovery. If this is what was intended by the insertion of citation 12, I would encourage moving this citation to the end of the sentence.
Author’s response: The term ‘standardised’ has been included when describing the warm-up as requested in the Methods section. The citation explaining the use of CMJ to determine recovery dynamics was moved to the end of the sentence as requested.

Reviewer’s comment: Participant details – participant height needs to be reported to 2dp. Can the authors further elaborate on what the “School of Athletic Excellence” is? For example, what standard is this, is it affiliated to a specific school, or region performance programme? This may not be clear to readers.
Author’s response: The participant height is reported to 2dp (1.77m). Further information on the excellence program has been included.

Reviewer’s comment: Multistage shuttle test – a brief sentence stating how performance or a score was derived is necessary (i.e. maximum number of shuttles completed, level, etc.). This information is currently not evident.
Author’s response: The VO2max was estimated based on the level completed, which has now been included in the Methods section.

Reviewer’s comment: CMJ – please state the unit of measurement for this test, and to what degree (e.g. recorded in cm to the nearest 0.1cm).
Author’s response: The CMJ was recorded to the full cm, not to the nearest 0.1cm. This has now been clarified.

Reviewer’s comment: 30-m sprint and agility protocols – please separate each test into a separate sentence. At present, they appear too cramped together and it is difficult to read (particularly with reference to figures and citations within). Also, please provide citations for the original studies for the T-drill and Illinois agility tests. These need to be credited within the manuscript.
Author’s response: The sentences for the description of the protocols have been separated as requested. The original studies have also been included as requested.

Reviewer’s comment: Repeated sprint and agility protocols – were these tests performed one after the other (what I interpret from the term ‘cycles’) or were they performed independently? Clarity is required to ensure the reader understands which of these interpretations is correct.
Author’s response: We have reworded the term ‘cycles’ to ‘repetitions’ to improve clarity. As mentioned in Research Design and in Figure 1, RSA was conducted in Session 2, whilst RTT and RIA were conducted in the same session with 15-min rest in-between each protocol. The Figure reinforces this design, and rewording ‘cycles’ to ‘repetitions’ improves the clarity of the protocols.

Reviewer’s comment: The sentence starting “Immediately after each repeated agility cycle…” does not make sense. Please rephrase this.
Author’s response: This has been reworded to, “The participant’s heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions”

Reviewer’s comment: Statistical analysis – Reference 16 seems a weak reference for this point. I encourage the authors to consider a more suitable citation for selecting this acceptable correlation cut off point.
Author’s response: Additional reference included as requested.

Results
Reviewer’s comment: The section of the results dedicated to the muscle soreness and CMJ differences (or lack thereof) should be condensed to a singular sentence and placed at the foot of the results section. This is a methodological control and not a key finding from your study, therefore, it should not feature so early and heavily within your results section. Stating that there were no differences, providing ES, and significance values would suffice.
Author’s response: Simplified and included at the end of the Results section as requested.

Reviewer’s comment: Table 3 – I am interested in your rationale for not providing ES to demonstrate differences (or lack thereof) for your test-retest data? This would add value in my opinion. I would also encourage the authors to present CV% data to 1dp both within this table and throughout the manuscript.
Author’s response: Thank you for your comment. The ES was not reported as we based the measurement error on CV, with larger CV% values exhibiting greater measurement error and poorer test-retest reliability. In our opinion, inclusion of ES would not provide any further clarity about the reliability of the tests, beyond that of CV.

Reviewer’s comment: I notice that the authors use terminology such as “excellent” or “moderate” test-retest reliability, yet they have not included these threshold within the statistical analysis section of the methods. Please include this detail so that the reader understands your criteria.
Author’s response: Included as requested.

Reviewer’s comment: I also note that the authors say “most RIA performance measures exhibited excellent test-retest reliability…”. Could the authors rephrase this to say “all RIA performance measures except… measures exhibited excellent test-retest reliability”? This would be more accurate and less ambiguous for the reader. This comment also applies later in this section where the authors refer to “a few variables…”.
Author’s response: Thank you for your suggestion. We have reworded this to, “Most RIA measures exhibited excellent test-retest reliability (ICC = 0.92-0.97), good levels of agreement (ratio LOA = 1.05-1.06) and low measurement error (CV = 2.17 - 2.68%) (Table 3). However, FI and average RPE demonstrated moderate test-retest reliability (ICC = 0.87 and 0.76, respectively), poorer levels of agreement (ratio LOA = 2.57 and 2.23, respectively) and higher measurement error (CV = 25.3 and 15.8%, respectively, Table 3).” The next paragraph has been rewritten to, “For the RTT, excellent test-retest reliability (ICC = 0.91), good levels of agreement (ratio LOA = 1.08) and low measurement error (CV = 2.17-2.68%) were identified for TT, BT and AT (Table 3), although maximum RPE demonstrated higher levels of measurement error (CV = 12.3%) between Sessions 4 and 5 (Table 3).”

Discussion
Reviewer’s comment: I found the discussion well-written and thought the authors accurately interpreted and attempted to explain their findings. However, similar to the introduction, I felt this section lacked depth when discussing the main findings of the study. I encourage the authors to discuss their findings in greater depth, utilising some of the reading attached to this review.
Author’s response: Thank you for your comments. The Discussion has been adjusted to address the comments below.

Reviewer’s comment: I encourage the authors to utilise the first paragraph of the discussion to summarise all of their main findings, and then progress to discussing each point in turn.
Author’s response: Summary of findings included as requested in the first paragraph.

Reviewer’s comment: The third paragraph begins “another novelty of this study”, yet this appears to be the first time novelty is addressed in the discussion?
Author’s response: This specific text has been deleted to improve clarity.

Reviewer’s comment: The fourth paragraph can draw upon a wealth of correlational studies to further support the findings observed here.
Young WB, Dawson B, Henry GJ. Agility and Change-of-Direction Speed are Independent Skills: Implications for Training for Agility in Invasion Sports. International Journal of Sports Science & Coaching. 2015;10(1):159-169. doi:10.1260/1747-9541.10.1.1591
Spiteri T, Nimphius S, Hart NH, Specos C, Sheppard JM, Newton RU. Contribution of strength characteristics to change of direction and agility performance in female basketball athletes. J Strength Cond Res. 2014 Sep;28(9):2415-23. doi: 10.1519/JSC.0000000000000547. PMID: 24875426.7
Gabbett, T. J., Kelly, J. N., & Sheppard, J. M. (2008). Speed, Change of Direction Speed, and Reactive Agility of Rugby League Players. The Journal of Strength & Conditioning Research, 22, 174-181. http://dx.doi.org/10.1519/JSC.0b013e31815ef7008
Sheppard, J.M.; Young, W.B.; Doyle, T.L.A.; Sheppard, T.A.; Newton, R.U. An evaluation of a new test of reactive agility and its relationship to sprint speed and change of direction speed. J. Sci Med. Sport 2006, 9, 342–349.9
Author’s response: A sentence with relevant references listed above were included as requested.

Reviewer’s comment: The conclusion should be reframed around the key objectives of the study (i.e. that it is reliable and valid). The mention of aerobic and anaerobic energy systems seems misplaced here.
Author’s response: A sentence on the validity of the RIA protocols has been included.

References
Reviewer’s comment: While a variety of key texts are identified and listed within the bibliography and cited throughout the manuscript, a number of key texts are omitted. I encourage the authors to further familiarise themselves with relevant studies assessing reliability and validity of COD and agility in team sport athletes, as well as the texts identified earlier within this review.
Dos’Santos, T.; Thomas, C.; Jones, P.A.; Comfort, P. Mechanical Determinants of Faster Change of Direction Speed Performance in Male Athletes. J. Strength Cond. Res. 2017, 31, 696–705.10
Gabbett, T.J.; Kelly, J.N.; Sheppard, J.M. Speed, change of direction speed, and reactive agility of rugby league players. J. Strength Cond. Res. 2008, 22, 174–181.8
Oliver, J.L.; Meyers, R.W. Reliability and generality of measures of acceleration, planned agility, and reactive agility. Int. J. Sports Physiol. Perform. 2009, 4, 345–354.11
Author’s response: Thank you for kindly providing the references. A number of these have been included in the paper as requested.
Reviewer’s comment: The present study aimed to explore the validity and reliability of repeated “agility” tests in a sample of youth rugby players. The study provides interesting data for an applied audience and is methodologically sound and accurate in the presentation of findings. This being said, I have identified a range of areas that should be improved prior to potential resubmissions of this manuscript. For example, the introduction (and study in its entirety) is inaccurate in its use of the term “agility” (opposed to COD which the authors actually assess). Furthermore, the introduction needs to be further developed to support the scope of the study. The authors need to create more of a rationale for running such a large number of correlations – exploring correlational analyses needs to be warranted based upon an assumption of a relationship (or lack thereof) between variables. In its current form, I am unsure whether this manuscript provides such rationale. The methods and results are accurate, but with a large number of small errors. Finally, the discussion addresses key discussion points formulated from the findings of the study, but similar to the introduction, lacks some critical depth. I do think that the authors have a publishable study here, however, a number of revisions will be required before I would be happy to approve this manuscript for indexing. I hope that my comments are received as I intend them and are useful in further strengthening this manuscript. I hope to receive the opportunity to review this manuscript again in its revised form.
Author’s response: Dear reviewer, thank you for your comments on the paper. We agree that your comments have improved the quality of our work.

Abstract
Reviewer’s comment: The abstract is concise and accurate to the details of the study. I have just a couple of minor comments (below):
The initial sentence is accurate and outlines the requirement of the study. However, I would encourage the authors to insert a word such as ‘simultaneously’ or ‘congruently’, or attempt to capture the lack of a suitable test to capture the performance of these abilities in combination.
Author’s response: The word ‘simultaneously’ has been included.

Reviewer’s comment: Abstract Methods – the inclusion of the repeated T-agility test appears suddenly with no prior reference. If this is a key concentration of your study, I would encourage to insert earlier in the abstract along with the RIA.
Author’s response: Repeated T-agility protocol now mentioned in the Background.

Reviewer’s comment: Abstract Results – please specify exact p values rather than just <0.05. CV% can be presented to 1dp for brevity.
Author’s response: We were unable to provide exact p-values, as several outcome measures from the repeated COD protocols, such as average time, best time, total time etc were presented as “performances”. This same approach was used for the r-values as well to ensure the word limit did not exceed journal guidelines. However, we did present the CV% to 1dp as requested.

Introduction
Reviewer’s comment: The introduction is short and concise. However, I feel some key information that would further support the rationale for your study has been omitted. I would encourage the authors to further develop this section. A suggestion would be to expand the second paragraph into two: the first discussing the validity aspects, the second the reliability aspects. I do not expect these changes to be too onerous, but believe it would strengthen this section of your manuscript if you were to implement these changes. I have some further specific comments below.
Author’s response: The second paragraph has been further developed into two as requested, with the first focusing on reliability and the second on validity.

Reviewer’s comment: The opening sentence would read better if it were split into two separate sentences.
Author’s response: We have simplified the first sentence by removing the context of ‘agility’, as we have converted all ‘agility’ terms to ‘change-of-direction’.

Reviewer’s comment: The authors should attempt to clarify during opening paragraph that they are discussing change of direction (COD) performance, rather than agility. Although COD performance is a key component of agility performance, the lack of an external stimuli prevents classification of agility. Although the tests identified by the authors in this opening paragraph have titled themselves ‘agility’ tests, by definition they are not. The authors should be careful in their stance here as the literature has evolved substantially since the inception of tests such as these, and appropriate classification and terminology is essential for the present study.
Young, W.; Dawson, B.; Henry, G. Agility and Change-of-Direction Speed are Independent Skills: Implications for Agility in Invasion Sports. Int. J. Sports Sci. Coach. 2015, 10, 159–169.1
Nimphius, S.; Callaghan, S.J.; Bezodis, N.E.; Lockie, R.G. Change of Direction and Agility Tests: Challenging Our Current Measures of Performance. Strength Cond. J. 2018, 40, 26–38. 2
Sheppard, J.M.; Young, W.B. Agility literature review: Classifications, training and testing. J. Sports Sci. 2006, 24, 919–932.3
Author’s response: Thank you for your suggestion. As mentioned earlier, we have changed all ‘agility’ terms into ‘change-of-direction’.

Reviewer’s comment: Regarding the reliability aspect of the study, two recent papers have suggested that reliability of COD and agility may be lower in adolescent and youth athletes (as per your sample). Although these authors observed this during maximal COD/agility performance, opposed to repeated as in the present study, I believe this would be worth including and would further warrant the exploration of your research question and study.
Taylor, J.M.; Cunningham, L.; Hood, P.; Thorne, B.; Irvin, G.; Weston, M. The reliability of a modified 505 test and change-of-direction deficit time in elite youth football players. Sci. Med. Footb. 2019, 3, 157–162.4
Dugdale JH, Sanders D, Hunter AM. Reliability of Change of Direction and Agility Assessments in Youth Soccer Players. Sports (Basel). 2020 Apr 18; 8 (4): 51. doi: 10.3390/sports8040051.5
James H. Dugdale, Calum A. Arthur, Dajo Sanders & Angus M. Hunter (2019) Reliability and validity of field-based fitness tests in youth soccer players, European Journal of Sport Science, 19:6, 745-756, DOI: 10.1080/17461391.2018.15567396
Author’s response: Thank you for your suggestion, we have included the references above in the paragraph explaining studies on the reliability of COD protocols.

Methods
Reviewer’s comment: The methods report the appropriate details relating to the study design, protocols, participants, and analyses performed. However, further clarity is required in a few places throughout this section (specific comments below). Further, there are a number of oversights throughout this section that require careful attention. None of these are major points, and should take little time to amend. Yet, need to be addressed prior to potential resubmissions of this manuscript.
Author’s response: Thank you for your comments, we have addressed these below.

Reviewer’s comment: Figure 1: I am unsure how much this figure aids the interpretation of the study design. I personally do not feel the visual nature of this schematic clarifies the description of the trials from the text, as the study is relatively simplistic. I do not insist that the authors remove this figure, however, please consider my suggestion that it may not add value to the manuscript and may just be a “figure for the sake of a figure”. Secondly, between the trials, the authors have stated that washout periods of > 7 days were implemented. The authors should consider including the range of days (e.g. 7-14 days), as this would provide further information to the reader and allow for more accurate interpretation of the results.
Author’s response: Thank you for your comment. We have decided to keep the figure to ensure clarity as we used a cross-over randomized design, which is unusual for a study examining the reliability and validity of protocols. We have included 7-14 days as a washout period as suggested in the figure.

Reviewer’s comment: Can a citation or elaboration be provided to rationalise the >15 min rest interval between attempts?
Author’s response: A citation has been included with a study that used a 15-minute recovery period in-between repeated sprint and COD protocols in the one testing session.

Reviewer’s comment: The term ‘respectively’ needs to be included after the explanation of the 1-10 muscle rating scale. Also, a brief sentence as to why this was collected should accompany this methodological point.
Author’s response: Included as requested.

Reviewer’s comment: The authors state that the warm up was the same prior to each session. I would encourage using the term ‘standardised’ when describing this in their methods to add certainty around their controls. Also, a citation needs to accompany the statement that the CMJ test was implemented to assess recovery. If this is what was intended by the insertion of citation 12, I would encourage moving this citation to the end of the sentence.
Author’s response: The term ‘standardised’ has been included when describing the warm-up as requested in the Methods section. The citation explaining the use of CMJ to determine recovery dynamics was moved to the end of the sentence as requested.

Reviewer’s comment: Participant details – participant height needs to be reported to 2dp. Can the authors further elaborate on what the “School of Athletic Excellence” is? For example, what standard is this, is it affiliated to a specific school, or region performance programme? This may not be clear to readers.
Author’s response: The participant height is reported to 2dp (1.77m). Further information on the excellence program has been included.

Reviewer’s comment: Multistage shuttle test – a brief sentence stating how performance or a score was derived is necessary (i.e. maximum number of shuttles completed, level, etc.). This information is currently not evident.
Author’s response: The VO2max was estimated based on the level completed, which has now been included in the Methods section.

Reviewer’s comment: CMJ – please state the unit of measurement for this test, and to what degree (e.g. recorded in cm to the nearest 0.1cm).
Author’s response: The CMJ was recorded to the full cm, not to the nearest 0.1cm. This has now been clarified.

Reviewer’s comment: 30-m sprint and agility protocols – please separate each test into a separate sentence. At present, they appear too cramped together and it is difficult to read (particularly with reference to figures and citations within). Also, please provide citations for the original studies for the T-drill and Illinois agility tests. These need to be credited within the manuscript.
Author’s response: The sentences for the description of the protocols have been separated as requested. The original studies have also been included as requested.

Reviewer’s comment: Repeated sprint and agility protocols – were these tests performed one after the other (what I interpret from the term ‘cycles’) or were they performed independently? Clarity is required to ensure the reader understands which of these interpretations is correct.
Author’s response: We have reworded the term ‘cycles’ to ‘repetitions’ to improve clarity. As mentioned in Research Design and in Figure 1, RSA was conducted in Session 2, whilst RTT and RIA were conducted in the same session with 15-min rest in-between each protocol. The Figure reinforces this design, and rewording ‘cycles’ to ‘repetitions’ improves the clarity of the protocols.

Reviewer’s comment: The sentence starting “Immediately after each repeated agility cycle…” does not make sense. Please rephrase this.
Author’s response: This has been reworded to, “The participant’s heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions”

Reviewer’s comment: Statistical analysis – Reference 16 seems a weak reference for this point. I encourage the authors to consider a more suitable citation for selecting this acceptable correlation cut off point.
Author’s response: Additional reference included as requested.

Results
Reviewer’s comment: The section of the results dedicated to the muscle soreness and CMJ differences (or lack thereof) should be condensed to a singular sentence and placed at the foot of the results section. This is a methodological control and not a key finding from your study, therefore, it should not feature so early and heavily within your results section. Stating that there were no differences, providing ES, and significance values would suffice.
Author’s response: Simplified and included at the end of the Results section as requested.

Reviewer’s comment: Table 3 – I am interested in your rationale for not providing ES to demonstrate differences (or lack thereof) for your test-retest data? This would add value in my opinion. I would also encourage the authors to present CV% data to 1dp both within this table and throughout the manuscript.
Author’s response: Thank you for your comment. The ES was not reported as we based the measurement error on CV, with larger CV% values exhibiting greater measurement error and poorer test-retest reliability. In our opinion, inclusion of ES would not provide any further clarity about the reliability of the tests, beyond that of CV.

Reviewer’s comment: I notice that the authors use terminology such as “excellent” or “moderate” test-retest reliability, yet they have not included these threshold within the statistical analysis section of the methods. Please include this detail so that the reader understands your criteria.
Author’s response: Included as requested.

Reviewer’s comment: I also note that the authors say “most RIA performance measures exhibited excellent test-retest reliability…”. Could the authors rephrase this to say “all RIA performance measures except… measures exhibited excellent test-retest reliability”? This would be more accurate and less ambiguous for the reader. This comment also applies later in this section where the authors refer to “a few variables…”.
Author’s response: Thank you for your suggestion. We have reworded this to, “Most RIA measures exhibited excellent test-retest reliability (ICC = 0.92-0.97), good levels of agreement (ratio LOA = 1.05-1.06) and low measurement error (CV = 2.17 - 2.68%) (Table 3). However, FI and average RPE demonstrated moderate test-retest reliability (ICC = 0.87 and 0.76, respectively), poorer levels of agreement (ratio LOA = 2.57 and 2.23, respectively) and higher measurement error (CV = 25.3 and 15.8%, respectively, Table 3).” The next paragraph has been rewritten to, “For the RTT, excellent test-retest reliability (ICC = 0.91), good levels of agreement (ratio LOA = 1.08) and low measurement error (CV = 2.17-2.68%) were identified for TT, BT and AT (Table 3), although maximum RPE demonstrated higher levels of measurement error (CV = 12.3%) between Sessions 4 and 5 (Table 3).”

Discussion
Reviewer’s comment: I found the discussion well-written and thought the authors accurately interpreted and attempted to explain their findings. However, similar to the introduction, I felt this section lacked depth when discussing the main findings of the study. I encourage the authors to discuss their findings in greater depth, utilising some of the reading attached to this review.
Author’s response: Thank you for your comments. The Discussion has been adjusted to address the comments below.

Reviewer’s comment: I encourage the authors to utilise the first paragraph of the discussion to summarise all of their main findings, and then progress to discussing each point in turn.
Author’s response: Summary of findings included as requested in the first paragraph.

Reviewer’s comment: The third paragraph begins “another novelty of this study”, yet this appears to be the first time novelty is addressed in the discussion?
Author’s response: This specific text has been deleted to improve clarity.

Reviewer’s comment: The fourth paragraph can draw upon a wealth of correlational studies to further support the findings observed here.
Young WB, Dawson B, Henry GJ. Agility and Change-of-Direction Speed are Independent Skills: Implications for Training for Agility in Invasion Sports. International Journal of Sports Science & Coaching. 2015;10(1):159-169. doi:10.1260/1747-9541.10.1.1591
Spiteri T, Nimphius S, Hart NH, Specos C, Sheppard JM, Newton RU. Contribution of strength characteristics to change of direction and agility performance in female basketball athletes. J Strength Cond Res. 2014 Sep;28(9):2415-23. doi: 10.1519/JSC.0000000000000547. PMID: 24875426.7
Gabbett, T. J., Kelly, J. N., & Sheppard, J. M. (2008). Speed, Change of Direction Speed, and Reactive Agility of Rugby League Players. The Journal of Strength & Conditioning Research, 22, 174-181. http://dx.doi.org/10.1519/JSC.0b013e31815ef7008
Sheppard, J.M.; Young, W.B.; Doyle, T.L.A.; Sheppard, T.A.; Newton, R.U. An evaluation of a new test of reactive agility and its relationship to sprint speed and change of direction speed. J. Sci Med. Sport 2006, 9, 342–349.9
Author’s response: A sentence with relevant references listed above were included as requested.

Reviewer’s comment: The conclusion should be reframed around the key objectives of the study (i.e. that it is reliable and valid). The mention of aerobic and anaerobic energy systems seems misplaced here.
Author’s response: A sentence on the validity of the RIA protocols has been included.

References
Reviewer’s comment: While a variety of key texts are identified and listed within the bibliography and cited throughout the manuscript, a number of key texts are omitted. I encourage the authors to further familiarise themselves with relevant studies assessing reliability and validity of COD and agility in team sport athletes, as well as the texts identified earlier within this review.
Dos’Santos, T.; Thomas, C.; Jones, P.A.; Comfort, P. Mechanical Determinants of Faster Change of Direction Speed Performance in Male Athletes. J. Strength Cond. Res. 2017, 31, 696–705.10
Gabbett, T.J.; Kelly, J.N.; Sheppard, J.M. Speed, change of direction speed, and reactive agility of rugby league players. J. Strength Cond. Res. 2008, 22, 174–181.8
Oliver, J.L.; Meyers, R.W. Reliability and generality of measures of acceleration, planned agility, and reactive agility. Int. J. Sports Physiol. Perform. 2009, 4, 345–354.11
Author’s response: Thank you for kindly providing the references. A number of these have been included in the paper as requested.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 08 Oct 2021

Kenji Doma, Sport & Exercise Science, James Cook University, Douglas, 4814, Australia

08 Oct 2021

Author Response

Reviewer’s comment: The present study aimed to explore the validity and reliability of repeated “agility” tests in a sample of youth rugby players. The study provides interesting data for an ... Continue reading Reviewer’s comment: The present study aimed to explore the validity and reliability of repeated “agility” tests in a sample of youth rugby players. The study provides interesting data for an applied audience and is methodologically sound and accurate in the presentation of findings. This being said, I have identified a range of areas that should be improved prior to potential resubmissions of this manuscript. For example, the introduction (and study in its entirety) is inaccurate in its use of the term “agility” (opposed to COD which the authors actually assess). Furthermore, the introduction needs to be further developed to support the scope of the study. The authors need to create more of a rationale for running such a large number of correlations – exploring correlational analyses needs to be warranted based upon an assumption of a relationship (or lack thereof) between variables. In its current form, I am unsure whether this manuscript provides such rationale. The methods and results are accurate, but with a large number of small errors. Finally, the discussion addresses key discussion points formulated from the findings of the study, but similar to the introduction, lacks some critical depth. I do think that the authors have a publishable study here, however, a number of revisions will be required before I would be happy to approve this manuscript for indexing. I hope that my comments are received as I intend them and are useful in further strengthening this manuscript. I hope to receive the opportunity to review this manuscript again in its revised form.
Author’s response: Dear reviewer, thank you for your comments on the paper. We agree that your comments have improved the quality of our work.

Abstract
Reviewer’s comment: The abstract is concise and accurate to the details of the study. I have just a couple of minor comments (below):
The initial sentence is accurate and outlines the requirement of the study. However, I would encourage the authors to insert a word such as ‘simultaneously’ or ‘congruently’, or attempt to capture the lack of a suitable test to capture the performance of these abilities in combination.
Author’s response: The word ‘simultaneously’ has been included.

Reviewer’s comment: Abstract Methods – the inclusion of the repeated T-agility test appears suddenly with no prior reference. If this is a key concentration of your study, I would encourage to insert earlier in the abstract along with the RIA.
Author’s response: Repeated T-agility protocol now mentioned in the Background.

Reviewer’s comment: Abstract Results – please specify exact p values rather than just <0.05. CV% can be presented to 1dp for brevity.
Author’s response: We were unable to provide exact p-values, as several outcome measures from the repeated COD protocols, such as average time, best time, total time etc were presented as “performances”. This same approach was used for the r-values as well to ensure the word limit did not exceed journal guidelines. However, we did present the CV% to 1dp as requested.

Introduction
Reviewer’s comment: The introduction is short and concise. However, I feel some key information that would further support the rationale for your study has been omitted. I would encourage the authors to further develop this section. A suggestion would be to expand the second paragraph into two: the first discussing the validity aspects, the second the reliability aspects. I do not expect these changes to be too onerous, but believe it would strengthen this section of your manuscript if you were to implement these changes. I have some further specific comments below.
Author’s response: The second paragraph has been further developed into two as requested, with the first focusing on reliability and the second on validity.

Reviewer’s comment: The opening sentence would read better if it were split into two separate sentences.
Author’s response: We have simplified the first sentence by removing the context of ‘agility’, as we have converted all ‘agility’ terms to ‘change-of-direction’.

Reviewer’s comment: The authors should attempt to clarify during opening paragraph that they are discussing change of direction (COD) performance, rather than agility. Although COD performance is a key component of agility performance, the lack of an external stimuli prevents classification of agility. Although the tests identified by the authors in this opening paragraph have titled themselves ‘agility’ tests, by definition they are not. The authors should be careful in their stance here as the literature has evolved substantially since the inception of tests such as these, and appropriate classification and terminology is essential for the present study.
Young, W.; Dawson, B.; Henry, G. Agility and Change-of-Direction Speed are Independent Skills: Implications for Agility in Invasion Sports. Int. J. Sports Sci. Coach. 2015, 10, 159–169.1
Nimphius, S.; Callaghan, S.J.; Bezodis, N.E.; Lockie, R.G. Change of Direction and Agility Tests: Challenging Our Current Measures of Performance. Strength Cond. J. 2018, 40, 26–38. 2
Sheppard, J.M.; Young, W.B. Agility literature review: Classifications, training and testing. J. Sports Sci. 2006, 24, 919–932.3
Author’s response: Thank you for your suggestion. As mentioned earlier, we have changed all ‘agility’ terms into ‘change-of-direction’.

Reviewer’s comment: Regarding the reliability aspect of the study, two recent papers have suggested that reliability of COD and agility may be lower in adolescent and youth athletes (as per your sample). Although these authors observed this during maximal COD/agility performance, opposed to repeated as in the present study, I believe this would be worth including and would further warrant the exploration of your research question and study.
Taylor, J.M.; Cunningham, L.; Hood, P.; Thorne, B.; Irvin, G.; Weston, M. The reliability of a modified 505 test and change-of-direction deficit time in elite youth football players. Sci. Med. Footb. 2019, 3, 157–162.4
Dugdale JH, Sanders D, Hunter AM. Reliability of Change of Direction and Agility Assessments in Youth Soccer Players. Sports (Basel). 2020 Apr 18; 8 (4): 51. doi: 10.3390/sports8040051.5
James H. Dugdale, Calum A. Arthur, Dajo Sanders & Angus M. Hunter (2019) Reliability and validity of field-based fitness tests in youth soccer players, European Journal of Sport Science, 19:6, 745-756, DOI: 10.1080/17461391.2018.15567396
Author’s response: Thank you for your suggestion, we have included the references above in the paragraph explaining studies on the reliability of COD protocols.

Methods
Reviewer’s comment: The methods report the appropriate details relating to the study design, protocols, participants, and analyses performed. However, further clarity is required in a few places throughout this section (specific comments below). Further, there are a number of oversights throughout this section that require careful attention. None of these are major points, and should take little time to amend. Yet, need to be addressed prior to potential resubmissions of this manuscript.
Author’s response: Thank you for your comments, we have addressed these below.

Reviewer’s comment: Figure 1: I am unsure how much this figure aids the interpretation of the study design. I personally do not feel the visual nature of this schematic clarifies the description of the trials from the text, as the study is relatively simplistic. I do not insist that the authors remove this figure, however, please consider my suggestion that it may not add value to the manuscript and may just be a “figure for the sake of a figure”. Secondly, between the trials, the authors have stated that washout periods of > 7 days were implemented. The authors should consider including the range of days (e.g. 7-14 days), as this would provide further information to the reader and allow for more accurate interpretation of the results.
Author’s response: Thank you for your comment. We have decided to keep the figure to ensure clarity as we used a cross-over randomized design, which is unusual for a study examining the reliability and validity of protocols. We have included 7-14 days as a washout period as suggested in the figure.

Reviewer’s comment: Can a citation or elaboration be provided to rationalise the >15 min rest interval between attempts?
Author’s response: A citation has been included with a study that used a 15-minute recovery period in-between repeated sprint and COD protocols in the one testing session.

Reviewer’s comment: The term ‘respectively’ needs to be included after the explanation of the 1-10 muscle rating scale. Also, a brief sentence as to why this was collected should accompany this methodological point.
Author’s response: Included as requested.

Reviewer’s comment: The authors state that the warm up was the same prior to each session. I would encourage using the term ‘standardised’ when describing this in their methods to add certainty around their controls. Also, a citation needs to accompany the statement that the CMJ test was implemented to assess recovery. If this is what was intended by the insertion of citation 12, I would encourage moving this citation to the end of the sentence.
Author’s response: The term ‘standardised’ has been included when describing the warm-up as requested in the Methods section. The citation explaining the use of CMJ to determine recovery dynamics was moved to the end of the sentence as requested.

Reviewer’s comment: Participant details – participant height needs to be reported to 2dp. Can the authors further elaborate on what the “School of Athletic Excellence” is? For example, what standard is this, is it affiliated to a specific school, or region performance programme? This may not be clear to readers.
Author’s response: The participant height is reported to 2dp (1.77m). Further information on the excellence program has been included.

Reviewer’s comment: Multistage shuttle test – a brief sentence stating how performance or a score was derived is necessary (i.e. maximum number of shuttles completed, level, etc.). This information is currently not evident.
Author’s response: The VO2max was estimated based on the level completed, which has now been included in the Methods section.

Reviewer’s comment: CMJ – please state the unit of measurement for this test, and to what degree (e.g. recorded in cm to the nearest 0.1cm).
Author’s response: The CMJ was recorded to the full cm, not to the nearest 0.1cm. This has now been clarified.

Reviewer’s comment: 30-m sprint and agility protocols – please separate each test into a separate sentence. At present, they appear too cramped together and it is difficult to read (particularly with reference to figures and citations within). Also, please provide citations for the original studies for the T-drill and Illinois agility tests. These need to be credited within the manuscript.
Author’s response: The sentences for the description of the protocols have been separated as requested. The original studies have also been included as requested.

Reviewer’s comment: Repeated sprint and agility protocols – were these tests performed one after the other (what I interpret from the term ‘cycles’) or were they performed independently? Clarity is required to ensure the reader understands which of these interpretations is correct.
Author’s response: We have reworded the term ‘cycles’ to ‘repetitions’ to improve clarity. As mentioned in Research Design and in Figure 1, RSA was conducted in Session 2, whilst RTT and RIA were conducted in the same session with 15-min rest in-between each protocol. The Figure reinforces this design, and rewording ‘cycles’ to ‘repetitions’ improves the clarity of the protocols.

Reviewer’s comment: The sentence starting “Immediately after each repeated agility cycle…” does not make sense. Please rephrase this.
Author’s response: This has been reworded to, “The participant’s heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions”

Reviewer’s comment: Statistical analysis – Reference 16 seems a weak reference for this point. I encourage the authors to consider a more suitable citation for selecting this acceptable correlation cut off point.
Author’s response: Additional reference included as requested.

Results
Reviewer’s comment: The section of the results dedicated to the muscle soreness and CMJ differences (or lack thereof) should be condensed to a singular sentence and placed at the foot of the results section. This is a methodological control and not a key finding from your study, therefore, it should not feature so early and heavily within your results section. Stating that there were no differences, providing ES, and significance values would suffice.
Author’s response: Simplified and included at the end of the Results section as requested.

Reviewer’s comment: Table 3 – I am interested in your rationale for not providing ES to demonstrate differences (or lack thereof) for your test-retest data? This would add value in my opinion. I would also encourage the authors to present CV% data to 1dp both within this table and throughout the manuscript.
Author’s response: Thank you for your comment. The ES was not reported as we based the measurement error on CV, with larger CV% values exhibiting greater measurement error and poorer test-retest reliability. In our opinion, inclusion of ES would not provide any further clarity about the reliability of the tests, beyond that of CV.

Reviewer’s comment: I notice that the authors use terminology such as “excellent” or “moderate” test-retest reliability, yet they have not included these threshold within the statistical analysis section of the methods. Please include this detail so that the reader understands your criteria.
Author’s response: Included as requested.

Reviewer’s comment: I also note that the authors say “most RIA performance measures exhibited excellent test-retest reliability…”. Could the authors rephrase this to say “all RIA performance measures except… measures exhibited excellent test-retest reliability”? This would be more accurate and less ambiguous for the reader. This comment also applies later in this section where the authors refer to “a few variables…”.
Author’s response: Thank you for your suggestion. We have reworded this to, “Most RIA measures exhibited excellent test-retest reliability (ICC = 0.92-0.97), good levels of agreement (ratio LOA = 1.05-1.06) and low measurement error (CV = 2.17 - 2.68%) (Table 3). However, FI and average RPE demonstrated moderate test-retest reliability (ICC = 0.87 and 0.76, respectively), poorer levels of agreement (ratio LOA = 2.57 and 2.23, respectively) and higher measurement error (CV = 25.3 and 15.8%, respectively, Table 3).” The next paragraph has been rewritten to, “For the RTT, excellent test-retest reliability (ICC = 0.91), good levels of agreement (ratio LOA = 1.08) and low measurement error (CV = 2.17-2.68%) were identified for TT, BT and AT (Table 3), although maximum RPE demonstrated higher levels of measurement error (CV = 12.3%) between Sessions 4 and 5 (Table 3).”

Discussion
Reviewer’s comment: I found the discussion well-written and thought the authors accurately interpreted and attempted to explain their findings. However, similar to the introduction, I felt this section lacked depth when discussing the main findings of the study. I encourage the authors to discuss their findings in greater depth, utilising some of the reading attached to this review.
Author’s response: Thank you for your comments. The Discussion has been adjusted to address the comments below.

Reviewer’s comment: I encourage the authors to utilise the first paragraph of the discussion to summarise all of their main findings, and then progress to discussing each point in turn.
Author’s response: Summary of findings included as requested in the first paragraph.

Reviewer’s comment: The third paragraph begins “another novelty of this study”, yet this appears to be the first time novelty is addressed in the discussion?
Author’s response: This specific text has been deleted to improve clarity.

Reviewer’s comment: The fourth paragraph can draw upon a wealth of correlational studies to further support the findings observed here.
Young WB, Dawson B, Henry GJ. Agility and Change-of-Direction Speed are Independent Skills: Implications for Training for Agility in Invasion Sports. International Journal of Sports Science & Coaching. 2015;10(1):159-169. doi:10.1260/1747-9541.10.1.1591
Spiteri T, Nimphius S, Hart NH, Specos C, Sheppard JM, Newton RU. Contribution of strength characteristics to change of direction and agility performance in female basketball athletes. J Strength Cond Res. 2014 Sep;28(9):2415-23. doi: 10.1519/JSC.0000000000000547. PMID: 24875426.7
Gabbett, T. J., Kelly, J. N., & Sheppard, J. M. (2008). Speed, Change of Direction Speed, and Reactive Agility of Rugby League Players. The Journal of Strength & Conditioning Research, 22, 174-181. http://dx.doi.org/10.1519/JSC.0b013e31815ef7008
Sheppard, J.M.; Young, W.B.; Doyle, T.L.A.; Sheppard, T.A.; Newton, R.U. An evaluation of a new test of reactive agility and its relationship to sprint speed and change of direction speed. J. Sci Med. Sport 2006, 9, 342–349.9
Author’s response: A sentence with relevant references listed above were included as requested.

Reviewer’s comment: The conclusion should be reframed around the key objectives of the study (i.e. that it is reliable and valid). The mention of aerobic and anaerobic energy systems seems misplaced here.
Author’s response: A sentence on the validity of the RIA protocols has been included.

References
Reviewer’s comment: While a variety of key texts are identified and listed within the bibliography and cited throughout the manuscript, a number of key texts are omitted. I encourage the authors to further familiarise themselves with relevant studies assessing reliability and validity of COD and agility in team sport athletes, as well as the texts identified earlier within this review.
Dos’Santos, T.; Thomas, C.; Jones, P.A.; Comfort, P. Mechanical Determinants of Faster Change of Direction Speed Performance in Male Athletes. J. Strength Cond. Res. 2017, 31, 696–705.10
Gabbett, T.J.; Kelly, J.N.; Sheppard, J.M. Speed, change of direction speed, and reactive agility of rugby league players. J. Strength Cond. Res. 2008, 22, 174–181.8
Oliver, J.L.; Meyers, R.W. Reliability and generality of measures of acceleration, planned agility, and reactive agility. Int. J. Sports Physiol. Perform. 2009, 4, 345–354.11
Author’s response: Thank you for kindly providing the references. A number of these have been included in the paper as requested.
Reviewer’s comment: The present study aimed to explore the validity and reliability of repeated “agility” tests in a sample of youth rugby players. The study provides interesting data for an applied audience and is methodologically sound and accurate in the presentation of findings. This being said, I have identified a range of areas that should be improved prior to potential resubmissions of this manuscript. For example, the introduction (and study in its entirety) is inaccurate in its use of the term “agility” (opposed to COD which the authors actually assess). Furthermore, the introduction needs to be further developed to support the scope of the study. The authors need to create more of a rationale for running such a large number of correlations – exploring correlational analyses needs to be warranted based upon an assumption of a relationship (or lack thereof) between variables. In its current form, I am unsure whether this manuscript provides such rationale. The methods and results are accurate, but with a large number of small errors. Finally, the discussion addresses key discussion points formulated from the findings of the study, but similar to the introduction, lacks some critical depth. I do think that the authors have a publishable study here, however, a number of revisions will be required before I would be happy to approve this manuscript for indexing. I hope that my comments are received as I intend them and are useful in further strengthening this manuscript. I hope to receive the opportunity to review this manuscript again in its revised form.
Author’s response: Dear reviewer, thank you for your comments on the paper. We agree that your comments have improved the quality of our work.

Abstract
Reviewer’s comment: The abstract is concise and accurate to the details of the study. I have just a couple of minor comments (below):
The initial sentence is accurate and outlines the requirement of the study. However, I would encourage the authors to insert a word such as ‘simultaneously’ or ‘congruently’, or attempt to capture the lack of a suitable test to capture the performance of these abilities in combination.
Author’s response: The word ‘simultaneously’ has been included.

Reviewer’s comment: Abstract Methods – the inclusion of the repeated T-agility test appears suddenly with no prior reference. If this is a key concentration of your study, I would encourage to insert earlier in the abstract along with the RIA.
Author’s response: Repeated T-agility protocol now mentioned in the Background.

Reviewer’s comment: Abstract Results – please specify exact p values rather than just <0.05. CV% can be presented to 1dp for brevity.
Author’s response: We were unable to provide exact p-values, as several outcome measures from the repeated COD protocols, such as average time, best time, total time etc were presented as “performances”. This same approach was used for the r-values as well to ensure the word limit did not exceed journal guidelines. However, we did present the CV% to 1dp as requested.

Introduction
Reviewer’s comment: The introduction is short and concise. However, I feel some key information that would further support the rationale for your study has been omitted. I would encourage the authors to further develop this section. A suggestion would be to expand the second paragraph into two: the first discussing the validity aspects, the second the reliability aspects. I do not expect these changes to be too onerous, but believe it would strengthen this section of your manuscript if you were to implement these changes. I have some further specific comments below.
Author’s response: The second paragraph has been further developed into two as requested, with the first focusing on reliability and the second on validity.

Reviewer’s comment: The opening sentence would read better if it were split into two separate sentences.
Author’s response: We have simplified the first sentence by removing the context of ‘agility’, as we have converted all ‘agility’ terms to ‘change-of-direction’.

Reviewer’s comment: The authors should attempt to clarify during opening paragraph that they are discussing change of direction (COD) performance, rather than agility. Although COD performance is a key component of agility performance, the lack of an external stimuli prevents classification of agility. Although the tests identified by the authors in this opening paragraph have titled themselves ‘agility’ tests, by definition they are not. The authors should be careful in their stance here as the literature has evolved substantially since the inception of tests such as these, and appropriate classification and terminology is essential for the present study.
Young, W.; Dawson, B.; Henry, G. Agility and Change-of-Direction Speed are Independent Skills: Implications for Agility in Invasion Sports. Int. J. Sports Sci. Coach. 2015, 10, 159–169.1
Nimphius, S.; Callaghan, S.J.; Bezodis, N.E.; Lockie, R.G. Change of Direction and Agility Tests: Challenging Our Current Measures of Performance. Strength Cond. J. 2018, 40, 26–38. 2
Sheppard, J.M.; Young, W.B. Agility literature review: Classifications, training and testing. J. Sports Sci. 2006, 24, 919–932.3
Author’s response: Thank you for your suggestion. As mentioned earlier, we have changed all ‘agility’ terms into ‘change-of-direction’.

Reviewer’s comment: Regarding the reliability aspect of the study, two recent papers have suggested that reliability of COD and agility may be lower in adolescent and youth athletes (as per your sample). Although these authors observed this during maximal COD/agility performance, opposed to repeated as in the present study, I believe this would be worth including and would further warrant the exploration of your research question and study.
Taylor, J.M.; Cunningham, L.; Hood, P.; Thorne, B.; Irvin, G.; Weston, M. The reliability of a modified 505 test and change-of-direction deficit time in elite youth football players. Sci. Med. Footb. 2019, 3, 157–162.4
Dugdale JH, Sanders D, Hunter AM. Reliability of Change of Direction and Agility Assessments in Youth Soccer Players. Sports (Basel). 2020 Apr 18; 8 (4): 51. doi: 10.3390/sports8040051.5
James H. Dugdale, Calum A. Arthur, Dajo Sanders & Angus M. Hunter (2019) Reliability and validity of field-based fitness tests in youth soccer players, European Journal of Sport Science, 19:6, 745-756, DOI: 10.1080/17461391.2018.15567396
Author’s response: Thank you for your suggestion, we have included the references above in the paragraph explaining studies on the reliability of COD protocols.

Methods
Reviewer’s comment: The methods report the appropriate details relating to the study design, protocols, participants, and analyses performed. However, further clarity is required in a few places throughout this section (specific comments below). Further, there are a number of oversights throughout this section that require careful attention. None of these are major points, and should take little time to amend. Yet, need to be addressed prior to potential resubmissions of this manuscript.
Author’s response: Thank you for your comments, we have addressed these below.

Reviewer’s comment: Figure 1: I am unsure how much this figure aids the interpretation of the study design. I personally do not feel the visual nature of this schematic clarifies the description of the trials from the text, as the study is relatively simplistic. I do not insist that the authors remove this figure, however, please consider my suggestion that it may not add value to the manuscript and may just be a “figure for the sake of a figure”. Secondly, between the trials, the authors have stated that washout periods of > 7 days were implemented. The authors should consider including the range of days (e.g. 7-14 days), as this would provide further information to the reader and allow for more accurate interpretation of the results.
Author’s response: Thank you for your comment. We have decided to keep the figure to ensure clarity as we used a cross-over randomized design, which is unusual for a study examining the reliability and validity of protocols. We have included 7-14 days as a washout period as suggested in the figure.

Reviewer’s comment: Can a citation or elaboration be provided to rationalise the >15 min rest interval between attempts?
Author’s response: A citation has been included with a study that used a 15-minute recovery period in-between repeated sprint and COD protocols in the one testing session.

Reviewer’s comment: The term ‘respectively’ needs to be included after the explanation of the 1-10 muscle rating scale. Also, a brief sentence as to why this was collected should accompany this methodological point.
Author’s response: Included as requested.

Reviewer’s comment: The authors state that the warm up was the same prior to each session. I would encourage using the term ‘standardised’ when describing this in their methods to add certainty around their controls. Also, a citation needs to accompany the statement that the CMJ test was implemented to assess recovery. If this is what was intended by the insertion of citation 12, I would encourage moving this citation to the end of the sentence.
Author’s response: The term ‘standardised’ has been included when describing the warm-up as requested in the Methods section. The citation explaining the use of CMJ to determine recovery dynamics was moved to the end of the sentence as requested.

Reviewer’s comment: Participant details – participant height needs to be reported to 2dp. Can the authors further elaborate on what the “School of Athletic Excellence” is? For example, what standard is this, is it affiliated to a specific school, or region performance programme? This may not be clear to readers.
Author’s response: The participant height is reported to 2dp (1.77m). Further information on the excellence program has been included.

Reviewer’s comment: Multistage shuttle test – a brief sentence stating how performance or a score was derived is necessary (i.e. maximum number of shuttles completed, level, etc.). This information is currently not evident.
Author’s response: The VO2max was estimated based on the level completed, which has now been included in the Methods section.

Reviewer’s comment: CMJ – please state the unit of measurement for this test, and to what degree (e.g. recorded in cm to the nearest 0.1cm).
Author’s response: The CMJ was recorded to the full cm, not to the nearest 0.1cm. This has now been clarified.

Reviewer’s comment: 30-m sprint and agility protocols – please separate each test into a separate sentence. At present, they appear too cramped together and it is difficult to read (particularly with reference to figures and citations within). Also, please provide citations for the original studies for the T-drill and Illinois agility tests. These need to be credited within the manuscript.
Author’s response: The sentences for the description of the protocols have been separated as requested. The original studies have also been included as requested.

Reviewer’s comment: Repeated sprint and agility protocols – were these tests performed one after the other (what I interpret from the term ‘cycles’) or were they performed independently? Clarity is required to ensure the reader understands which of these interpretations is correct.
Author’s response: We have reworded the term ‘cycles’ to ‘repetitions’ to improve clarity. As mentioned in Research Design and in Figure 1, RSA was conducted in Session 2, whilst RTT and RIA were conducted in the same session with 15-min rest in-between each protocol. The Figure reinforces this design, and rewording ‘cycles’ to ‘repetitions’ improves the clarity of the protocols.

Reviewer’s comment: The sentence starting “Immediately after each repeated agility cycle…” does not make sense. Please rephrase this.
Author’s response: This has been reworded to, “The participant’s heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions”

Reviewer’s comment: Statistical analysis – Reference 16 seems a weak reference for this point. I encourage the authors to consider a more suitable citation for selecting this acceptable correlation cut off point.
Author’s response: Additional reference included as requested.

Results
Reviewer’s comment: The section of the results dedicated to the muscle soreness and CMJ differences (or lack thereof) should be condensed to a singular sentence and placed at the foot of the results section. This is a methodological control and not a key finding from your study, therefore, it should not feature so early and heavily within your results section. Stating that there were no differences, providing ES, and significance values would suffice.
Author’s response: Simplified and included at the end of the Results section as requested.

Reviewer’s comment: Table 3 – I am interested in your rationale for not providing ES to demonstrate differences (or lack thereof) for your test-retest data? This would add value in my opinion. I would also encourage the authors to present CV% data to 1dp both within this table and throughout the manuscript.
Author’s response: Thank you for your comment. The ES was not reported as we based the measurement error on CV, with larger CV% values exhibiting greater measurement error and poorer test-retest reliability. In our opinion, inclusion of ES would not provide any further clarity about the reliability of the tests, beyond that of CV.

Reviewer’s comment: I notice that the authors use terminology such as “excellent” or “moderate” test-retest reliability, yet they have not included these threshold within the statistical analysis section of the methods. Please include this detail so that the reader understands your criteria.
Author’s response: Included as requested.

Reviewer’s comment: I also note that the authors say “most RIA performance measures exhibited excellent test-retest reliability…”. Could the authors rephrase this to say “all RIA performance measures except… measures exhibited excellent test-retest reliability”? This would be more accurate and less ambiguous for the reader. This comment also applies later in this section where the authors refer to “a few variables…”.
Author’s response: Thank you for your suggestion. We have reworded this to, “Most RIA measures exhibited excellent test-retest reliability (ICC = 0.92-0.97), good levels of agreement (ratio LOA = 1.05-1.06) and low measurement error (CV = 2.17 - 2.68%) (Table 3). However, FI and average RPE demonstrated moderate test-retest reliability (ICC = 0.87 and 0.76, respectively), poorer levels of agreement (ratio LOA = 2.57 and 2.23, respectively) and higher measurement error (CV = 25.3 and 15.8%, respectively, Table 3).” The next paragraph has been rewritten to, “For the RTT, excellent test-retest reliability (ICC = 0.91), good levels of agreement (ratio LOA = 1.08) and low measurement error (CV = 2.17-2.68%) were identified for TT, BT and AT (Table 3), although maximum RPE demonstrated higher levels of measurement error (CV = 12.3%) between Sessions 4 and 5 (Table 3).”

Discussion
Reviewer’s comment: I found the discussion well-written and thought the authors accurately interpreted and attempted to explain their findings. However, similar to the introduction, I felt this section lacked depth when discussing the main findings of the study. I encourage the authors to discuss their findings in greater depth, utilising some of the reading attached to this review.
Author’s response: Thank you for your comments. The Discussion has been adjusted to address the comments below.

Reviewer’s comment: I encourage the authors to utilise the first paragraph of the discussion to summarise all of their main findings, and then progress to discussing each point in turn.
Author’s response: Summary of findings included as requested in the first paragraph.

Reviewer’s comment: The third paragraph begins “another novelty of this study”, yet this appears to be the first time novelty is addressed in the discussion?
Author’s response: This specific text has been deleted to improve clarity.

Reviewer’s comment: The fourth paragraph can draw upon a wealth of correlational studies to further support the findings observed here.
Young WB, Dawson B, Henry GJ. Agility and Change-of-Direction Speed are Independent Skills: Implications for Training for Agility in Invasion Sports. International Journal of Sports Science & Coaching. 2015;10(1):159-169. doi:10.1260/1747-9541.10.1.1591
Spiteri T, Nimphius S, Hart NH, Specos C, Sheppard JM, Newton RU. Contribution of strength characteristics to change of direction and agility performance in female basketball athletes. J Strength Cond Res. 2014 Sep;28(9):2415-23. doi: 10.1519/JSC.0000000000000547. PMID: 24875426.7
Gabbett, T. J., Kelly, J. N., & Sheppard, J. M. (2008). Speed, Change of Direction Speed, and Reactive Agility of Rugby League Players. The Journal of Strength & Conditioning Research, 22, 174-181. http://dx.doi.org/10.1519/JSC.0b013e31815ef7008
Sheppard, J.M.; Young, W.B.; Doyle, T.L.A.; Sheppard, T.A.; Newton, R.U. An evaluation of a new test of reactive agility and its relationship to sprint speed and change of direction speed. J. Sci Med. Sport 2006, 9, 342–349.9
Author’s response: A sentence with relevant references listed above were included as requested.

Reviewer’s comment: The conclusion should be reframed around the key objectives of the study (i.e. that it is reliable and valid). The mention of aerobic and anaerobic energy systems seems misplaced here.
Author’s response: A sentence on the validity of the RIA protocols has been included.

References
Reviewer’s comment: While a variety of key texts are identified and listed within the bibliography and cited throughout the manuscript, a number of key texts are omitted. I encourage the authors to further familiarise themselves with relevant studies assessing reliability and validity of COD and agility in team sport athletes, as well as the texts identified earlier within this review.
Dos’Santos, T.; Thomas, C.; Jones, P.A.; Comfort, P. Mechanical Determinants of Faster Change of Direction Speed Performance in Male Athletes. J. Strength Cond. Res. 2017, 31, 696–705.10
Gabbett, T.J.; Kelly, J.N.; Sheppard, J.M. Speed, change of direction speed, and reactive agility of rugby league players. J. Strength Cond. Res. 2008, 22, 174–181.8
Oliver, J.L.; Meyers, R.W. Reliability and generality of measures of acceleration, planned agility, and reactive agility. Int. J. Sports Physiol. Perform. 2009, 4, 345–354.11
Author’s response: Thank you for kindly providing the references. A number of these have been included in the paper as requested.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 3

VERSION 3 PUBLISHED 17 Jun 2020

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 3 (revision) 22 Nov 21		read
Version 2 (revision) 08 Oct 21	read	read
Version 1 17 Jun 20	read	read

James Dugdale, University of Stirling, Stirling, UK; Edinburgh Napier University, Edinburgh, UK
Aaron T. Scanlan, Central Queensland University, Rockhampton, Australia

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

6 Views

22 Nov 2021 | for Version 3

Aaron T. Scanlan, Human Exercise and Training Laboratory, School of Health, Medical and Applied Sciences, Central Queensland University, Rockhampton, Qld, Australia

6 Views Cite this report Responses(0)

Approved

Thank you for making the requested changes, I have no further comments to add.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Exercise and sport science

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

5 Views

08 Nov 2021 | for Version 2

James Dugdale, School of Applied Sciences, University of Stirling, Stirling, UK; Edinburgh Napier University, Edinburgh, UK

5 Views Cite this report Responses(1)

Approved

Thank you to the authors for their time and efforts taken to revise this manuscript. I am of the opinion that these revisions have strengthened the manuscript substantially.

I am happy to approve the manuscript in its current form.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Fitness testing; youth athletes; talent identification; talent development; physical performance; strength and conditioning

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Back to all reports

Reviewer Report

12 Views

11 Oct 2021 | for Version 2

Aaron T. Scanlan, Human Exercise and Training Laboratory, School of Health, Medical and Applied Sciences, Central Queensland University, Rockhampton, Qld, Australia

12 Views Cite this report Responses(1)

Approved

Abstract - thank you for adding "Beep test", but for consistency with the article text, consider changing this to "Multistage Shuttle test".
Introduction, paragraph 4, second aim - is RSA meant to be aerobic capacity here? In either case, aerobic capacity needs to be included in this aim specifically alongside the other fitness attributes.
Methods, 30-m sprint and agility protocols - be sure to check headings as well given previous suggestions on use of the term "agility". Also in this section, please indicate if the position of the timing gates were kept consistent across all participants with mention of the height (above ground) and distance apart available.
Statistical analysis - still use of "construct validity" on the fourth line in this paragraph as per previous revisions.
Discussion - there is a random equals sign on the last line of the fifth paragraph.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Exercise and sport science

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

22 Nov 2021

Kenji Doma, Sport & Exercise Science, James Cook University, Douglas, 4814, Australia

Reviewer’s comment: Thank you for the opportunity to further review this submission. I thank the authors for giving such extensive consideration to the previous reviewer suggestions and making relevant changes where applicable. The manuscript has been strengthened, and I only have some further minor comments for consideration:
Author’s response: Thank you very much for your comments, we believe your input has strengthened the quality of our work.

Reviewer’s comment: Abstract - thank you for adding "Beep test", but for consistency with the article text, consider changing this to "Multistage Shuttle test".
Author’s response: Amended as requested.

Reviewer’s comment: Introduction, paragraph 4, second aim - is RSA meant to be aerobic capacity here? In either case, aerobic capacity needs to be included in this aim specifically alongside the other fitness attributes.
Author’s response: This is a great pickup, now amended to aerobic capacity.

Reviewer’s comment: Methods, 30-m sprint and agility protocols - be sure to check headings as well given previous suggestions on use of the term "agility". Also in this section, please indicate if the position of the timing gates were kept consistent across all participants with mention of the height (above ground) and distance apart available.
Author’s response: The headings have now been changed to change-of-direction, and the height and position of the gates were described as being consistent for each test.

Reviewer’s comment: Statistical analysis - still use of "construct validity" on the fourth line in this paragraph as per previous revisions.
Author’s response: Changed to construct validity.

Reviewer’s comment: Discussion - there is a random equals sign on the last line of the fifth paragraph.
Author’s response: Thank you, this is now amended.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

20 Views

03 Aug 2021 | for Version 1

Aaron T. Scanlan, Human Exercise and Training Laboratory, School of Health, Medical and Applied Sciences, Central Queensland University, Rockhampton, Qld, Australia

20 Views Cite this report Responses(1)

Approved With Reservations

Abstract, Methods – indicate whether you are referring to the multi-stage fitness test (i.e., Beep test) here or if just an assessment of multi-stage fitness was performed.
Abstract – some justification of using the repeated T-agility test as the other standard repeated change-of-direction performance to assess convergent agility of the repeated Illinois Agility test is needed in the Abstract.
Abstract – the focus of exploring the contribution of different fitness attributes to repeated Illinois Agility test performance is not made in the Abstract. Consider including this as a secondary objective and/or making it clear why this analysis is needed.
Abstract, Conclusions – make it clear that this test is useful for Rugby League coaches specifically, and change “athlete” to plural form or indicate “team” here instead.
Introduction, opening paragraph – in this paragraph you identify “agility” and hint that it involves physical and cognitive components. So in essence, you are examining change-of-direction speed (the physical component) rather than “agility” per se. Consider making this clear and using the term “change-of-direction speed” or “change-of-direction performance” thereafter when referring to tests and attributes that are purely physical without the cognitive component.
Introduction, opening paragraph – stronger rationale is needed justifying the inclusion of the Agility T-test and Illinois Agility test in rugby league. Can you add a sentence or two outlining why these tests are suited to the sport and therefore the focus of your study?
Introduction, end of second paragraph – here indicate whether the validity and reliability of this test has not been investigated at all or just specifically in Rugby League athletes. Also, at the end of this sentence make it clear that you are referring to the usability of this test.
Introduction, aims – for the second aim, you are not comparing the RIA measures to other measures, but instead correlating them, so please change this aim accordingly. Also, consider including a little rationale around this aim in the previous paragraph as it is unclear as to why this is important.
Introduction, aims – it might pay to include a sentence stating why examining convergent validity (e.g. to show that you are assessing similar attributes with a new test that is more practical and specific to Rugby League when compared to a standard, generic test routinely used) and retest reliability (e.g. to detect the inherent error in the test and ascertain whether it can be reliably adopted in practice to assess repeated measurements in athletes) are needed for practical uptake of the test, which would strengthen the rationale of the first and third aims as well in the Introduction section.
Introduction, hypotheses – you only mention speed and anaerobic capacity as your fitness attributes here, but you also included aerobic fitness (Multistage Fitness test)?
Methods, Research design – why was the order flipped between session 4 and session 5? It seems like you would want to the athletes to complete the same exact session across both when assessing retest reliability as it is introducing a confounding factor? Also, no hyphen needed between “15” and “minutes” here.
Methods, Participants – can you provide some further indication as to the specific playing level of the athletes? The name of the program is great, but this is not exactly clear for all readers.
Figure 1 – consider changing the session numbers, as in text you identify the Multistage Fitness test as session 1, but here you indicate the other fitness testing as session 1. Consistency needed.
Methods, Participants – add “in” after “partaking” and make parents/guardians singular rather than plural.
Methods, Multistage Shuttle test – stay consistent with capitalising the test names like this one as it is done inconsistently throughout. Also, hyphenate “20m” here.
Methods, Countermovement jump test – hyphenate “1 cm” here. Also, try to indicate what units the key outcomes from each test were reported in (e.g. mL/kg/min, cm, s).
Methods, 30-m Sprint and Agility protocols – separate statements on linear and change-of-direction speed here. Also, hyphenate “10m” and “5m” for the Illinois Agility test.
Methods, repeat protocols – to calculate average HR and RPE, was HR measured from when the test started to when the test finished in 1-second intervals? And was RPE taken after each effort or just after all efforts for each specific test? These are not quite clear.
Methods, Statistical analysis – you mention “construct” validity for the first time here. If this is a key aim and aspect of the study (i.e., correlating performance during the test with fitness attributes), then this needs to be established earlier (i.e., introduction and aims).
Methods, Statistical analysis – an r value of 0.5 seems quite low to establish convergent validity (only 25% shared variance)? In this regard, what was the cut-off for construct validity?
Results, first paragraph – clarify whether this is CMJ height specifically.
Results, second paragraph – make it clear that you are referring to the third and fourth sessions here (and make sure this is consistent as you identify these as the fourth and fifth sessions earlier).
Results, third paragraph – you do not indicate the maximum RPE was taken as an outcome in the methods anywhere (only average), yet it is listed here? Same for maximum HR, which appears later in this section also.
Table 1 – shouldn’t RIA and RTT be the two key tests going from left to right rather than RIA and RSA? In fact, it is not clear why RSA is included here given you identify RIA vs. RTT for convergent validity assessment throughout the manuscript. Please adjust or rework earlier sections.
Discussion, first paragraph – some minor errors, but change “was” to “were” in the 7^th line.
Discussion, first paragraph – at the end of this paragraph, some explanation as to how yours (and the previous studies) support that anaerobic fitness is predominantly stressed in the repeated COD tests. I am assuming the strong correlations with mostly anaerobic fitness attributes, but this is an assumption and should be explained further to clarify this statement.
Discussion, fourth paragraph – be careful with your statement suggesting they are “key attributes for RL athletes”. You did not show this but state it, so rework or remove this part to focus specifically on what your data show.
Discussion, concluding paragraph – here you highlight how anaerobic and aerobic fitness underpin test performance, but previously you focus on anaerobic fitness in other sections and in this section. Please make sure the message is consistent throughout.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Exercise and sport science

Respond to this report

Responses (1)

Author Response

08 Oct 2021

Kenji Doma, Sport & Exercise Science, James Cook University, Douglas, 4814, Australia

Reviewer’s comment: Thank you for the opportunity to review this manuscript. A logical study with potential practical impact was presented exploring the validity and reliability of a repeated change-of-direction speed test suited to rugby league. This aspect of fitness testing is lacking in the literature and of use for end-users, so this manuscript has merit for publication. I do however have some suggestions and queries listed below that will help strengthen some aspects of the work that should be addressed by the authors:
Author’s response: Dear Reviewer, thank you very much for your comments. We believe your feedback has improved the quality of our work.

Reviewer’s comment: Abstract, Methods – indicate whether you are referring to the multi-stage fitness test (i.e., Beep test) here or if just an assessment of multi-stage fitness was performed.
Author’s response: A Beep test was performed, which is now included in the Abstract.

Reviewer’s comment: Abstract – some justification of using the repeated T-agility test as the other standard repeated change-of-direction performance to assess convergent agility of the repeated Illinois Agility test is needed in the Abstract.
Author’s response: This is a valid point. The Background now includes, “Thus, the current study examined the convergent validity of a repeated Illinois Agility (RIA) protocol with the repeated T-agility protocol, and the repeatability of the RIA protocol in adolescent Rugby League players.”

Reviewer’s comment: Abstract – the focus of exploring the contribution of different fitness attributes to repeated Illinois Agility test performance is not made in the Abstract. Consider including this as a secondary objective and/or making it clear why this analysis is needed.
Author’s response: Thank you for the suggestion. We have now included, “Furthermore, aerobic capacity and anaerobic and COD performance were assessed to determine whether these physical qualities were important contributors to the RIA protocol.”

Reviewer’s comment: Abstract, Conclusions – make it clear that this test is useful for Rugby League coaches specifically, and change “athlete” to plural form or indicate “team” here instead.
Author’s response: The conclusion has been rewritten to, “The current study has demonstrated the RIA to be a simple, valid and reliable field test for RL athletes that can provide coaches with information about their team’s ability to sustain high intensity, multi-directional running efforts.”

Reviewer’s comment: Introduction, opening paragraph – in this paragraph you identify “agility” and hint that it involves physical and cognitive components. So in essence, you are examining change-of-direction speed (the physical component) rather than “agility” per se. Consider making this clear and using the term “change-of-direction speed” or “change-of-direction performance” thereafter when referring to tests and attributes that are purely physical without the cognitive component.
Author’s response: All the terms with ‘agility’ has been replaced with ‘change-of-direction’ performance or protocols throughout the text.

Reviewer’s comment: Introduction, opening paragraph – stronger rationale is needed justifying the inclusion of the Agility T-test and Illinois Agility test in rugby league. Can you add a sentence or two outlining why these tests are suited to the sport and therefore the focus of your study?
Author’s response: We have removed Agility T-test and Illinois Agility test from the first paragraph, and focused the justification of the Illinois Agility test for RL in the second paragraph (see lines 41-47).

Reviewer’s comment: Introduction, end of second paragraph – here indicate whether the validity and reliability of this test has not been investigated at all or just specifically in Rugby League athletes. Also, at the end of this sentence make it clear that you are referring to the usability of this test.
Author’s response: We separated the second paragraph into two paragraphs. Thus, we highlighted that no studies have examined the validity and reliability of the RIA protocol at the end of the fourth paragraph.

Reviewer’s comment: Introduction, aims – for the second aim, you are not comparing the RIA measures to other measures, but instead correlating them, so please change this aim accordingly. Also, consider including a little rationale around this aim in the previous paragraph as it is unclear as to why this is important.
Author’s response: Changed to ‘correlating’ as requested. The previous paragraph expanded on the need to examine the validity of the RIA protocol, which we now hope provides further justification of our aims.

Reviewer’s comment: Introduction, aims – it might pay to include a sentence stating why examining convergent validity (e.g. to show that you are assessing similar attributes with a new test that is more practical and specific to Rugby League when compared to a standard, generic test routinely used) and retest reliability (e.g. to detect the inherent error in the test and ascertain whether it can be reliably adopted in practice to assess repeated measurements in athletes) are needed for practical uptake of the test, which would strengthen the rationale of the first and third aims as well in the Introduction section.
Author’s response: This is a great suggestion. We have now worded the end of the Introduction to, “Examining the convergent validity of the RIA protocol will determine whether this novel assessment exhibits similar attributes to a standardised COD protocol (i.e., RTT). In addition, the reliability of the RIA will determine whether this test can be reliably adopted in practice by accounting for the inherent error of the test across repeated measurements. The quality of these psychometric properties will provide coaches with a tool to assist in monitoring and training RL athletes as well as in talent development and identification.”

Reviewer’s comment: Introduction, hypotheses – you only mention speed and anaerobic capacity as your fitness attributes here, but you also included aerobic fitness (Multistage Fitness test)?
Author’s response: Included aerobic capacity in the hypotheses.

Reviewer’s comment: Methods, Research design – why was the order flipped between session 4 and session 5? It seems like you would want to the athletes to complete the same exact session across both when assessing retest reliability as it is introducing a confounding factor? Also, no hyphen needed between “15” and “minutes” here.
Author’s response: This is a good question. We employed a cross-over design for sessions 4 and 5 to limit the confounder of any learning effects (i.e., were participants still able to exhibit reproducible measures when the protocols were swapped around, as opposed to the participants becoming accustomed to the order effects between sessions). We have also included the hyphen.

Reviewer’s comment: Methods, Participants – can you provide some further indication as to the specific playing level of the athletes? The name of the program is great, but this is not exactly clear for all readers.
Author’s response: Further description of the excellence program has been provided.

Reviewer’s comment: Figure 1 – consider changing the session numbers, as in text you identify the Multistage Fitness test as session 1, but here you indicate the other fitness testing as session 1. Consistency needed.
Author’s response: Session 1 included in the Multistage Fitness test as requested (see updated Figure 1).

Reviewer’s comment: Methods, Participants – add “in” after “partaking” and make parents/guardians singular rather than plural.
Author’s response: Amended as requested.

Reviewer’s comment: Methods, Multistage Shuttle test – stay consistent with capitalising the test names like this one as it is done inconsistently throughout. Also, hyphenate “20m” here.
Author’s response: Amended the test name to upper case as requested and hyphenated to 20-m.

Reviewer’s comment: Methods, Countermovement jump test – hyphenate “1 cm” here. Also, try to indicate what units the key outcomes from each test were reported in (e.g. mL/kg/min, cm, s).
Author’s response: Hyphenated to 1-cm and reported units of measure as requested for each test.

Reviewer’s comment: Methods, 30-m Sprint and Agility protocols – separate statements on linear and change-of-direction speed here. Also, hyphenate “10m” and “5m” for the Illinois Agility test.
Author’s response: Statements have been separated and hyphenated as requested.

Reviewer’s comment: Methods, repeat protocols – to calculate average HR and RPE, was HR measured from when the test started to when the test finished in 1-second intervals? And was RPE taken after each effort or just after all efforts for each specific test? These are not quite clear.
Author’s response: We have included “ “The participant’s instantaneous heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions. – see lines 138-141.

Reviewer’s comment: Methods, Statistical analysis – you mention “construct” validity for the first time here. If this is a key aim and aspect of the study (i.e., correlating performance during the test with fitness attributes), then this needs to be established earlier (i.e., introduction and aims).
Author’s response: We have taken out “construct validity” and mentioned aerobic capacity, power and COD as “contributors to the RIA protocol”.

Reviewer’s comment: Methods, Statistical analysis – an r value of 0.5 seems quite low to establish convergent validity (only 25% shared variance)? In this regard, what was the cut-off for construct validity?
Author’s response: This has been re-established to a minimum r-value of 0.7 for both convergent validity and contributors to the RIA protocol.

Reviewer’s comment: Results, first paragraph – clarify whether this is CMJ height specifically.
Author’s response: Height included as requested.

Reviewer’s comment: Results, second paragraph – make it clear that you are referring to the third and fourth sessions here (and make sure this is consistent as you identify these as the fourth and fifth sessions earlier).
Author’s response: Sessions numbers included as requested, and we also applied this approach in the next paragraph for the RTT protocol to ensure clarity.

Reviewer’s comment: Results, third paragraph – you do not indicate the maximum RPE was taken as an outcome in the methods anywhere (only average), yet it is listed here? Same for maximum HR, which appears later in this section also.
Author’s response: This is a great pick up. In the methods section, we have now included, “The participant’s heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions.”

Reviewer’s comment: Table 1 – shouldn’t RIA and RTT be the two key tests going from left to right rather than RIA and RSA? In fact, it is not clear why RSA is included here given you identify RIA vs. RTT for convergent validity assessment throughout the manuscript. Please adjust or rework earlier sections.
Author’s response: Table 1 has now been amended as requested. RSA was included more to determine the importance of the athlete’s ability to sustain speed and effort over repeated bouts, which has now been included in the second aim.

Reviewer’s comment: Discussion, first paragraph – some minor errors, but change “was” to “were” in the 7th line.
Author’s response: Amended as requested.

Reviewer’s comment: Discussion, first paragraph – at the end of this paragraph, some explanation as to how yours (and the previous studies) support that anaerobic fitness is predominantly stressed in the repeated COD tests. I am assuming the strong correlations with mostly anaerobic fitness attributes, but this is an assumption and should be explained further to clarify this statement.
Author’s response: Closer to the end of this paragraph, we have included a sentence that reads, “The comparable measures between RIA, RTT and RSA suggests that anaerobic fitness, in conjunction with efficient recovery dynamics during short periods of rest in-between explosive activities, are essential qualities for optimal performance in an RIA protocol.”

Reviewer’s comment: Discussion, fourth paragraph – be careful with your statement suggesting they are “key attributes for RL athletes”. You did not show this but state it, so rework or remove this part to focus specifically on what your data show.
Author’s response: Removed as requested.

Reviewer’s comment: Discussion, concluding paragraph – here you highlight how anaerobic and aerobic fitness underpin test performance, but previously you focus on anaerobic fitness in other sections and in this section. Please make sure the message is consistent throughout.
Author’s response: We have now included aerobic capacity as possible important contributors to the RIA protocol in the Abstract, Introduction (aim and hypothesis) and earlier sections in the Discussion.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

29 Views

09 Nov 2020 | for Version 1

James Dugdale, School of Applied Sciences, University of Stirling, Stirling, UK; Edinburgh Napier University, Edinburgh, UK

29 Views Cite this report Responses(1)

Approved With Reservations

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

References

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Fitness testing; youth athletes; talent identification; talent development; physical performance; strength and conditioning

Respond to this report

Responses (1)

Author Response

08 Oct 2021

Kenji Doma, Sport & Exercise Science, James Cook University, Douglas, 4814, Australia

Reviewer’s comment: The present study aimed to explore the validity and reliability of repeated “agility” tests in a sample of youth rugby players. The study provides interesting data for an applied audience and is methodologically sound and accurate in the presentation of findings. This being said, I have identified a range of areas that should be improved prior to potential resubmissions of this manuscript. For example, the introduction (and study in its entirety) is inaccurate in its use of the term “agility” (opposed to COD which the authors actually assess). Furthermore, the introduction needs to be further developed to support the scope of the study. The authors need to create more of a rationale for running such a large number of correlations – exploring correlational analyses needs to be warranted based upon an assumption of a relationship (or lack thereof) between variables. In its current form, I am unsure whether this manuscript provides such rationale. The methods and results are accurate, but with a large number of small errors. Finally, the discussion addresses key discussion points formulated from the findings of the study, but similar to the introduction, lacks some critical depth. I do think that the authors have a publishable study here, however, a number of revisions will be required before I would be happy to approve this manuscript for indexing. I hope that my comments are received as I intend them and are useful in further strengthening this manuscript. I hope to receive the opportunity to review this manuscript again in its revised form.
Author’s response: Dear reviewer, thank you for your comments on the paper. We agree that your comments have improved the quality of our work.

Abstract
Reviewer’s comment: The abstract is concise and accurate to the details of the study. I have just a couple of minor comments (below):
The initial sentence is accurate and outlines the requirement of the study. However, I would encourage the authors to insert a word such as ‘simultaneously’ or ‘congruently’, or attempt to capture the lack of a suitable test to capture the performance of these abilities in combination.
Author’s response: The word ‘simultaneously’ has been included.

Reviewer’s comment: Abstract Methods – the inclusion of the repeated T-agility test appears suddenly with no prior reference. If this is a key concentration of your study, I would encourage to insert earlier in the abstract along with the RIA.
Author’s response: Repeated T-agility protocol now mentioned in the Background.

Reviewer’s comment: Abstract Results – please specify exact p values rather than just <0.05. CV% can be presented to 1dp for brevity.
Author’s response: We were unable to provide exact p-values, as several outcome measures from the repeated COD protocols, such as average time, best time, total time etc were presented as “performances”. This same approach was used for the r-values as well to ensure the word limit did not exceed journal guidelines. However, we did present the CV% to 1dp as requested.

Introduction
Reviewer’s comment: The introduction is short and concise. However, I feel some key information that would further support the rationale for your study has been omitted. I would encourage the authors to further develop this section. A suggestion would be to expand the second paragraph into two: the first discussing the validity aspects, the second the reliability aspects. I do not expect these changes to be too onerous, but believe it would strengthen this section of your manuscript if you were to implement these changes. I have some further specific comments below.
Author’s response: The second paragraph has been further developed into two as requested, with the first focusing on reliability and the second on validity.

Reviewer’s comment: The opening sentence would read better if it were split into two separate sentences.
Author’s response: We have simplified the first sentence by removing the context of ‘agility’, as we have converted all ‘agility’ terms to ‘change-of-direction’.

Reviewer’s comment: The authors should attempt to clarify during opening paragraph that they are discussing change of direction (COD) performance, rather than agility. Although COD performance is a key component of agility performance, the lack of an external stimuli prevents classification of agility. Although the tests identified by the authors in this opening paragraph have titled themselves ‘agility’ tests, by definition they are not. The authors should be careful in their stance here as the literature has evolved substantially since the inception of tests such as these, and appropriate classification and terminology is essential for the present study.
Young, W.; Dawson, B.; Henry, G. Agility and Change-of-Direction Speed are Independent Skills: Implications for Agility in Invasion Sports. Int. J. Sports Sci. Coach. 2015, 10, 159–169.1
Nimphius, S.; Callaghan, S.J.; Bezodis, N.E.; Lockie, R.G. Change of Direction and Agility Tests: Challenging Our Current Measures of Performance. Strength Cond. J. 2018, 40, 26–38. 2
Sheppard, J.M.; Young, W.B. Agility literature review: Classifications, training and testing. J. Sports Sci. 2006, 24, 919–932.3
Author’s response: Thank you for your suggestion. As mentioned earlier, we have changed all ‘agility’ terms into ‘change-of-direction’.

Reviewer’s comment: Regarding the reliability aspect of the study, two recent papers have suggested that reliability of COD and agility may be lower in adolescent and youth athletes (as per your sample). Although these authors observed this during maximal COD/agility performance, opposed to repeated as in the present study, I believe this would be worth including and would further warrant the exploration of your research question and study.
Taylor, J.M.; Cunningham, L.; Hood, P.; Thorne, B.; Irvin, G.; Weston, M. The reliability of a modified 505 test and change-of-direction deficit time in elite youth football players. Sci. Med. Footb. 2019, 3, 157–162.4
Dugdale JH, Sanders D, Hunter AM. Reliability of Change of Direction and Agility Assessments in Youth Soccer Players. Sports (Basel). 2020 Apr 18; 8 (4): 51. doi: 10.3390/sports8040051.5
James H. Dugdale, Calum A. Arthur, Dajo Sanders & Angus M. Hunter (2019) Reliability and validity of field-based fitness tests in youth soccer players, European Journal of Sport Science, 19:6, 745-756, DOI: 10.1080/17461391.2018.15567396
Author’s response: Thank you for your suggestion, we have included the references above in the paragraph explaining studies on the reliability of COD protocols.

Methods
Reviewer’s comment: The methods report the appropriate details relating to the study design, protocols, participants, and analyses performed. However, further clarity is required in a few places throughout this section (specific comments below). Further, there are a number of oversights throughout this section that require careful attention. None of these are major points, and should take little time to amend. Yet, need to be addressed prior to potential resubmissions of this manuscript.
Author’s response: Thank you for your comments, we have addressed these below.

Reviewer’s comment: Figure 1: I am unsure how much this figure aids the interpretation of the study design. I personally do not feel the visual nature of this schematic clarifies the description of the trials from the text, as the study is relatively simplistic. I do not insist that the authors remove this figure, however, please consider my suggestion that it may not add value to the manuscript and may just be a “figure for the sake of a figure”. Secondly, between the trials, the authors have stated that washout periods of > 7 days were implemented. The authors should consider including the range of days (e.g. 7-14 days), as this would provide further information to the reader and allow for more accurate interpretation of the results.
Author’s response: Thank you for your comment. We have decided to keep the figure to ensure clarity as we used a cross-over randomized design, which is unusual for a study examining the reliability and validity of protocols. We have included 7-14 days as a washout period as suggested in the figure.

Reviewer’s comment: Can a citation or elaboration be provided to rationalise the >15 min rest interval between attempts?
Author’s response: A citation has been included with a study that used a 15-minute recovery period in-between repeated sprint and COD protocols in the one testing session.

Reviewer’s comment: The term ‘respectively’ needs to be included after the explanation of the 1-10 muscle rating scale. Also, a brief sentence as to why this was collected should accompany this methodological point.
Author’s response: Included as requested.

Reviewer’s comment: The authors state that the warm up was the same prior to each session. I would encourage using the term ‘standardised’ when describing this in their methods to add certainty around their controls. Also, a citation needs to accompany the statement that the CMJ test was implemented to assess recovery. If this is what was intended by the insertion of citation 12, I would encourage moving this citation to the end of the sentence.
Author’s response: The term ‘standardised’ has been included when describing the warm-up as requested in the Methods section. The citation explaining the use of CMJ to determine recovery dynamics was moved to the end of the sentence as requested.

Reviewer’s comment: Participant details – participant height needs to be reported to 2dp. Can the authors further elaborate on what the “School of Athletic Excellence” is? For example, what standard is this, is it affiliated to a specific school, or region performance programme? This may not be clear to readers.
Author’s response: The participant height is reported to 2dp (1.77m). Further information on the excellence program has been included.

Reviewer’s comment: Multistage shuttle test – a brief sentence stating how performance or a score was derived is necessary (i.e. maximum number of shuttles completed, level, etc.). This information is currently not evident.
Author’s response: The VO2max was estimated based on the level completed, which has now been included in the Methods section.

Reviewer’s comment: CMJ – please state the unit of measurement for this test, and to what degree (e.g. recorded in cm to the nearest 0.1cm).
Author’s response: The CMJ was recorded to the full cm, not to the nearest 0.1cm. This has now been clarified.

Reviewer’s comment: 30-m sprint and agility protocols – please separate each test into a separate sentence. At present, they appear too cramped together and it is difficult to read (particularly with reference to figures and citations within). Also, please provide citations for the original studies for the T-drill and Illinois agility tests. These need to be credited within the manuscript.
Author’s response: The sentences for the description of the protocols have been separated as requested. The original studies have also been included as requested.

Reviewer’s comment: Repeated sprint and agility protocols – were these tests performed one after the other (what I interpret from the term ‘cycles’) or were they performed independently? Clarity is required to ensure the reader understands which of these interpretations is correct.
Author’s response: We have reworded the term ‘cycles’ to ‘repetitions’ to improve clarity. As mentioned in Research Design and in Figure 1, RSA was conducted in Session 2, whilst RTT and RIA were conducted in the same session with 15-min rest in-between each protocol. The Figure reinforces this design, and rewording ‘cycles’ to ‘repetitions’ improves the clarity of the protocols.

Reviewer’s comment: The sentence starting “Immediately after each repeated agility cycle…” does not make sense. Please rephrase this.
Author’s response: This has been reworded to, “The participant’s heart rate (HR, Polar Heart Rate Monitor, Polar H10, Finland) and rating of perceived-exertion (RPE, Borg category scale 1-10) were collected at the completion of each repetition of the RSA, RTT and RIA protocols. The maximum and average HR and RPE values were then reported from the 6 repetitions”

Reviewer’s comment: Statistical analysis – Reference 16 seems a weak reference for this point. I encourage the authors to consider a more suitable citation for selecting this acceptable correlation cut off point.
Author’s response: Additional reference included as requested.

Results
Reviewer’s comment: The section of the results dedicated to the muscle soreness and CMJ differences (or lack thereof) should be condensed to a singular sentence and placed at the foot of the results section. This is a methodological control and not a key finding from your study, therefore, it should not feature so early and heavily within your results section. Stating that there were no differences, providing ES, and significance values would suffice.
Author’s response: Simplified and included at the end of the Results section as requested.

Reviewer’s comment: Table 3 – I am interested in your rationale for not providing ES to demonstrate differences (or lack thereof) for your test-retest data? This would add value in my opinion. I would also encourage the authors to present CV% data to 1dp both within this table and throughout the manuscript.
Author’s response: Thank you for your comment. The ES was not reported as we based the measurement error on CV, with larger CV% values exhibiting greater measurement error and poorer test-retest reliability. In our opinion, inclusion of ES would not provide any further clarity about the reliability of the tests, beyond that of CV.

Reviewer’s comment: I notice that the authors use terminology such as “excellent” or “moderate” test-retest reliability, yet they have not included these threshold within the statistical analysis section of the methods. Please include this detail so that the reader understands your criteria.
Author’s response: Included as requested.

Reviewer’s comment: I also note that the authors say “most RIA performance measures exhibited excellent test-retest reliability…”. Could the authors rephrase this to say “all RIA performance measures except… measures exhibited excellent test-retest reliability”? This would be more accurate and less ambiguous for the reader. This comment also applies later in this section where the authors refer to “a few variables…”.
Author’s response: Thank you for your suggestion. We have reworded this to, “Most RIA measures exhibited excellent test-retest reliability (ICC = 0.92-0.97), good levels of agreement (ratio LOA = 1.05-1.06) and low measurement error (CV = 2.17 - 2.68%) (Table 3). However, FI and average RPE demonstrated moderate test-retest reliability (ICC = 0.87 and 0.76, respectively), poorer levels of agreement (ratio LOA = 2.57 and 2.23, respectively) and higher measurement error (CV = 25.3 and 15.8%, respectively, Table 3).” The next paragraph has been rewritten to, “For the RTT, excellent test-retest reliability (ICC = 0.91), good levels of agreement (ratio LOA = 1.08) and low measurement error (CV = 2.17-2.68%) were identified for TT, BT and AT (Table 3), although maximum RPE demonstrated higher levels of measurement error (CV = 12.3%) between Sessions 4 and 5 (Table 3).”

Discussion
Reviewer’s comment: I found the discussion well-written and thought the authors accurately interpreted and attempted to explain their findings. However, similar to the introduction, I felt this section lacked depth when discussing the main findings of the study. I encourage the authors to discuss their findings in greater depth, utilising some of the reading attached to this review.
Author’s response: Thank you for your comments. The Discussion has been adjusted to address the comments below.

Reviewer’s comment: I encourage the authors to utilise the first paragraph of the discussion to summarise all of their main findings, and then progress to discussing each point in turn.
Author’s response: Summary of findings included as requested in the first paragraph.

Reviewer’s comment: The third paragraph begins “another novelty of this study”, yet this appears to be the first time novelty is addressed in the discussion?
Author’s response: This specific text has been deleted to improve clarity.

Reviewer’s comment: The fourth paragraph can draw upon a wealth of correlational studies to further support the findings observed here.
Young WB, Dawson B, Henry GJ. Agility and Change-of-Direction Speed are Independent Skills: Implications for Training for Agility in Invasion Sports. International Journal of Sports Science & Coaching. 2015;10(1):159-169. doi:10.1260/1747-9541.10.1.1591
Spiteri T, Nimphius S, Hart NH, Specos C, Sheppard JM, Newton RU. Contribution of strength characteristics to change of direction and agility performance in female basketball athletes. J Strength Cond Res. 2014 Sep;28(9):2415-23. doi: 10.1519/JSC.0000000000000547. PMID: 24875426.7
Gabbett, T. J., Kelly, J. N., & Sheppard, J. M. (2008). Speed, Change of Direction Speed, and Reactive Agility of Rugby League Players. The Journal of Strength & Conditioning Research, 22, 174-181. http://dx.doi.org/10.1519/JSC.0b013e31815ef7008
Sheppard, J.M.; Young, W.B.; Doyle, T.L.A.; Sheppard, T.A.; Newton, R.U. An evaluation of a new test of reactive agility and its relationship to sprint speed and change of direction speed. J. Sci Med. Sport 2006, 9, 342–349.9
Author’s response: A sentence with relevant references listed above were included as requested.

Reviewer’s comment: The conclusion should be reframed around the key objectives of the study (i.e. that it is reliable and valid). The mention of aerobic and anaerobic energy systems seems misplaced here.
Author’s response: A sentence on the validity of the RIA protocols has been included.

References
Reviewer’s comment: While a variety of key texts are identified and listed within the bibliography and cited throughout the manuscript, a number of key texts are omitted. I encourage the authors to further familiarise themselves with relevant studies assessing reliability and validity of COD and agility in team sport athletes, as well as the texts identified earlier within this review.
Dos’Santos, T.; Thomas, C.; Jones, P.A.; Comfort, P. Mechanical Determinants of Faster Change of Direction Speed Performance in Male Athletes. J. Strength Cond. Res. 2017, 31, 696–705.10
Gabbett, T.J.; Kelly, J.N.; Sheppard, J.M. Speed, change of direction speed, and reactive agility of rugby league players. J. Strength Cond. Res. 2008, 22, 174–181.8
Oliver, J.L.; Meyers, R.W. Reliability and generality of measures of acceleration, planned agility, and reactive agility. Int. J. Sports Physiol. Perform. 2009, 4, 345–354.11
Author’s response: Thank you for kindly providing the references. A number of these have been included in the paper as requested.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Austin DJ, Gabbett T, Jenkins DJ: Repeated high-intensity exercise in a professional rugby league. J Strength Cond Res. 2011; 25(7): 1898–1904. PubMed Abstract | Publisher Full Text

[2] 2. Dos'Santos T, Thomas C, Jones PA, et al.: Mechanical Determinants of Faster Change of Direction Speed Performance in Male Athletes. J Strength Cond Res. 2017; 31(3): 696–705. PubMed Abstract | Publisher Full Text

[3] 3. Gabbett TJ, Kelly JN, Sheppard JM: Speed, change of direction speed, and reactive agility of rugby league players. J Strength Cond Res. 2008; 22(1): 174–181. PubMed Abstract | Publisher Full Text

[4] 4. Raya MA, Gailey RS, Gaunaurd IA, et al.: Comparison of three agility tests with male servicemembers: Edgren Side Step Test, T-Test, and Illinois Agility Test. J Rehabil Res Dev. 2013; 50(7): 951–960. PubMed Abstract | Publisher Full Text

[5] 5. Nimphius S, Callaghan SJ, Bezodis NE, et al.: Change of direction and agility tests: Challenging our current measures of performance. Strength Cond J. 2018; 40(1): 26–38. Publisher Full Text

[6] 6. King T, Jenkins D, Gabbett T: A time-motion analysis of professional rugby league match-play. J Sports Sci. 2009; 27(3): 213–219. PubMed Abstract | Publisher Full Text

[7] 7. Wong DP, Chan GS, Smith AW: Repeated-sprint and change-of-direction abilities in physically active individuals and soccer players: Training and testing implications. J Strength Cond Res. 2012; 26(9): 2324–2330. PubMed Abstract | Publisher Full Text

[8] 8. Haj-Sassi R, Dardouri W, Gharbi Z, et al.: Reliability and validity of a new repeated agility test as a measure of anaerobic and explosive power. J Strength Cond Res. 2011; 25(2): 472–480. PubMed Abstract | Publisher Full Text

[9] 9. Fessi MS, Makni E, Elloumi M, et al.: Reliability and criterion-related validity of a new repeated agility test. Biol Sport. 2016; 33(2): 159–164. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. Wheeler K: Agility skill execution in rugby union. In: Faculty of Science, Health and Education. University of Sunshine Coast. 2010. Reference Source

[11] 11. Durandt J, du Toit S, Borresen J, et al.: Fitness and body composition profiling of elite junior South African rugby players. South Afr J Sports Med. 2006; 18(2): 38–45. Publisher Full Text

[12] 12. Dugdale JH, Arthur CA, Sanders D, et al.: Reliability and validity of field-based fitness tests in youth soccer players. Eur J Sport Sci. 2019; 19(6): 745–756. PubMed Abstract | Publisher Full Text

[13] 13. Dugdale JH, Sanders D, Hunter AM: Reliability of Change of Direction and Agility Assessments in Youth Soccer Players. Sports (Basel). 2020; 8(4): 51. PubMed Abstract | Publisher Full Text | Free Full Text

[14] 14. Ramsbottom R, Brewer J, Williams C: A progressive shuttle run test to estimate maximal oxygen uptake. Br J Sports Med. 1988; 22(4): 141–144. PubMed Abstract | Publisher Full Text | Free Full Text

[15] 15. Doma K, Nicholls A, Gahreman D, et al.: The effect of a resistance training session on physiological and thermoregulatory measures of sub-maximal running performance in the heat in heat-acclimatized men. Sports Med Open. 2019; 5(1): 21. PubMed Abstract | Publisher Full Text | Free Full Text

[16] 16. Doma K, Schumann M, Leicht AS, et al.: The repeated bout effect of traditional resistance exercises on running performance across 3 bouts. Appl Physiol Nutr Metab. 2017; 42(9): 978–985. PubMed Abstract | Publisher Full Text

[17] 17. Ramírez-Campillo R, Meylan C, Alvarez C, et al.: Effects of in-season low-volume high-intensity plyometric training on explosive actions and endurance of young soccer players. J Strength Cond Res. 2014; 28(5): 1335–42. PubMed Abstract | Publisher Full Text

[18] 18. Doma K, Schumann M, Sinclair WH, et al.: The repeated bout effect of typical lower body strength training sessions on sub-maximal running performance and hormonal response. Eur J Appl Physiol. 2015; 115(8): 1789–99. PubMed Abstract | Publisher Full Text

[19] 19. Semenick D: Tests and measurements: The t-test. Strength & Conditioning Journal. 1990; 12(1): 36–37. Reference Source

[20] 20. Cureton TK: Illinois agility run. Champaign, Ill: Stipes Publishing Company. 1970. Reference Source

[21] 21. Zamunér AR, Moreno MA, Camargo TM, et al.: Assessment of subjective perceived exertion at the anaerobic threshold with the borg CR-10 scale. J Sports Sci Med. 2011; 10(1): 130–6. PubMed Abstract | Free Full Text

[22] 22. Abma IL, Rovers M, van der Wees PJ: Appraising convergent validity of patient-reported outcome measures in systematic reviews: constructing hypotheses and interpreting outcomes. BMC Res Notes. 2016; 9: 226. PubMed Abstract | Publisher Full Text | Free Full Text

[23] 23. Cordier R, Chen YW, Speyer R, et al.: Child-report measures of occupational performance: a systematic review. PLoS One. 2016; 11(1): e0147751. PubMed Abstract | Publisher Full Text | Free Full Text

[24] 24. Leicht AS, Sealey RM, Sinclair WH: The reliability of VO_2(peak) determination in healthy females during an incremental arm ergometry test. Int J Sports Med. 2009; 30(7): 509–15. PubMed Abstract | Publisher Full Text

[25] 25. Nevill AM, Atkinson G: Assessing agreement between measurements recorded on a ratio scale in sports medicine and sports science. Br J Sports Med. 1997; 31(4): 314–8. PubMed Abstract | Publisher Full Text | Free Full Text

[26] 26. Doma K: Convergent validity and reliability of a novel repeated agility protocol in junior rugby league players. James Cook University. (dataset). 2020; 89b303814f7501cfee8dd987fb606ad5.

[27] 27. Gabbett TJ: Physiological and anthropometric characteristics of junior rugby league players over a competitive season. J Strength Cond Res. 2005; 19(4): 764–771. PubMed Abstract

[28] 28. Zagatto AM, Ardigò LP, Barbieri FA, et al.: Performance and metabolic demand of a new repeated-sprint ability test in basketball players: does the number of changes of direction matter? J Strength Cond Res. 2017; 31(9): 2438–2446. PubMed Abstract | Publisher Full Text

[29] 29. Daneshfar A, Gahreman DE, Koozehchian MS, et al.: Multi directional repeated sprint is a valid and reliable test for assessment of junior handball players. Front Physiol. 2018; 9(1): 317. PubMed Abstract | Publisher Full Text | Free Full Text

[30] 30. Doma K, Deakin GB, Leicht AS, et al.: The reliability of running economy among trained distance runners and field-based players. J Exerc Sci Phys Fit. 2012; 10(2): 90–96. Publisher Full Text

[31] 31. Lamb KL, Eston RG, Corns D: Reliability of ratings of perceived exertion during progressive treadmill exercise. Br J Sports Med. 1999; 33(5): 336–9. PubMed Abstract | Publisher Full Text | Free Full Text

[32] 32. Scott TJ, Black CR, Quinn J, et al.: Validity and reliability of the session-RPE method for quantifying training in Australian football: a comparison of the CR10 and CR100 scales. J Strength Cond Res. 2013; 27(1): 270–6. PubMed Abstract | Publisher Full Text

[33] 33. Vera J, Perales JC, Jiménez R, et al.: A test-retest assessment of the effects of mental load on ratings of affect, arousal and perceived exertion during submaximal cycling. J Sports Sci. 2018; 36(22): 2521–2530. PubMed Abstract | Publisher Full Text

[34] 34. Billaut F, Bishop DJ, Schaerz S, et al.: Influence of knowledge of sprint number on pacing during repeated-sprint exercise. Med Sci Sport Exerc. 2011; 43(4): 665–672. PubMed Abstract | Publisher Full Text

[35] 35. Sheppard JM, Young WB, Doyle TL, et al.: An evaluation of a new test of reactive agility and its relationship to sprint speed and change of direction speed. J Sci Med Sport. 2006; 9(4): 342–349. PubMed Abstract | Publisher Full Text

[36] 36. Spiteri T, Nimphius S, Hart NH, et al.: Contribution of strength characteristics to change of direction and agility performance in female basketball athletes. J Strength Cond Res. 2014; 28(9): 2415–2423. PubMed Abstract | Publisher Full Text

[37] 37. Bishop D, Edge J: Determinants of repeated-sprint ability in females matched for single-sprint performance. Eur J Appl Physiol. 2006; 97(4): 373–9. PubMed Abstract | Publisher Full Text

[38] 38. Gharbi Z, Dardouri W, Haj-Sassi R, et al.: Aerobic and anaerobic determinants of repeated sprint ability in team sports athletes. Biol Sport. 2015; 32(3): 207–12. PubMed Abstract | Publisher Full Text | Free Full Text

[39] 39. Glaister M: Multiple sprint work: physiological responses, mechanisms of fatigue and the influence of aerobic fitness. Sports Med. 2005; 35(9): 757–77. PubMed Abstract | Publisher Full Text

[40] 40. Buchheit M, Ufland P: Effect of endurance training on performance and muscle reoxygenation rate during repeated-sprint running. Eur J Appl Physiol. 2011; 111(2): 293–301. PubMed Abstract | Publisher Full Text

[41] 41. Sahlin K, Henriksson J: Buffer capacity and lactate accumulation in skeletal muscle of trained and untrained men. Acta Physiol Scand. 1984; 122(3): 331–9. PubMed Abstract | Publisher Full Text

[42] 42. Bogdanis GC, Nevill ME, Boobis LH, et al.: Contribution of phosphocreatine and aerobic metabolism to energy supply during repeated sprint exercise. J Appl Physiol (1985). 1996; 80(3): 876–84. PubMed Abstract | Publisher Full Text

Convergent validity and reliability of a novel repeated agility protocol in junior rugby league players

Abstract

Keywords

Revised Amendments from Version 2

Introduction

Methods

Research design

Figure 1. Schematic of the research design consisting of four sessions, including best-effort agility, speed, repeat sprint ability (RSA), repeated T-test agility (RTT) and repeated Illinois agility (RIA) measures.

Participants

Multistage shuttle test

Countermovement jump test

30-m Sprint and Change-of-direction protocols

Figure 2.

Repeat Sprint and Change-of-direction Protocols

Statistical analysis

Results

Table 1. Relationship between performance measures of the repeated Illinois agility test (RIA), repeated T-agility test (RTT) and repeated sprint ability (RSA).

Table 2. Pearson correlation coefficients between repeated performances, perceptual and physiological indices (repeated Illinois agility [RIA], repeated T-agility test [RTT]) with aerobic capacity, leg power, speed, and agility test performance measures.

Discussion

Data availability

Underlying data

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated