Sample size variation in single-time post-dose assessment <i>vs</i> multi-time post-dose assessment

Sarfaraz Sayyed; Ashwini Mathur; Asha Kamath

doi:10.12688/f1000research.124917.1

Home Browse Sample size variation in single-time post-dose assessment vs multi-time...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Method Article

Sample size variation in single-time post-dose assessment vs multi-time post-dose assessment

[version 1; peer review: 2 not approved]

Sarfaraz Sayyed¹, Ashwini Mathur², Asha Kamath³

PUBLISHED 21 Dec 2022

Author details Author details

¹ Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, Telangana, 500080, India
² Clinical Technology & Innovation, Novartis Ireland Ltd., Dublin, Ireland
³ Department of Data Science, Manipal academy of higher studies, Manipal, karnataka, 576104, India

Sarfaraz Sayyed
Roles: Formal Analysis, Investigation, Visualization, Writing – Original Draft Preparation

Ashwini Mathur
Roles: Conceptualization, Supervision, Writing – Review & Editing

Asha Kamath
Roles: Conceptualization, Supervision, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Manipal Academy of Higher Education gateway.

Abstract

Background: Many randomized trials measure a continuous outcome simultaneously at baseline and after taking the drug. For a single continuous post-treatment outcome, the sample size calculation is simple, but if there are assessments at multiple time points post-treatment then this longitudinal data may give more insights by analyzing the data using the repeated measures method. Also, if the sample size is calculated using the single time-point method for longitudinal data, it may lead to a larger than required sample size, increasing the cost and time.
Methods: In this research, an effort is made to determine the size of the sample for repeated measures case and then compared with the single post-baseline case. The sample sizes were examined under different scenarios for the continuous type of response variable. Under Mean contrast and Diff contrast the sample sizes were calculated with different correlations. These two scenarios were again examined under compound symmetry as well as Auto regressive of order 1 type of correlation structure in longitudinal data. The graphical presentation is given for better visualization of the scenarios.
Results: Sample size required for highly correlated longitudinal data using multi timepoint sample size derivation method led to much smaller sample size requirement as compared to single timepoint sample size calculation method.
Conclusions: This study will help researchers to make better decisions in choosing the right method for sample size determination which may reduce the time and cost of carrying out the experiment. Also, we must carefully assess which method to go with when the correlation is weak. More complex correlation structures are not studied in this article but can be studied in the same fashion.

Keywords

sample size estimation; longitudinal study; repeated measure analysis; univariate analysis

Corresponding authors: Ashwini Mathur, Asha Kamath

Competing interests: The author (Sarfaraz Sayyed) and the co-author (Ashwini Mathur) are employed by Novartis Healthcare Pvt. Ltd.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2022 Sayyed S et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Sayyed S, Mathur A and Kamath A. Sample size variation in single-time post-dose assessment vs multi-time post-dose assessment [version 1; peer review: 2 not approved]. F1000Research 2022, 11:1550 (https://doi.org/10.12688/f1000research.124917.1) First published: 21 Dec 2022, 11:1550 (https://doi.org/10.12688/f1000research.124917.1) Latest published: 05 Sep 2024, 11:1550 (https://doi.org/10.12688/f1000research.124917.2)

Introduction

‘Sample size’ is a term used in any experiment for defining the number of subjects selected or observed in the experiment. Sample used in an experiment is a subset of the entire target population. The target population refers to a group of individuals among whom the research is intended to be conducted in order to draw conclusions about the population. For e.g., for a type II diabetic test drug all humans who are suffering from type II diabetes would form the target population.

The number of participants in the sample effects the precision of our estimate and the power of the study to draw conclusions. The probability with which one would reject null hypothesis when null hypothesis is not true is said to be the power of the statistical test. So, power gives an idea on the chances of not committing an error of Type II. The size of the sample and the effect size are major factors which affects the power of a study. These are key concepts in designing a clinical trial.¹ Sample size calculation is an important activity in setting up clinical trials to ensure achievement of trial objectives.

As an illustration, consider a study to compare the performance of a professional athlete taking a particular protein shake versus athletes who do not consume any special protein shakes. Narrowing down attention to a portion of the wider group is essential to enable tracking of the eating habits of every elite athlete in the world. This would entail choosing 100 professional athletes for our study at random; in this case, 100 would be the sample size. Based on the data gathered from a sample of 100 elite athletes, the study’s findings potentially characterize the population of all athletes in the sports industry. Lack of full coverage of the target population would result in the outcomes of the study having a margin of error. Given the limitation to examine every situation, the absolute precision of the effect of protein shake on the athlete’s performance would be hard to measure. Sampling error² is the term used to describe this level of uncertainty or inaccuracy. It affects the estimator’s precision, which is a metric that is important for the chosen target population of all professional athletes.

Although sampling error cannot be completely eliminated, it can be reduced. A larger sample typically has a narrower margin of error. We require an appropriate sample size to examine and contrast in order to provide an accurate picture of the effects of protein shake consumption on performance. Note that there comes a point where increasing the sample size has no further effect on the sampling error (law of diminishing returns).³

Background

Longitudinal studies take a longer time but help in determining causality as well as monitoring the trend over time. To see how sample size calculation was addressed in published longitudinal studies we searched the databases such as Scopus, Web of Science, PubMed, ScienceDirect, and Google Scholar using a range of key terms: “designing clinical trials”, “sample size calculation”, “longitudinal studies”, “randomized trials” and “repeated measures”. The ensuing literature review did not reveal much information on details of how the sample size was calculated for these published longitudinal studies.⁴^,⁵

Formulae for deriving sample size in longitudinal studies is available from several papers.⁶^,⁷ Basagaña, Liao and Spiegelma⁸ published a study in which the power as well as the sample size are discussed for time-varying exposures, but how this is practically applied to a longitudinal study design and its outcome is undocumented in published papers. Pourhoseingholi et al.,⁹ and Karimollah¹⁰ both published about the importance of various components for calculating the sample size in medical studies or clinical trials where often there would be more than one post baseline assessment, but sample size calculation is shown assuming single post baseline assessment. Manja and Lakshminrusimha published a two-part study¹¹^,¹² which does give a good explanation on clinical research design, but sample size is not discussed in detail.

Most of the published studies which have assessments at multiple time points calculate the sample size based on the change from study end time point to baseline whereas a smaller number of papers emphasize on the use of multiple time points into consideration for calculating the sample size.⁶^–⁸

The need for this research was prompted by this lack of proper usage of sample size calculation for longitudinal studies and to further explore which method for sample size calculation should be used in a longitudinal study resulting in correlated outcome data.

Objective

To explore the variation in sample size by considering multiple time point assessment versus the change from baseline to a single endpoint.

Notation and framework

In an experiment for testing certain hypothesis with parallel group design, two or more independent groups are considered to be treated under different scenarios in order to compare the outcome of the scenarios. In our study we would consider the objective of comparison of two drugs.

Let $Y_{ij}$ (X_ij) be the outcome of interest at j^th (j = 1, 2, 3, …, t) time point for the i^th (i = 1, 2, 3, …, n) patient in the two groups.

For the parallel group design, these 2n patients will be divided into two groups with 1: 1 ratio where one arm is assigned to receive the test drug and the other arm is assigned to receive the comparator drug.

Let $μ_{1} & μ_{2}$ be the population mean change from baseline for the test drug and comparator drug respectively.

Let $\bar{Y} & \bar{X}$ be the sample mean change from baseline outcome for the test drug and comparator drug respectively, where $\bar{Y} \sim N (μ_{1}, \frac{σ^{2}}{n})$ and $\bar{X} \sim N (μ_{2}, \frac{σ^{2}}{n})$ .

Methods

Method for Sample size with single time point assessment analysis

Change at single post-baseline assessment from baseline (Single time assessment analysis)

In a parallel group design study with two arms of equal size let the hypothesis be set as:

$H_{0} : μ_{1} - μ_{2} = δ = 0$ , There is no difference between the effects of test and comparator drug.

$H_{a} : μ_{1} - μ_{2} = δ > 0$ , Test drug is having larger effect as compared to comparator.

The test statistic assuming common standard deviation $σ$ for both arms will be given by

(1)

T = \frac{\bar{Y} - \bar{X}}{\sqrt{Var (\bar{Y} - \bar{X})}} = \frac{\bar{Y} - \bar{X}}{\sqrt{\frac{σ^{2}}{n} + \frac{σ^{2}}{n}}} = \frac{\bar{Y} - \bar{X}}{\sqrt{\frac{2 σ^{2}}{n}}}

Now If $H_{0}$ is true (and $μ_{1} = μ_{2}),$ then $T \sim N (0, 1)$ , else if $H_{a}$ is true (i.e., $δ > 0)$ , then $T$ will still be a normal but with a mean greater than zero.

If Type II error is denoted by $β$ then power will be simply $1 - β$ and power is the probability to reject $H_{0}$ when $H_{a}$ is true. In probability equation it could be written as

(2)

Pr [Reject H_{0}| H_{a} is true] = Pr [T > z_{1 - α}| δ > 0] = 1 - β,

where

z_{1 - α}

is the threshold or the critical value which is

(1 - α)

quantile from Gaussian distribution and

α

is the type I error or the level of significance.

(3)

T \sim N (\frac{δ}{\sqrt{\frac{2 σ^{2}}{n}}}, 1), Under H_{a} \to T - \frac{δ}{\sqrt{\frac{2 σ^{2}}{n}}} = Z \sim N (0, 1)

(4)

\begin{matrix} Power function = Pr [T > z_{1 - α}| δ > 0] = Pr \{\frac{\bar{Y} - \bar{X} - δ}{\sqrt{\frac{2 σ^{2}}{n}}} > z_{1 - α} - \frac{δ}{\sqrt{\frac{2 σ^{2}}{n}}}| δ > 0\} \\ = Pr \{Z > z_{1 - α} - \frac{δ}{\sqrt{\frac{2 σ^{2}}{n}}}| δ > 0\} \\ = 1 - Φ (z_{1 - α} - \sqrt{2 n} (\frac{δ}{2 σ})), where Φ (z) = Pr (Z \leq z) \end{matrix}

Now in any study we would be looking for

(5)

\begin{matrix} 1 - Φ (z_{1 - α} - \sqrt{\frac{n}{2}} (\frac{δ}{σ})) \geq 1 - β \\ \to β \geq Φ (z_{1 - α} - \sqrt{\frac{n}{2}} (\frac{δ}{σ})) or z_{β} \geq z_{1 - α} - \sqrt{\frac{n}{2}} (\frac{δ}{σ}) \end{matrix}

After solving equation 5, we get

(6)

n \geq \frac{2 {(z_{1 - α} + z_{1 - β})}^{2} σ^{2}}{{(μ_{1} - μ_{2})}^{2}} = \frac{2 {(z_{α} + z_{β})}^{2}}{{[(μ_{1} - μ_{2}) / σ]}^{2}}

Here n is the sample size required per arm. We will use these formulae in calculating the sample size for single post baseline time point analysis.

Method for Sample size with multiple time point assessment analysis

Change from baseline with post baseline assessment at multiple timepoints (Multiple time points analysis)

In a parallel group design study with two arms of equal size and assessments taken at multiple time points let the hypothesis be set as:

$H_{a} : ψ_{c} = 0$ , there is no difference between the effects of test and comparator drug.

$H_{a} : ψ_{c} > 0$ , test drug is having larger effect as compared to comparator.

Where $ψ_{c} = CΛ$ ,

Let $C = [\begin{matrix} c_{1} \\ c_{2} \\ ⋮ \\ c_{i} \\ ⋮ \\ c_{t} \end{matrix}]$ be the contrast to be tested for the hypothesis and let $Λ = [\begin{matrix} μ_{11} - μ_{21} \\ μ_{12} - μ_{22} \\ ⋮ \\ μ_{1 i} - μ_{2 i} \\ ⋮ \\ μ_{1 t} - μ_{2 t} \end{matrix}]$

$μ_{1 i} & μ_{2 i}$ are the mean effect in arm one and arm two at time point “i” respectively in a study with t time points. $c_{i}$ can take any value depending on the hypothesis we want to test.

For e.g., if we want to see the difference between two drugs for change from baseline when t=2, then $c_{1} = - 1 and c_{2} = 1$ and the resulting $ψ_{c}$ will be

(7)

\begin{matrix} ψ_{c} = CΛ = [- 1 1] . [\begin{matrix} μ_{11} - μ_{21} \\ μ_{12} - μ_{22} \end{matrix}] \\ = - (μ_{11} - μ_{21}) + μ_{12} - μ_{22} \\ = (μ_{12} - μ_{11}) - (μ_{22} - μ_{21}) \\ = change in Drug 1 - change in Drug 2 . \end{matrix}

Common Variance covariance matrix = [\begin{matrix} σ_{1}^{2} & σ_{12} & \dots & σ_{1 t} \\ σ_{12} & σ_{2}^{2} & ⋱ & σ_{2 t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ σ_{i 1} & σ_{i 2} & ⋱ & σ_{i t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ σ_{1 t} & σ_{2 t} & \dots & σ_{t}^{2} \end{matrix}]

Where $σ_{i}^{2}$ is the variance at time point i and $σ_{ij}$ represents the covariance between time point i and j.

The test statistic assuming similar variance-covariance matrix for both arms will be given by

(8)

T = \frac{ψ_{c}}{\sqrt{Var (ψ_{c})}} = \frac{ψ_{c}}{\sqrt{Var (CΛ)}} = \frac{ψ_{c}}{\sqrt{C^{'} . Var (Λ) . C}}

Consider the $Var (Λ)$ ,

Var (Λ) = Var [\begin{matrix} μ_{11} - μ_{21} \\ μ_{12} - μ_{22} \\ ⋮ \\ μ_{1 i} - μ_{2 i} \\ ⋮ \\ μ_{1 t} - μ_{2 t} \end{matrix}] = Var [\begin{matrix} δ_{1} \\ δ_{2} \\ ⋮ \\ δ_{i} \\ ⋮ \\ δ_{t} \end{matrix}] = [\begin{matrix} \frac{2}{n} . σ_{1}^{2} & \frac{4}{n} . σ_{12} & \dots & \frac{4}{n} . σ_{1 t} \\ \frac{4}{n} . σ_{12} & \frac{2}{n} . σ_{2}^{2} & ⋱ & \frac{4}{n} . σ_{2 t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ \frac{4}{n} . σ_{i 1} & \frac{4}{n} . σ_{i 2} & ⋱ & \frac{4}{n} . σ_{i t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ \frac{4}{n} . σ_{1 t} & \frac{4}{n} . σ_{2 t} & \dots & \frac{2}{n} . σ_{t}^{2} \end{matrix}]

(9)

Var (Λ) = \frac{2}{n} . [\begin{matrix} σ_{1}^{2} & 2 σ_{12} & \dots & 2 σ_{1 t} \\ σ_{12} & σ_{2}^{2} & ⋱ & σ_{2 t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ σ_{i 1} & 2 σ_{i 2} & ⋱ & 2 σ_{i t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 2 σ_{1 t} & 2 σ_{2 t} & \dots & σ_{t}^{2} \end{matrix}]

Solving for $C^{'} . Var (Λ) . C$ , we get

(10)

\begin{matrix} C^{'} . Var (Λ) . C = [\begin{matrix} c_{1} c_{2} \dots c_{i} \dots c_{t} \end{matrix}] . \frac{2}{n} . [\begin{matrix} σ_{1}^{2} & 2 σ_{12} & \dots & 2 σ_{1 t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ σ_{i 1} & 2 σ_{i 2} & ⋱ & 2 σ_{it} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 2 σ_{1 t} & 2 σ_{2 t} & \dots & σ_{t}^{2} \end{matrix}] [\begin{matrix} c_{1} \\ c_{2} \\ ⋮ \\ c_{i} \\ ⋮ \\ c_{t} \end{matrix}] \\ = \frac{2}{n} . [\sum_{i = 1}^{n} c_{i}^{2} σ_{i}^{2} + 2 \sum_{i < j}^{n} c_{i} c_{j} σ_{ij}] = \frac{2}{n} . σ_{c}^{2}, \\ where σ_{c}^{2} = [\sum_{i = 1}^{n} c_{i}^{2} σ_{i}^{2} + 2 \sum_{i < j}^{n} c_{i} c_{j} σ_{ij}] \end{matrix}

Solving equations Equation 8 and Equation 9 for T we get

(11)

\begin{matrix} T = \frac{ψ_{c}}{\sqrt{C^{'} . Var (Λ) . C}} = \frac{ψ_{c}}{\sqrt{\frac{2}{n} . [\sum_{i = 1}^{n} c_{i}^{2} σ_{i}^{2} + 2 \sum_{i < j}^{n} c_{i} c_{j} σ_{ij}]}} \\ = \sqrt{\frac{n}{2}} (\frac{ψ_{c}}{\sqrt{σ_{c}^{2}}}) \end{matrix}

Now, If we follow similar steps as we did in single time point analysis above, we get

(12)

β \geq Φ (z_{1 - α} - \sqrt{\frac{n}{2}} (\frac{ψ_{c}}{\sqrt{σ_{c}^{2}}})) or z_{β} \geq z_{1 - α} - \sqrt{\frac{n}{2}} (\frac{ψ_{c}}{\sqrt{σ_{c}^{2}}})

And solving Equation 12, we get¹³

(13)

\begin{matrix} n \geq \frac{2 {(z_{α} + z_{β})}^{2} σ_{c}^{2}}{ψ_{c}^{2}} \\ with \\ ψ_{c} = \sum_{i = 1}^{t} c_{i} (μ_{1 i} - μ_{2 i}) and σ_{c}^{2} = \sum_{i = 1}^{t} c_{i}^{2} σ_{i}^{2} + 2 \sum_{i < j}^{t} c_{i} c_{j} σ_{ij} \end{matrix}

$σ_{i}^{2} = common variance in the two groups at timepoint i .$

$σ_{i j} = common covariance in the two groups between timepoint i and j .$

$c_{i} = contrast applied at timepoint i and t represents the number of time points .$

We will use the formulae specified in Equation 13 to calculate the sample size for multiple time point analysis.

Calculation of sample size

Appropriate sample size was calculated for multi-time and single time cases with different scenarios to achieve an overall mean treatment difference (0.9 points) between two treatment groups with common standard deviation (SD) (3.6 points) allowing 5% two-sided type I error and 85% power. The effect size and standard deviation used here are based on a real study.¹⁴ This was a three-year study with primary endpoint assessment at end of each year but the sample size calculation in this study was done on the basis of single time point. Since this study failed to recruit the expected number of patients and also had lot of missing data, the characteristics for the primary endpoints from the second year were used as it had equal number of patients in both arms and stabilized assessments.

Sample size (single time point case)

We considered a two-arm parallel group scenario with one baseline and one post baseline timepoints to assess on change from baseline in absolute scale. Using the formulae in Equation 6 above for single timepoint analysis the sample size required per arm was 287 cases to show statistical significance.

Sample size (longitudinal case)

Here again we considered two arm parallel groups with multiple timepoints and for studying we investigated six cases i.e., three, four, five, six, eight, and 10 timepoints. Each of this case corresponds to number of assessments including baseline. Three timepoints corresponded to the case with one baseline and two post baseline assessments, four timepoints corresponded to the case with one baseline and three post baseline assessments, five timepoints corresponded to the case with one baseline and four post baseline assessments and so on.

Figure 1 and Figure 2 represents each of this case as a line in the plot under different contrast types and correlation structures.

Figure 1. Simulation results with compound symmetry correlation structure.

Figure 2. Simulation results with auto regressive order 1 correlation structure.

Keeping the SD as 3.6 we tried to vary over two different correlation structures:

Compound symmetry (CS)

Compound Symmetry just means that all the variances are equal and all the covariances are equal. So, the same variance and covariance are used for all subjects. In compound symmetry the covariances across the subjects and the variances (pooled within the group) of the different repeated measures are homogeneous.

[\begin{matrix} σ^{2} & σ^{2} ρ & σ^{2} ρ & \dots & σ^{2} ρ \\ σ^{2} ρ & σ^{2} & σ^{2} ρ & \dots & σ^{2} ρ \\ σ^{2} ρ & σ^{2} ρ & σ^{2} & \dots & σ^{2} ρ \\ ⋮ & ⋱ & ⋮ & ⋮ & ⋮ \\ σ^{2} ρ & σ^{2} ρ & σ^{2} ρ & \dots & σ^{2} \end{matrix}]

Where σ² is the common variance assumed to be similar over time and ρ is the assumed correlation. Order of variance covariance matrix will be $t$ × $t$ . ‘t’ is the number of time points.

Auto regressive of order 1(AR1)

This is the homogeneous variance first-order autoregressive structure. Any two elements that are adjacent have a correlation that is equal to rho (ρ), those separated by a third will have correlation ρ², and so on. rho is restricted such that –1< ρ <1.

[\begin{matrix} σ^{2} & σ^{2} ρ & σ^{2} ρ^{2} & \dots & σ^{2} ρ^{t - 1} \\ σ^{2} ρ & σ^{2} & σ^{2} ρ & \dots & σ^{2} ρ^{t - 2} \\ σ^{2} ρ^{2} & σ^{2} ρ & σ^{2} & \dots & σ^{2} ρ^{t - 3} \\ ⋮ & ⋱ & ⋮ & ⋮ & ⋮ \\ σ^{2} ρ^{t - 1} & σ^{2} ρ^{t - 2} & σ^{2} ρ^{t - 3} & \dots & σ^{2} \end{matrix}]

Where σ² is the common variance assumed to be similar over time and ρ is the assumed correlation. Order of variance covariance matrix will be $t$ x $t$ . ‘t’ is the number of time points.

Also, we considered different scenarios of how we want to analyze the results at the end as different contrasts as described below.

Contrast for repeated measures

We tried to investigate two types of contrasts.

1. Time-related contrasts i.e., mean over time (mean contrast).
Rationale: This contrast comes from the chronic types of disease where regular medicine is required, and the effect remains only for some time and then the disease condition reverses back.
This will be labelled in the legend of Figure 1 as CS_mean(i) and in the legend of Figure 2 as AR1_mean(i). For e.g., for five timepoints the contrast would look like c(-1, ¼, ¼, ¼, ¼).
2. Mean Difference (change at last time point from baseline) (diff contrast).
Rationale: This contrast comes from the types of disease where each dose reduces the disease severity and the over the course it is totally removed from the patient’s body. Here the total effect at the end of the treatment course as compared to the baseline is of interest.
This will be labelled in the legend of Figure 1 as CS_diff(i) and in the legend of Figure 2 as AR1_diff(i). For e.g., for five timepoints the contrast would look like c(-1, 0, 0, 0, 1).

Sample size was calculated for correlation ranging from 0 – 1 with intervals of 0.05 for both the plots Figure 1 and Figure 2. Sample Size was derived using the formulae mentioned in Equation 13.

CS variance structure with ‘mean over time’ and ‘mean difference’ contrasts

The red horizontal line represents the sample size from single time point assessment approach.

CS_diff(i), CS_mean(i) – i represents the no. of visits used for sample size calculation, i = 3,4,5,6,8,10.

AR(1) covariance structure with ‘mean over time’ and ‘mean difference’ contrasts

The red horizontal line represents the sample size from single time point assessment approach.

AR1_diff(i), AR1_mean(i) – i represents the no. of visits used for sample size calculation, i = 3,4,5,6,8,10.

Results

Under CS type of variance covariance structure (Figure 1)

All the trend lines for mean difference type of contrast overlaps each other. For mean difference type of contrast, the sample size doesn’t change for an increase/decrease in the number of visits. It changes with the correlation i.e., highly correlated (rho > 0.5) timepoints would need less sample size as compared to low correlated timepoints. Also, for correlation = 0.5 the multiple assessment sample size coincides with that of single time point assessment.

However, the sample size does vary when the contrast is set to mean over time. Multiple time point assessment with more timepoints requires less sample size as compared to that of multiple time point assessment with less time points for e.g., the multiple time point assessment with three timepoint requires 86 per arm with correlation 0.8 and the multiple time point assessment with 10 timepoints requires 64 per arm. On the same lines the multiple time point assessment with three timepoints requires 258 per arm with correlation 0.4 and the multiple time point assessment with 10 timepoints requires 192 per arm. This trend shows that the sample size required reduces when the correlation increases. For correlation = 0.1 to 0.35 the sample size coincides with that of single time point assessment.

Under AR(1) type of variance covariance structure (Figure 2)

Under mean over time contrast the multiple time point assessment requires lower sample size as compared to single time point assessment (287 per arm) for correlation greater than 0.35 and the sample size increases as correlation goes below 0.35.

Whereas for mean difference contrast the multiple time point assessment requires lower sample size as compared to single time point assessment (287 per arm) for correlation greater than 0.7 but requires higher sample size for correlation less than 0.7.

The trend changes shape for mean difference contrast vs mean over time contrast. Also, at certain point the increase in sample size attenuates for e.g., in case of mean difference type contrast with 10 time points the sample size required doesn’t changes when correlation drops below 0.55.

Discussion

One of the hurdles in considering the longitudinal methodology for sample size calculation is the assumption on the covariance matrix. It is often easy to estimate the variance of single timepoint as compared to estimating the variance-covariance matrix for multiple time points.

The above derivations were done for trial design with parallel group, 1:1 ratio and two arms. If the ratio changes or if we have more than two arms or if the design is crossover, then the effective overall sample size would get effected in both the cases i.e., sample size with single time point as well as sample size with multiple timepoints, but the trend would remain the same as shown above in the figures and the results will still hold good. Similar trends should hold for other variance – covariance structures though they have not been simulated here.

Conclusion

Sample size changes depending on the analysis type and the data collected. Both the graphs in Figure 1 and Figure 2 in this study reveal that if response is assessed at multiple timepoints and the correlation between the paired observations is high (> 0.6) then one should consider using repeated measures analysis and consequently determine the size of the sample that is based on the multiple time points scenario which results in lower sample size requirement as compared to the sample size derived assuming single timepoint response assessment. This would reduce the cost, resources, and time in conducting the experiment fastening the new drug development. Also, repeated measures analyses will not drop the patients in which they have certain missing data as compared to single point analysis where the patient will be dropped if the response is missing hence may help in retaining the power.

Sample size derivation using longitudinal design method for studies with multiple assessments can be considered of substantial benefit in cost and time although the challenge of estimating the variance-covariance matrix remains.

Software availability

Software available from: The Comprehensive R Archive Network (https://cran.r-project.org/)

Source code available from: https://github.com/Sarfaraz-Sayyed/Sample-Size-Variation.

Archived source code at time of publication: https://zenodo.org/badge/latestdoi/547747570.¹⁵

License: MIT License.

Data availability

No data are associated with this article.

Acknowledgement

We would like to thank Novartis Healthcare Pvt. Ltd. and Manipal Academy of Higher Education for the support to carry out this research and all the reviewers for providing their valuable feedback and suggestions.

References

1. Fleiss JL: Design and Analysis of Clinical Experiments. New York:John Wiley & Sons;1986.
2. Biemer PP, Lyberg LE: Introduction to Survey quality. Hoboken, New Jersey:John Wiley and Sons;2003.
3. Arshed N, Rehman HU, Nazim M, et al.: Evading Law Of Diminishing Returns, A Case Of Human Capital Development. Journal of Contemporary Issues in Business and Government. 2021; 27(5): 2569–2584.
4. Khelif A, Saleh MN, Salama A, et al.: Changes in Health-Related Quality of Life with Long-term Eltrombopag Treatment in Adults with Persistent/Chronic Immune Thrombocytopenia: Findings from the EXTEND Study. Am. J. Hematol. 2019; 94: 200–208. PubMed Abstract | Publisher Full Text
5. Palileo-Villanueva LM, Dans AL: Composite endpoints. J. Clin. Epidemiol. 2020; 128: 157–158. Publisher Full Text
6. Bloch DA: Sample size requirements and the cost of a randomized clinical trial with repeated measurements. Stat. Med. 1986; 5(6): 663–667. PubMed Abstract | Publisher Full Text
7. Hedekar D, Gibbons RD, Waternaux C: Sample size estimation for longitudinal designs with attrition: Comparing time-related contrasts between two groups. J. Educ. Behav. Stat. 1999; 24(1): 70–93. Publisher Full Text
8. Basagana X, Liao X, Spiegelman D: Power and sample size calculations for longitudinal studies estimating a main effect of a time-varying exposure. Stat. Methods Med. Res. 2011; 20(5): 471–487. PubMed Abstract | Publisher Full Text
9. Pourhoseingholi MA, Vahedi M, Rahimzadeh M: Gastroenterol Hepatol Bed Bench. Winter. Sample size calculation in medical studies. 2013; 6(1): 14–17.
10. Hajian-Tilaki K: Sample size estimation in epidemiologic studies. Caspian J. Intern. Med. 2011; 2(4): 289–298. PubMed Abstract
11. Veena Manja MD, Satyan Lakshminrusimha MD: Epidemiology and Clinical Research Design, Part 1: Principles. NeoReviews. 2015; 16(2): e94–e108. PubMed Abstract | Publisher Full Text
12. Manja V MD, Lakshminrusimha S MD: Epidemiology and Clinical Research Design, Part 1: Study Types. Neoreviews. NeoReviews. 2014; 15(12): e558–e569. PubMed Abstract | Publisher Full Text
13. Diggle PJ, Heagerty P, Liang K-Y: Analysis of Longitudinal Data. Second EditionNew York:Oxford Statistical Science Series;2002.
14. Black DM, Reid IR, Cauley JA, et al.: The Effect of 6 Versus 9 Years of Zoledronic AcidTreatment in Osteoporosis: A Randomized SecondExtension to the HORIZON-Pivotal Fracture Trial (PFT). J. Bone Miner. Res. 2014; 30(5): 934–944. Publisher Full Text
15. Sarfaraz-Sayyed: Sarfaraz-Sayyed/Sample-Size-Variation: Sample Size variation (V1.0.0). Zenodo. Software.2022. Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 21 Dec 2022

Author details Author details

Sarfaraz Sayyed
Roles: Formal Analysis, Investigation, Visualization, Writing – Original Draft Preparation

Ashwini Mathur
Roles: Conceptualization, Supervision, Writing – Review & Editing

Asha Kamath
Roles: Conceptualization, Supervision, Writing – Review & Editing

Competing interests

The author (Sarfaraz Sayyed) and the co-author (Ashwini Mathur) are employed by Novartis Healthcare Pvt. Ltd.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (2)

version 2

Revised

Published: 05 Sep 2024, 11:1550

https://doi.org/10.12688/f1000research.124917.2

version 1

Published: 21 Dec 2022, 11:1550

https://doi.org/10.12688/f1000research.124917.1

© 2022 Sayyed S et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Sayyed S, Mathur A and Kamath A. Sample size variation in single-time post-dose assessment vs multi-time post-dose assessment [version 1; peer review: 2 not approved]. F1000Research 2022, 11:1550 (https://doi.org/10.12688/f1000research.124917.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 21 Dec 2022

Views

Reviewer Report 10 Jul 2024

Ronald Geskus, Oxford University, Oxford, England, UK

Not Approved

https://doi.org/10.5256/f1000research.137160.r294678

The authors describe some formulas for sample size calculations with longitudinal data with a continuous outcome. The authors only consider one specific approach, namely generalized least squares and comparing changes from baseline. This latter approach is discouraged, see e.g. https://www.bmj.com/content/323/7321/1123.

More general methods to perform sample size and power calculations with longitudinal data have been described and implemented in software (e.g. https://journal.r-project.org/articles/RJ-2022-022/).

Also, there are some descriptions that need further attention.

1. The introduction describes the basic principles of hypothesis testing. This can be assumed known and therefore I suggest to leave it out of the paper.

2. Page 3: The authors describe sampling error as the uncertainty arising from: "the limitation to examine every situation, the absolute precision of the effect of protein shake on the athlete's performance would be hard to measure." I am not sure whether I understand what is written here, but I think it is not correct.

3. Page 3: Sampling error decreases with increasing sample size, but it is not true that " there comes a point where increasing the sample size has no further effect on the sampling error".

4. Formula (2): power is typically calculated for a specific choice of the effect size, not for all delta>0 at once.

5. Page 7: "Appropriate sample size was calculated for multi-time and single time cases with different scenarios to achieve an overall mean treatment difference (0.9 points)." What is meant with mean difference in the situation with more than 2 time points? The difference could vary per time point but still give a mean difference of 0.9.
The last sentence of that paragraph is irrelevant information.

6. How was \sigma_{ij} chosen?

7. Page 8: make more explicit that a discrete AR(1) is assumed. There also exitsts a continuous AR(1)

8. Page 9: The authors write: "For correlation = 0.1 to 0.35 the sample size coincides with that of single time point assessment." I don't see this in the figure

9. Page 9: I don't understand the rationale "effect remains only for some time and then the disease condition reverses back' and "each dose reduces the disease severity and the over the course it is totally removed from the patient’s body" for the two contrasts.

10. Explain why the sample size goes down to zero if the correlation approaches 1. A correlation of 1 does not make sense. It implies that the second measurement is completely determined by the first, and the intervention cannot have any impact.

11. Legends Fig 1 and 2 have been mixed up.

Is the rationale for developing the new method (or application) clearly explained?

No
Is the description of the method technically sound?

Yes
Are sufficient details provided to allow replication of the method development and its use by others?

Yes
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Biostatistics

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

CITE

Report a concern

Author Response 05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

05 Sep 2024

Author Response

Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to ... Continue reading Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to propose a framework, but we have modified it now to make it more precise.

- Regarding methods to perform sample size calculation, we agree with you, but our intention here is to highlight using simple methods to the clinical researchers, the difference in samples required when we have longitudinal information available under 2 different correlation structures.

Comment 1: We kept this as this would be helpful to new researchers not having statistical background.
Comment 2: We have modified it for better understanding.
Comment 3: We have modified and added a note to have more clarity.
Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion.
Comment 5: We have modified the sentence for better clarity and also removed the last sentence of the paragraph.
Comment 6: Sigma was taken from a trial data (reference given) and was assumed to be same over all the time periods.
Comment 7: Thanks for bringing this up, we have updated this in the manuscript now.
Comment 8: This was wrongly written under figure 1, it was supposed to be under figure 2 conclusion. we have made the necessary changes now.
Comment 9: The first rationale refers to chronic diseases where the disease can usually be controlled but not cured. So here we are trying to see that the average effect from the treatment is the one which is expected to control the disease. Have changed the wordings for better understanding. The second rationale is for infectious diseases where with the right treatment regimen the disease will be cured. Have now tried to put in better words.
Comment 10: Agree with your comment on both the extreme cases that a correlation of 1 would imply all the effect is due to intervention not leaving any space for error and also that correlation 0 would mean no effect from the intervention. We have now revised the plots to start from 0.05 till 0.9
Comment 11: Have made the necessary changes.

- Regarding clarifying the rationale, basically in clinical trials the use of all time point data is not generally used. The effect at intermittent time points and the covariance structure between time points would be helpful which otherwise not taken into consideration results in large sample size.

- Regarding conclusion on the methods, we have updated the conclusion to have more insights, how taking into correlation would benefit and also how the effects diminish even if too many timepoints are added.
Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to propose a framework, but we have modified it now to make it more precise.

- Regarding methods to perform sample size calculation, we agree with you, but our intention here is to highlight using simple methods to the clinical researchers, the difference in samples required when we have longitudinal information available under 2 different correlation structures.

Comment 1: We kept this as this would be helpful to new researchers not having statistical background.
Comment 2: We have modified it for better understanding.
Comment 3: We have modified and added a note to have more clarity.
Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion.
Comment 5: We have modified the sentence for better clarity and also removed the last sentence of the paragraph.
Comment 6: Sigma was taken from a trial data (reference given) and was assumed to be same over all the time periods.
Comment 7: Thanks for bringing this up, we have updated this in the manuscript now.
Comment 8: This was wrongly written under figure 1, it was supposed to be under figure 2 conclusion. we have made the necessary changes now.
Comment 9: The first rationale refers to chronic diseases where the disease can usually be controlled but not cured. So here we are trying to see that the average effect from the treatment is the one which is expected to control the disease. Have changed the wordings for better understanding. The second rationale is for infectious diseases where with the right treatment regimen the disease will be cured. Have now tried to put in better words.
Comment 10: Agree with your comment on both the extreme cases that a correlation of 1 would imply all the effect is due to intervention not leaving any space for error and also that correlation 0 would mean no effect from the intervention. We have now revised the plots to start from 0.05 till 0.9
Comment 11: Have made the necessary changes.

- Regarding clarifying the rationale, basically in clinical trials the use of all time point data is not generally used. The effect at intermittent time points and the covariance structure between time points would be helpful which otherwise not taken into consideration results in large sample size.

- Regarding conclusion on the methods, we have updated the conclusion to have more insights, how taking into correlation would benefit and also how the effects diminish even if too many timepoints are added.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

05 Sep 2024

Author Response

Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to ... Continue reading Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to propose a framework, but we have modified it now to make it more precise.

- Regarding methods to perform sample size calculation, we agree with you, but our intention here is to highlight using simple methods to the clinical researchers, the difference in samples required when we have longitudinal information available under 2 different correlation structures.

Comment 1: We kept this as this would be helpful to new researchers not having statistical background.
Comment 2: We have modified it for better understanding.
Comment 3: We have modified and added a note to have more clarity.
Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion.
Comment 5: We have modified the sentence for better clarity and also removed the last sentence of the paragraph.
Comment 6: Sigma was taken from a trial data (reference given) and was assumed to be same over all the time periods.
Comment 7: Thanks for bringing this up, we have updated this in the manuscript now.
Comment 8: This was wrongly written under figure 1, it was supposed to be under figure 2 conclusion. we have made the necessary changes now.
Comment 9: The first rationale refers to chronic diseases where the disease can usually be controlled but not cured. So here we are trying to see that the average effect from the treatment is the one which is expected to control the disease. Have changed the wordings for better understanding. The second rationale is for infectious diseases where with the right treatment regimen the disease will be cured. Have now tried to put in better words.
Comment 10: Agree with your comment on both the extreme cases that a correlation of 1 would imply all the effect is due to intervention not leaving any space for error and also that correlation 0 would mean no effect from the intervention. We have now revised the plots to start from 0.05 till 0.9
Comment 11: Have made the necessary changes.

- Regarding clarifying the rationale, basically in clinical trials the use of all time point data is not generally used. The effect at intermittent time points and the covariance structure between time points would be helpful which otherwise not taken into consideration results in large sample size.

- Regarding conclusion on the methods, we have updated the conclusion to have more insights, how taking into correlation would benefit and also how the effects diminish even if too many timepoints are added.
Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to propose a framework, but we have modified it now to make it more precise.

- Regarding methods to perform sample size calculation, we agree with you, but our intention here is to highlight using simple methods to the clinical researchers, the difference in samples required when we have longitudinal information available under 2 different correlation structures.

Comment 1: We kept this as this would be helpful to new researchers not having statistical background.
Comment 2: We have modified it for better understanding.
Comment 3: We have modified and added a note to have more clarity.
Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion.
Comment 5: We have modified the sentence for better clarity and also removed the last sentence of the paragraph.
Comment 6: Sigma was taken from a trial data (reference given) and was assumed to be same over all the time periods.
Comment 7: Thanks for bringing this up, we have updated this in the manuscript now.
Comment 8: This was wrongly written under figure 1, it was supposed to be under figure 2 conclusion. we have made the necessary changes now.
Comment 9: The first rationale refers to chronic diseases where the disease can usually be controlled but not cured. So here we are trying to see that the average effect from the treatment is the one which is expected to control the disease. Have changed the wordings for better understanding. The second rationale is for infectious diseases where with the right treatment regimen the disease will be cured. Have now tried to put in better words.
Comment 10: Agree with your comment on both the extreme cases that a correlation of 1 would imply all the effect is due to intervention not leaving any space for error and also that correlation 0 would mean no effect from the intervention. We have now revised the plots to start from 0.05 till 0.9
Comment 11: Have made the necessary changes.

- Regarding clarifying the rationale, basically in clinical trials the use of all time point data is not generally used. The effect at intermittent time points and the covariance structure between time points would be helpful which otherwise not taken into consideration results in large sample size.

- Regarding conclusion on the methods, we have updated the conclusion to have more insights, how taking into correlation would benefit and also how the effects diminish even if too many timepoints are added.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 30 May 2024

Kiranmoy Das, Indian Statistical Institute, Kolkata, West Bengal, India

Not Approved

https://doi.org/10.5256/f1000research.137160.r273771

Reviewer’s comments on the manuscript “Sample-size variation in single-time post-dose assessment vs multi-time post-dose assessment”

The manuscript attempts to address a very important and interesting problem. However, the presentation is not quite clear, there are lots of typos and grammar issues here and there. The methodology developed in the manuscript can be improved. My comments are given below. After addressing these comments and after a substantial revision, I can make some positive comment on the manuscript.
1. Introduction: Please provide some good motivation and nice description of the problem. There are several typos. Please avoid the phrase For e.g., this should be For example, and also “The size of the sample and the effect size are major factors which affects”, this should be like “The sample size and effect size are major factors that affect”.
2. Methods: Equation (1), you are assuming here that the variance sigma^2 is known. Please specify it, and also how appropriate this assumption is in real application?
3. For the longitudinal studies, your method depends on the auto-correlation \rho. You need to estimate \rho before calculating the sample size. Please clarify that and also discuss how to estimate \rho. Maybe you run a pilot trial and get an estimate?
4. Both the figures 1 and 2 focus on how the sample size varies with correlation \rho. For a longitudinal trial, it maybe more important to know how it covaries with the number of time points. Correlation is something that cannot be adjusted, but the number of time points can be adjusted as per the requirement. I think this issue has been ignored in the manuscript.

Is the rationale for developing the new method (or application) clearly explained?

No
Is the description of the method technically sound?

Yes
Are sufficient details provided to allow replication of the method development and its use by others?

No
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Biostatistics, Longitudinal data analysis, Bayesian modeling

CITE

Report a concern

Author Response 05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

05 Sep 2024

Author Response

Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. ... Continue reading Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. 1.
Comment 3: An explanation of rho estimation is now added.
Comment 4: We have added a paragraph under the conclusion section to address this.

We have added a few lines in the Introduction section to clarify the rationale.
Regarding details for replication of the method, details on assumptions are specified in the section “Calculation of sample size” in the manuscript. The software availability section provides the link to the related R program residing in Github.
Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. 1.
Comment 3: An explanation of rho estimation is now added.
Comment 4: We have added a paragraph under the conclusion section to address this.

We have added a few lines in the Introduction section to clarify the rationale.
Regarding details for replication of the method, details on assumptions are specified in the section “Calculation of sample size” in the manuscript. The software availability section provides the link to the related R program residing in Github.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

05 Sep 2024

Author Response

Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. ... Continue reading Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. 1.
Comment 3: An explanation of rho estimation is now added.
Comment 4: We have added a paragraph under the conclusion section to address this.

We have added a few lines in the Introduction section to clarify the rationale.
Regarding details for replication of the method, details on assumptions are specified in the section “Calculation of sample size” in the manuscript. The software availability section provides the link to the related R program residing in Github.
Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. 1.
Comment 3: An explanation of rho estimation is now added.
Comment 4: We have added a paragraph under the conclusion section to address this.

We have added a few lines in the Introduction section to clarify the rationale.
Regarding details for replication of the method, details on assumptions are specified in the section “Calculation of sample size” in the manuscript. The software availability section provides the link to the related R program residing in Github.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 21 Dec 2022

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2	3
Version 2 (revision) 05 Sep 24	read	read	read
Version 1 21 Dec 22	read	read

Kiranmoy Das, Indian Statistical Institute, Kolkata, India
Ronald Geskus, Oxford University, Oxford, UK
Simon Vandekar, Vanderbilt University Medical Center, Tennessee, USA

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

3 Views

21 Nov 2024 | for Version 2

Simon Vandekar, Vanderbilt University Medical Center, Tennessee, USA

3 Views Cite this report Responses(0)

Not Approved

The paper doesn’t reference the huge body of literature on power analysis in longitudinal studies. There are 14 references and only of them is a classical text on longitudinal data analysis (Diggle et al). For example, the book by Ahn, Heo, Zhang “Sample Size Calculations for Clustered and Longitudinal Outcomes in Clinical Research.” There are numerous R package for longitudinal power analyses, but these were not discussed (e.g. https://journal.r-project.org/articles/RJ-2022-022/). The results and topic are not novel and overlook critical references that have studied the topic. Given the very general topic, I would have expected more references to the existing literature.

There need to be more references for the mathematics in the paper, such as “Method for Sample size with single time point assessment analysis.” Grammar in the paper needs to be reviewed. There are some notation errors, e.g. in equations (8) and (9) Var(\Gamma) doesn’t make sense because Gamma is defined as a parameter (so it has zero variance).

Is the rationale for developing the new method (or application) clearly explained?

Yes
Is the description of the method technically sound?

Partly
Are sufficient details provided to allow replication of the method development and its use by others?

Yes
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

References

1. Ahn C, Heo M, Zhang S: Sample Size Calculations for Clustered and Longitudinal Outcomes in Clinical Research. 2014. Publisher Full Text

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Biostatistics

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

4 Views

23 Oct 2024 | for Version 2

Kiranmoy Das, Indian Statistical Institute, Kolkata, West Bengal, India

4 Views Cite this report Responses(0)

Approved

I have seen the revised version of the paper and the author's comments.
I am quite satisfied and I have no additional comments.
I recommend the acceptance of the manuscript

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Biostatistics, Longitudinal data analysis, Bayesian modeling

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

10 Views

28 Sep 2024 | for Version 2

Ronald Geskus, Oxford University, Oxford, England, UK

10 Views Cite this report Responses(0)

Not Approved

The article still contains several typos, sentences that abruptly end and capital letters that are misplaced. I suggest the authors do a careful check of typos. Also, I still found a couple of errors, as well as a few statements that are not clear to me.

1. "It affects the estimator’s precision, which is a metric that is important for the chosen target population of all professional athletes". Precision is a characteristic of the data. Explain why precision is an important metric for the target population.

2. The authors found only two papers (references 3 and 4) in their literature review on longitudinal studies? There must be many more.

3. Spiegelma -> Spiegelman

4. Page 4: Make clear that the normal distribution of the sample mean is an approximation, which becomes more accurate with increasing sample size (based on the central limit theorem)

5. The authors replied: "Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion." I doubt whether "H_a is true" and formula (2) avoid confusion. To my opinion, it is clearer to change "H_a is true" and "\delta > 0" by a specific value for \delta.

6. Page 5: H_a -> H_0 for null hypothesis, and \psi_c=C^t \Lambda, with "t" meaning transpose.

7. Formula (8) is not a test statistic, \psi_c is a property of the population not based on random variables. Also, \Lambda is a population property, not determined by random variables.

8. covar(Y_2-X_2,Y_1-X_1)=covar(Y_2,Y_1)+covar(X_2,X_1). I don't see where the value 4 comes from in (9).

9. I still find the section "Calculation of sample size" not very clearly phrased. Specify what is meant with "the last time point" (later it is written to be 3 years). What time trend does it imply? Replace "allowing" by "choosing"

10. Explain how the sample size of 287 is obtained. How large is the assumed difference under the null hypothesis, i.e. what is the chosen time point?

11. The rationale for the mean over time contrast chosen is still not clear to me. Why would one choose 1/4 at later time points, given that a linear trend is assumed?

12. Can the authors explain i) why the sample size for correlation=0.5 (compound symmetry) coincides with single time point assessment and ii) why the required sample size is larger if correlation<0.5?

13. I do not understand why with low correlation the single time point assessment performs best. Please explain.

14. I do not understand why the sample size for AR(1) mean over time contrast increases with increasing number of time points included if correlation > 0.35. Please explain.

15. Page 11: "Whereas for the mean diff contrast the sample size required with fewer visit is lower as compared to the sample size required with higher number of visits". It's not the number of visits that matters (it is always based on two time points only), but the distance between the visits. The required sample size for AR(1) mean difference contrast increases if the second time point is further away from the baseline. I don't think this is a general property. It will depend on the combination of serial correlation and difference in mean at the chosen time point. This needs further investigation.

16. The authors give a detailed introduction for readers who do not have a statistical background. It would be very useful to provide some guidance how the results can directly be used by them.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

biostatistics

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

15 Views

10 Jul 2024 | for Version 1

Ronald Geskus, Oxford University, Oxford, England, UK

15 Views Cite this report Responses(1)

Not Approved

Is the rationale for developing the new method (or application) clearly explained?

No
Is the description of the method technically sound?

Yes
Are sufficient details provided to allow replication of the method development and its use by others?

Yes
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Biostatistics

Respond to this report

Responses (1)

Author Response

05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to propose a framework, but we have modified it now to make it more precise.

- Regarding methods to perform sample size calculation, we agree with you, but our intention here is to highlight using simple methods to the clinical researchers, the difference in samples required when we have longitudinal information available under 2 different correlation structures.

Comment 1: We kept this as this would be helpful to new researchers not having statistical background.
Comment 2: We have modified it for better understanding.
Comment 3: We have modified and added a note to have more clarity.
Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion.
Comment 5: We have modified the sentence for better clarity and also removed the last sentence of the paragraph.
Comment 6: Sigma was taken from a trial data (reference given) and was assumed to be same over all the time periods.
Comment 7: Thanks for bringing this up, we have updated this in the manuscript now.
Comment 8: This was wrongly written under figure 1, it was supposed to be under figure 2 conclusion. we have made the necessary changes now.
Comment 9: The first rationale refers to chronic diseases where the disease can usually be controlled but not cured. So here we are trying to see that the average effect from the treatment is the one which is expected to control the disease. Have changed the wordings for better understanding. The second rationale is for infectious diseases where with the right treatment regimen the disease will be cured. Have now tried to put in better words.
Comment 10: Agree with your comment on both the extreme cases that a correlation of 1 would imply all the effect is due to intervention not leaving any space for error and also that correlation 0 would mean no effect from the intervention. We have now revised the plots to start from 0.05 till 0.9
Comment 11: Have made the necessary changes.

- Regarding clarifying the rationale, basically in clinical trials the use of all time point data is not generally used. The effect at intermittent time points and the covariance structure between time points would be helpful which otherwise not taken into consideration results in large sample size.

- Regarding conclusion on the methods, we have updated the conclusion to have more insights, how taking into correlation would benefit and also how the effects diminish even if too many timepoints are added.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

18 Views

30 May 2024 | for Version 1

Kiranmoy Das, Indian Statistical Institute, Kolkata, West Bengal, India

18 Views Cite this report Responses(1)

Not Approved

Is the rationale for developing the new method (or application) clearly explained?

No
Is the description of the method technically sound?

Yes
Are sufficient details provided to allow replication of the method development and its use by others?

No
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Biostatistics, Longitudinal data analysis, Bayesian modeling

Respond to this report

Responses (1)

Author Response

05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. 1.
Comment 3: An explanation of rho estimation is now added.
Comment 4: We have added a paragraph under the conclusion section to address this.

We have added a few lines in the Introduction section to clarify the rationale.
Regarding details for replication of the method, details on assumptions are specified in the section “Calculation of sample size” in the manuscript. The software availability section provides the link to the related R program residing in Github.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Fleiss JL: Design and Analysis of Clinical Experiments. New York:John Wiley & Sons;1986.

[2] 2. Biemer PP, Lyberg LE: Introduction to Survey quality. Hoboken, New Jersey:John Wiley and Sons;2003.

[3] 3. Arshed N, Rehman HU, Nazim M, et al.: Evading Law Of Diminishing Returns, A Case Of Human Capital Development. Journal of Contemporary Issues in Business and Government. 2021; 27(5): 2569–2584.

[4] 4. Khelif A, Saleh MN, Salama A, et al.: Changes in Health-Related Quality of Life with Long-term Eltrombopag Treatment in Adults with Persistent/Chronic Immune Thrombocytopenia: Findings from the EXTEND Study. Am. J. Hematol. 2019; 94: 200–208. PubMed Abstract | Publisher Full Text

[5] 5. Palileo-Villanueva LM, Dans AL: Composite endpoints. J. Clin. Epidemiol. 2020; 128: 157–158. Publisher Full Text

[6] 6. Bloch DA: Sample size requirements and the cost of a randomized clinical trial with repeated measurements. Stat. Med. 1986; 5(6): 663–667. PubMed Abstract | Publisher Full Text

[7] 7. Hedekar D, Gibbons RD, Waternaux C: Sample size estimation for longitudinal designs with attrition: Comparing time-related contrasts between two groups. J. Educ. Behav. Stat. 1999; 24(1): 70–93. Publisher Full Text

[8] 8. Basagana X, Liao X, Spiegelman D: Power and sample size calculations for longitudinal studies estimating a main effect of a time-varying exposure. Stat. Methods Med. Res. 2011; 20(5): 471–487. PubMed Abstract | Publisher Full Text

[9] 9. Pourhoseingholi MA, Vahedi M, Rahimzadeh M: Gastroenterol Hepatol Bed Bench. Winter. Sample size calculation in medical studies. 2013; 6(1): 14–17.

[10] 10. Hajian-Tilaki K: Sample size estimation in epidemiologic studies. Caspian J. Intern. Med. 2011; 2(4): 289–298. PubMed Abstract

[11] 11. Veena Manja MD, Satyan Lakshminrusimha MD: Epidemiology and Clinical Research Design, Part 1: Principles. NeoReviews. 2015; 16(2): e94–e108. PubMed Abstract | Publisher Full Text

[12] 12. Manja V MD, Lakshminrusimha S MD: Epidemiology and Clinical Research Design, Part 1: Study Types. Neoreviews. NeoReviews. 2014; 15(12): e558–e569. PubMed Abstract | Publisher Full Text

[13] 13. Diggle PJ, Heagerty P, Liang K-Y: Analysis of Longitudinal Data. Second EditionNew York:Oxford Statistical Science Series;2002.

[14] 14. Black DM, Reid IR, Cauley JA, et al.: The Effect of 6 Versus 9 Years of Zoledronic AcidTreatment in Osteoporosis: A Randomized SecondExtension to the HORIZON-Pivotal Fracture Trial (PFT). J. Bone Miner. Res. 2014; 30(5): 934–944. Publisher Full Text

[15] 15. Sarfaraz-Sayyed: Sarfaraz-Sayyed/Sample-Size-Variation: Sample Size variation (V1.0.0). Zenodo. Software.2022. Publisher Full Text

Sample size variation in single-time post-dose assessment vs multi-time post-dose assessment

Abstract

Keywords

Introduction

Background

Objective

Notation and framework

Methods

Method for Sample size with single time point assessment analysis

(1)

(2)

(3)

(4)

(5)

(6)

Method for Sample size with multiple time point assessment analysis

(7)

(8)

(9)

(10)

(11)

(12)

(13)

Calculation of sample size

Sample size (single time point case)

Sample size (longitudinal case)

Figure 1. Simulation results with compound symmetry correlation structure.

Figure 2. Simulation results with auto regressive order 1 correlation structure.

Contrast for repeated measures

CS variance structure with ‘mean over time’ and ‘mean difference’ contrasts

AR(1) covariance structure with ‘mean over time’ and ‘mean difference’ contrasts

Results

Under CS type of variance covariance structure (Figure 1)

Under AR(1) type of variance covariance structure (Figure 2)

Discussion

Conclusion

Software availability

Data availability

Acknowledgement

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated