Sample size variation in single-time post-dose assessment <i>vs</i> multi-time post-dose assessment

Sarfaraz Sayyed; Ashwini Mathur; Asha Kamath

doi:10.12688/f1000research.124917.2

Home Browse Sample size variation in single-time post-dose assessment vs multi-time...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Method Article

Revised

Sample size variation in single-time post-dose assessment vs multi-time post-dose assessment

[version 2; peer review: 1 approved, 2 not approved]

Sarfaraz Sayyed ¹, Ashwini Mathur², Asha Kamath³

PUBLISHED 05 Sep 2024

Author details Author details

¹ Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, Telangana, 500080, India
² Clinical Technology & Innovation, Novartis Ireland Ltd., Dublin, Ireland
³ Department of Data Science, Manipal academy of higher studies, Manipal, karnataka, 576104, India

Sarfaraz Sayyed
Roles: Formal Analysis, Investigation, Visualization, Writing – Original Draft Preparation

Ashwini Mathur
Roles: Conceptualization, Supervision, Writing – Review & Editing

Asha Kamath
Roles: Conceptualization, Supervision, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Manipal Academy of Higher Education gateway.

Abstract

Background: Many randomized trials measure a continuous outcome simultaneously at baseline and after taking the drug. For a single continuous post-treatment outcome, the sample size calculation is simple, but if there are assessments at multiple time points post-treatment then this longitudinal data may give more insights by analyzing the data using the repeated measures method. Also, if the sample size is calculated using the single time-point method for longitudinal data, it may lead to a larger than required sample size, increasing the cost and time.

Methods: In this research, an effort is made to determine the size of the sample for repeated measures case and then compared with the single post-baseline case. The sample sizes were examined under different scenarios for the continuous type of response variable. Under Mean contrast and Diff contrast the sample sizes were calculated with different correlations. These two scenarios were again examined under compound symmetry as well as Auto regressive of order 1 type of correlation structure in longitudinal data. The graphical presentation is given for better visualization of the scenarios.

Results: Sample size required for highly correlated longitudinal data using multi timepoint sample size derivation method led to much smaller sample size requirement as compared to single timepoint sample size calculation method.

Conclusions: This study will help researchers to make better decisions in choosing the right method for sample size determination which may reduce the time and cost of carrying out the experiment. Also, we must carefully assess which method to go with when the correlation is weak. More complex correlation structures are not studied in this article but can be studied in the same fashion.

Keywords

sample size estimation; longitudinal study; repeated measure analysis; univariate analysis

Corresponding author: Sarfaraz Sayyed

Competing interests: The author (Sarfaraz Sayyed) and the co-author (Ashwini Mathur) are employed by Novartis Healthcare Pvt. Ltd.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2024 Sayyed S et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Sayyed S, Mathur A and Kamath A. Sample size variation in single-time post-dose assessment vs multi-time post-dose assessment [version 2; peer review: 1 approved, 2 not approved]. F1000Research 2024, 11:1550 (https://doi.org/10.12688/f1000research.124917.2) First published: 21 Dec 2022, 11:1550 (https://doi.org/10.12688/f1000research.124917.1) Latest published: 05 Sep 2024, 11:1550 (https://doi.org/10.12688/f1000research.124917.2)

Revised Amendments from Version 1

The introduction section has been updated with respect to motivation and description of the problem.
Justification has been added for obtaining the estimates of variance and correlation.
Information have been added on the attenuation with too many time points.
More clarification has been added to the parameters used.
Better explanation has been provided for the rationale of the contrast.
Plots are placed in the right order and the correlation range has been changed to 0.1 - 0.95.

See the authors' detailed response to the review by Kiranmoy Das
See the authors' detailed response to the review by Ronald Geskus

Introduction

Understanding the concept of “sample size” is crucial for anyone involved in scientific research or clinical trials. The sample size refers to the number of subjects selected or observed in an experiment. This sample is a subset of the entire target population, which includes all individuals relevant to the study. For instance, in a study testing a new drug for type II diabetes, the target population would consist of all individuals suffering from this condition.

The sample size significantly influences the precision of our estimates and the study's power. The power of a statistical test is the probability that it will correctly reject the null hypothesis when it is false, thus avoiding a Type II error. Essentially, higher power means a greater chance of detecting a true effect.

Two primary factors impact the power of a study: the sample size and the effect size. A larger sample size generally increases the study's power, enhancing our ability to draw accurate conclusions. Effect size, on the other hand, measures the magnitude of the difference or relationship being studied.

In clinical trials, carefully calculating the sample size is essential. It ensures that the study is adequately powered to meet its objectives, providing reliable and meaningful results. This meticulous planning is fundamental to advancing medical knowledge and improving patient care.

By paying close attention to sample size, researchers can design robust experiments that yield trustworthy insights, ultimately contributing to scientific progress and better health outcomes.

As an illustration, consider a study to compare the performance of a professional athlete taking a particular protein shake versus athletes who do not consume any special protein shakes. Narrowing down attention to a portion of the wider group is essential to enable tracking of the eating habits of every elite athlete in the world. Suppose this entails choosing 100 professional athletes for our study at random; in this case, 100 would be the sample size. Based on the data gathered from a sample of 100 elite athletes, the study’s findings potentially characterize the population of all athletes in the sports industry. Lack of full coverage of the target population would result in the study's outcome having a margin of error. Sampling error¹ is the term used to describe this level of uncertainty or inaccuracy. It affects the estimator’s precision, which is a metric that is important for the chosen target population of all professional athletes.

Although sampling error cannot be completely eliminated, it can be reduced.² A larger sample typically has a narrower margin of error. We require an appropriate sample size to examine and provide an accurate picture of the effects of protein shake consumption on performance. Note that increasing the sample size will help in reducing the sampling error but it does not addresses the non-sampling errors.

Background

Longitudinal studies take longer but help determine causality and monitor the trend over time. To see how sample size calculation was addressed in published longitudinal studies we searched the databases such as Scopus, Web of Science, PubMed, ScienceDirect, and Google Scholar using a range of key terms: “designing clinical trials”, “sample size calculation”, “longitudinal studies”, “randomized trials” and “repeated measures”. The ensuing literature review did not reveal much information on details of how the sample size was calculated for these published longitudinal studies.³^,⁴

Formulae for deriving sample size in longitudinal studies is available from several papers.⁵^,⁶ Basagana, Liao and Spiegelma⁷ published a study in which the power as well as the sample size are discussed for time-varying exposures, but how this is practically applied to a longitudinal study design and its outcome is undocumented in published papers. Pourhoseingholi et al.,⁸ and Karimollah⁹ both published about the importance of various components for calculating the sample size in medical studies or clinical trials where often there would be more than one post baseline assessment, but sample size calculation is shown assuming single post baseline assessment. Manja and Lakshminrusimha published a two-part study¹⁰^,¹¹ which does give a good explanation on clinical research design, but sample size is not discussed in detail.

Most of the published studies which have assessments at multiple time points calculate the sample size based on the change from study end time point to baseline whereas a smaller number of papers emphasize on the use of multiple time points into consideration for calculating the sample size.⁵^–⁷

The need for this research was prompted by this lack of proper usage of sample size calculation for longitudinal studies and to further explore which method for sample size calculation should be used in a longitudinal study resulting in correlated outcome data.

Objective

To explore the variation in sample size by considering multiple time point assessment versus the change from baseline to a single endpoint.

Notation and framework

In an experiment for testing certain hypothesis with parallel group design, two or more independent groups are treated under different scenarios to compare the outcome of the scenarios. In our study we would consider the objective of comparison of two drugs.

Let $Y_{ij}$ (X_ij) be the outcome of interest at j^th (j = 1, 2, 3, …, t) time point for the i^th (i = 1, 2, 3, …, n) patient in the two groups.

For the parallel group design, these 2n patients will be divided into two groups with 1: 1 ratio where one arm is assigned to receive the test drug and the other arm is assigned to receive the comparator drug.

Let $μ_{1} & μ_{2}$ be the population mean for the test drug and comparator drug respectively.

Let $\bar{Y} & \bar{X}$ be the sample mean outcome for the test drug and comparator drug respectively, where $\bar{Y} \sim N (μ_{1}, \frac{σ^{2}}{n})$ and $\bar{X} \sim N (μ_{2}, \frac{σ^{2}}{n})$ .

Methods

Method for Sample size with single time point assessment analysis

Change at single post-baseline assessment (Single time assessment analysis)

In a parallel group design study with two arms of equal size let the hypothesis be set as:

$H_{0} : μ_{1} - μ_{2} = δ = 0$ , No difference between the effects of test drug and comparator drug.

$H_{a} : μ_{1} - μ_{2} = δ$ , Test drug effect (where δ > 0) is greater than comparator.

The test statistic assuming known common standard deviation $σ$ (estimated from previous clinical trial data with same molecule which could be phase1, phase2 trials for same indication or pivotal trials with same molecule for different indication) for both arms will be given by

(1)

T = \frac{\bar{Y} - \bar{X}}{\sqrt{Var (\bar{Y} - \bar{X})}} = \frac{\bar{Y} - \bar{X}}{\sqrt{\frac{σ^{2}}{n} + \frac{σ^{2}}{n}}} = \frac{\bar{Y} - \bar{X}}{\sqrt{\frac{2 σ^{2}}{n}}}

Now If $H_{0}$ is true (and $μ_{1} = μ_{2}),$ then $T \sim N (0, 1)$ , else if $H_{a}$ is true (i.e., $δ \neq 0)$ , then $T$ will still follow gaussian distribution but with a mean greater than zero.

If Type II error is denoted by $β$ then power will be simply $1 - β$ and power is the probability to reject $H_{0}$ when $H_{a}$ is true. In probability equation it could be written as

(2)

Pr [Reject H_{0}| H_{a} is true] = Pr [T > z_{1 - α}| δ] = 1 - β,

where

z_{1 - α}

is the threshold or the critical value which is

(1 - α)

quantile from Gaussian distribution and

α

is the type I error or the level of significance.

(3)

T \sim N (\frac{δ}{\sqrt{\frac{2 σ^{2}}{n}}}, 1), Under H_{a} \to T - \frac{δ}{\sqrt{\frac{2 σ^{2}}{n}}} = Z \sim N (0, 1)

(4)

\begin{matrix} Power function = Pr [T > z_{1 - α}| δ] = Pr \{\frac{\bar{Y} - \bar{X} - δ}{\sqrt{\frac{2 σ^{2}}{n}}} > z_{1 - α} - \frac{δ}{\sqrt{\frac{2 σ^{2}}{n}}}\} \\ = 1 - Φ (z_{1 - α} - \sqrt{2 n} (\frac{δ}{2 σ})), where Φ (z) = Pr (Z \leq z) \end{matrix}

Now in any study we would be looking for below inequality.

(5)

\begin{matrix} 1 - Φ (z_{1 - α} - \sqrt{\frac{n}{2}} (\frac{δ}{σ})) \geq 1 - β \\ \to β \geq Φ (z_{1 - α} - \sqrt{\frac{n}{2}} (\frac{δ}{σ})) or z_{β} \geq z_{1 - α} - \sqrt{\frac{n}{2}} (\frac{δ}{σ}) \end{matrix}

After solving equation 5, we get

(6)

n \geq \frac{2 {(z_{1 - α} + z_{1 - β})}^{2} σ^{2}}{{(μ_{1} - μ_{2})}^{2}} = \frac{2 {(z_{α} + z_{β})}^{2}}{{[(μ_{1} - μ_{2}) / σ]}^{2}}

Here, n is the sample size required per arm. We will use these formulae in calculating the sample size for single post baseline time point analysis.

Method for Sample size with multiple time point assessment analysis

Post baseline assessment at multiple timepoints (Multiple time points analysis)

In a parallel group design study with two arms of equal size and assessments taken at multiple time points let the hypothesis be set as:

$H_{a} : ψ_{c} = 0$ , there is no difference between the effects of test and comparator drug.

$H_{a} : ψ_{c} > 0$ , test drug is having larger effect as compared to comparator.

Where $ψ_{c} = CΛ$ ,

Let $C = [\begin{matrix} c_{1} \\ c_{2} \\ ⋮ \\ c_{i} \\ ⋮ \\ c_{t} \end{matrix}]$ be the contrast to be tested for the hypothesis and let $Λ = [\begin{matrix} μ_{11} - μ_{21} \\ μ_{12} - μ_{22} \\ ⋮ \\ μ_{1 i} - μ_{2 i} \\ ⋮ \\ μ_{1 t} - μ_{2 t} \end{matrix}]$

$μ_{1 i} & μ_{2 i}$ are the mean effect in arm one and arm two at time point “i” respectively in a study with t time points. $c_{i}$ can take any value depending on the hypothesis we want to test.

For example, if we want to see the difference between two drugs when t = 2, then $c_{1} = - 1 and c_{2} = 1$ and the resulting $ψ_{c}$ will be

(7)

\begin{matrix} ψ_{c} = CΛ = [- 1 1] . [\begin{matrix} μ_{11} - μ_{21} \\ μ_{12} - μ_{22} \end{matrix}] \\ = - (μ_{11} - μ_{21}) + μ_{12} - μ_{22} \\ = (μ_{12} - μ_{11}) - (μ_{22} - μ_{21}) \\ = effect in Drug 1 - effect in Drug 2 . \end{matrix}

Common Variance covariance matrix = [\begin{matrix} σ_{1}^{2} & σ_{12} & \dots & σ_{1 t} \\ σ_{12} & σ_{2}^{2} & ⋱ & σ_{2 t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ σ_{i 1} & σ_{i 2} & ⋱ & σ_{i t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ σ_{1 t} & σ_{2 t} & \dots & σ_{t}^{2} \end{matrix}]

Where $σ_{i}^{2}$ is the variance at time point i and $σ_{ij}$ represents the covariance between time point i and j.

The test statistic assuming similar variance-covariance matrix for both arms will be given by

(8)

T = \frac{ψ_{c}}{\sqrt{Var (ψ_{c})}} = \frac{ψ_{c}}{\sqrt{Var (CΛ)}} = \frac{ψ_{c}}{\sqrt{C^{'} . Var (Λ) . C}}

Consider the $Var (Λ)$ ,

Var (Λ) = Var [\begin{matrix} μ_{11} - μ_{21} \\ μ_{12} - μ_{22} \\ ⋮ \\ μ_{1 i} - μ_{2 i} \\ ⋮ \\ μ_{1 t} - μ_{2 t} \end{matrix}] = Var [\begin{matrix} δ_{1} \\ δ_{2} \\ ⋮ \\ δ_{i} \\ ⋮ \\ δ_{t} \end{matrix}] = [\begin{matrix} \frac{2}{n} . σ_{1}^{2} & \frac{4}{n} . σ_{12} & \dots & \frac{4}{n} . σ_{1 t} \\ \frac{4}{n} . σ_{12} & \frac{2}{n} . σ_{2}^{2} & ⋱ & \frac{4}{n} . σ_{2 t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ \frac{4}{n} . σ_{i 1} & \frac{4}{n} . σ_{i 2} & ⋱ & \frac{4}{n} . σ_{i t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ \frac{4}{n} . σ_{1 t} & \frac{4}{n} . σ_{2 t} & \dots & \frac{2}{n} . σ_{t}^{2} \end{matrix}]

(9)

Var (Λ) = \frac{2}{n} . [\begin{matrix} σ_{1}^{2} & 2 σ_{12} & \dots & 2 σ_{1 t} \\ σ_{12} & σ_{2}^{2} & ⋱ & σ_{2 t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ σ_{i 1} & 2 σ_{i 2} & ⋱ & 2 σ_{i t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 2 σ_{1 t} & 2 σ_{2 t} & \dots & σ_{t}^{2} \end{matrix}]

Solving for $C^{'} . Var (Λ) . C$ , we get

(10)

\begin{matrix} C^{'} . Var (Λ) . C = [\begin{matrix} c_{1} c_{2} \dots c_{i} \dots c_{t} \end{matrix}] . \frac{2}{n} . [\begin{matrix} σ_{1}^{2} & 2 σ_{12} & \dots & 2 σ_{1 t} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ σ_{i 1} & 2 σ_{i 2} & ⋱ & 2 σ_{it} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 2 σ_{1 t} & 2 σ_{2 t} & \dots & σ_{t}^{2} \end{matrix}] [\begin{matrix} c_{1} \\ c_{2} \\ ⋮ \\ c_{i} \\ ⋮ \\ c_{t} \end{matrix}] \\ = \frac{2}{n} . [\sum_{i = 1}^{n} c_{i}^{2} σ_{i}^{2} + 2 \sum_{i < j}^{n} c_{i} c_{j} σ_{ij}] = \frac{2}{n} . σ_{c}^{2}, \\ where σ_{c}^{2} = [\sum_{i = 1}^{n} c_{i}^{2} σ_{i}^{2} + 2 \sum_{i < j}^{n} c_{i} c_{j} σ_{ij}] \end{matrix}

Solving equations Equation 8 and Equation 9 for T we get

(11)

\begin{matrix} T = \frac{ψ_{c}}{\sqrt{C^{'} . Var (Λ) . C}} = \frac{ψ_{c}}{\sqrt{\frac{2}{n} . [\sum_{i = 1}^{n} c_{i}^{2} σ_{i}^{2} + 2 \sum_{i < j}^{n} c_{i} c_{j} σ_{ij}]}} \\ = \sqrt{\frac{n}{2}} (\frac{ψ_{c}}{\sqrt{σ_{c}^{2}}}) \end{matrix}

Now, if we follow similar steps as we did in single time point analysis above, we get the following inequality.

(12)

β \geq Φ (z_{1 - α} - \sqrt{\frac{n}{2}} (\frac{ψ_{c}}{\sqrt{σ_{c}^{2}}})) or z_{β} \geq z_{1 - α} - \sqrt{\frac{n}{2}} (\frac{ψ_{c}}{\sqrt{σ_{c}^{2}}})

And solving Equation 12, we get¹²

(13)

\begin{matrix} n \geq \frac{2 {(z_{α} + z_{β})}^{2} σ_{c}^{2}}{ψ_{c}^{2}} \\ with \\ ψ_{c} = \sum_{i = 1}^{t} c_{i} (μ_{1 i} - μ_{2 i}) and σ_{c}^{2} = \sum_{i = 1}^{t} c_{i}^{2} σ_{i}^{2} + 2 \sum_{i < j}^{t} c_{i} c_{j} σ_{ij} \end{matrix}

$σ_{i}^{2} = common variance in the two groups at timepoint i .$

$σ_{i j} = common covariance in the two groups between timepoint i and j .$

$c_{i} = contrast applied at timepoint i and t represents the number of time points .$

We will use the formulae specified in Equation 13 to calculate the sample size for multiple time point analysis.

Calculation of sample size

Appropriate sample size was calculated for multi-time and single time cases with different scenarios to achieve a difference of 0.9 points at the last time point between two treatment groups with an increasing trend from baseline. The common standard deviation (SD) used was 3.6 points allowing 5% two-sided type I error and 85% power. The effect size and standard deviation used here are based on a real study.¹³ This was a three-year study with primary endpoint assessment at the end of year 3, but the sample size calculation in this study was done based on single time point. Since this study failed to recruit the expected number of patients and had lots of missing data, the characteristics till the second year's assessment were used as it had equal numbers of patients in both arms and stabilized assessments.

Sample size (single time point case)

We considered a two-arm parallel group scenario with one baseline and one post baseline timepoints to assess the change from baseline in absolute scale. Using the formulae in Equation 6 above for single timepoint analysis the sample size required per arm was 287 cases to show statistical significance.

Sample size (longitudinal case)

Here again we considered two arm parallel groups with multiple timepoints and for studying we investigated six cases i.e., three, four, five, six, eight, and 10 timepoints. Each of these cases correspond to several assessments including baseline. Three timepoints corresponded to the case with one baseline and two post baseline assessments, four timepoints corresponded to the case with one baseline and three post baseline assessments, five timepoints corresponded to the case with one baseline and four post baseline assessments and so on.

Figure 1 and Figure 2 represents each of these cases as a line in the plot under different contrast types and correlation structures.

Figure 1. Simulation results with compound symmetry correlation structure.

Figure 2. Simulation results with discrete auto regressive of order 1 correlation structure.

Keeping the SD as 3.6 we tried to vary over two different correlation structures:

Compound symmetry (CS)

Compound Symmetry just means that all the variances are equal and all the covariances are equal. So, the same variance and covariance are used for all subjects. In compound symmetry the covariances across the subjects and the variances (pooled within the group) of the different repeated measures are homogeneous.

[\begin{matrix} σ^{2} & σ^{2} ρ & σ^{2} ρ & \dots & σ^{2} ρ \\ σ^{2} ρ & σ^{2} & σ^{2} ρ & \dots & σ^{2} ρ \\ σ^{2} ρ & σ^{2} ρ & σ^{2} & \dots & σ^{2} ρ \\ ⋮ & ⋱ & ⋮ & ⋮ & ⋮ \\ σ^{2} ρ & σ^{2} ρ & σ^{2} ρ & \dots & σ^{2} \end{matrix}]

Where σ² is the common variance assumed to be similar over time and ρ is the assumed correlation. The order of variance covariance matrix will be $t$ × $t$ , where ‘t’ is the number of time points. Generally, σ² and ρ are estimated from previous clinical trial data with same molecule which could be phase1, phase2 trials for same indication or pivotal trials with same molecule for different indication.

Discrete Auto regressive of order 1(AR1)

This is the homogeneous variance first-order autoregressive structure. Any two elements that are adjacent have a correlation that is equal to rho (ρ), those separated by a third will have correlation ρ², and so on. rho is restricted such that –1< ρ <1.

[\begin{matrix} σ^{2} & σ^{2} ρ & σ^{2} ρ^{2} & \dots & σ^{2} ρ^{t - 1} \\ σ^{2} ρ & σ^{2} & σ^{2} ρ & \dots & σ^{2} ρ^{t - 2} \\ σ^{2} ρ^{2} & σ^{2} ρ & σ^{2} & \dots & σ^{2} ρ^{t - 3} \\ ⋮ & ⋱ & ⋮ & ⋮ & ⋮ \\ σ^{2} ρ^{t - 1} & σ^{2} ρ^{t - 2} & σ^{2} ρ^{t - 3} & \dots & σ^{2} \end{matrix}]

Also, we considered different scenarios of how we want to analyze the results at the end as different contrasts as described below.

Contrast for repeated measures

We investigated two types of contrasts.

1. Time-related contrasts i.e., mean over time (mean contrast).
Rationale: This contrast is more applicable in the situation where the interest lies in observing the mean treatment effect on the subjects disease condition over the period of time the subject is exposed to a prescribed dosing regimen compared to baseline.
This will be labelled in the legend of Figure 1 as CS_mean(i) and in the legend of Figure 2 as AR1_mean(i). For example, for five timepoints the contrast would look like c(-1, ¼, ¼, ¼, ¼).
2. Mean Difference (change at last time point from baseline) (diff contrast).
Rationale: This contrast is more applicable in the situation where the interest lies in observing the treatment effect once the subject is exposed to a prescribed dosing regimen and what is the change at the end of the treatment exposure period in the subjects disease condition. Here the total effect at the end of the treatment course as compared to the baseline is of interest.
This will be labelled in the legend of Figure 1 as CS_diff(i) and in the legend of Figure 2 as AR1_diff(i). For example, for five timepoints the contrast would look like c(-1, 0, 0, 0, 1).

Sample size was calculated for correlation ranging from 0.1 – 0.95 with intervals of 0.05 for both the plots Figure 1 and Figure 2. Sample Size was derived using the formulae mentioned in Equation 13.

CS variance structure with ‘mean over time’ and ‘mean difference’ contrasts

The red horizontal line represents the sample size from the single time point assessment approach.

CS_diff(i), CS_mean(i) – i represents the no. of visits used for sample size calculation, i = 3,4,5,6,8,10.

AR (1) covariance structure with ‘mean over time’ and ‘mean difference’ contrasts

The red horizontal line represents the sample size from the single time point assessment approach.

AR1_diff(i), AR1_mean(i) – i represents the no. of visits used for sample size calculation, i = 3,4,5,6,8,10.

Results

Under CS type of variance covariance structure (Figure 1)

All the trend lines for mean difference type of contrast overlaps each other. For mean difference type of contrast, the sample size does not change for an increase/decrease in the number of visits. It changes with the correlation i.e., highly correlated (rho > 0.5) timepoints would need less sample size as compared to low correlated timepoints. Also, for correlation = 0.5 the multiple assessment sample size coincides with that of single time point assessment.

However, the sample size does vary when the contrast is set to mean over time. Multiple time point assessment with more timepoints requires less sample size as compared to that of multiple time point assessment with less time points for example, the multiple time point assessment with three timepoint requires 86 per arm with correlation 0.8 and the multiple time point assessment with 10 timepoints requires 64 per arm. On the same lines the multiple time point assessment with three timepoints requires 258 per arm with correlation 0.4 and the multiple time point assessment with 10 timepoints requires 192 per arm. This trend shows that the sample size required reduces when the correlation increases.

Under AR(1) type of variance covariance structure (Figure 2)

Under mean over time contrast the multiple time point assessment requires lower sample size as compared to single time point assessment (287 per arm) for correlation greater than 0.35 and the sample size increases as correlation goes below 0.35. For correlation 0.35 the sample size coincides with that of single time point assessment for all the cases except the case of 3 timepoints which requires slightly higher sample number.

Whereas for mean difference contrast the multiple time point assessment requires lower sample size as compared to single time point assessment (287 per arm) for correlation greater than 0.7 but requires higher sample size for correlation less than 0.7.

The trend changes shape for mean difference contrast vs mean over time contrast. Also, at certain point the increase in sample size attenuates for example, in case of mean difference type contrast with 10 time points the sample size required does not changes when correlation drops below 0.55.

Discussion

One of the hurdles in considering the longitudinal methodology for sample size calculation is the assumption on the covariance matrix. It is often easy to estimate the variance of single timepoint as compared to estimating the variance-covariance matrix for multiple time points.

The above derivations were done for trial design with parallel group, 1:1 ratio and two arms. If the ratio changes or if we have more than two arms or if the design is crossover, then the effective overall sample size would get effected in both the cases i.e., sample size with single time point as well as sample size with multiple timepoints, but the trend would remain the same as shown above in the figures and the results will still hold good. Similar trends should hold for other variance – covariance structures though they have not been simulated here.

Conclusion

Sample size changes depending on the analysis type and the data collected. Both the graphs in Figure 1 and Figure 2 in this study reveal that if response is assessed at multiple timepoints and the correlation between the paired observations is high (> 0.6) then one should consider using repeated measures analysis and consequently determine the size of the sample that is based on the multiple time points scenario which results in lower sample size requirement as compared to the sample size derived assuming single timepoint response assessment. This would reduce the cost, resources, and time in conducting the experiment fastening the new drug development. Also, repeated measures analyses will not drop the patients in which they have certain missing data as compared to single point analysis where the patient will be dropped if the response is missing hence may help in retaining the power.

Another thing to notice was that under CS type of covariance structure for the mean over time contrast the sample size required with fewer visit is more as compared to the sample size required with higher number of visits. Whereas for the mean diff contrast the sample size remained same irrespective of the number of visits. On the other hand, under AR(1) type of covariance structure for mean over time contrast the sample size increases as the number of visit increases for ρ > 0.35 but for ρ ≤ 0.35 the sample size decreases as the number of visits increase. Whereas for the mean diff contrast the sample size required with fewer visit is lower as compared to the sample size required with higher number of visits. This change with AR(1) may be due to the fact that the correlation weakens as number of visit increases.

Sample size derivation using longitudinal design method for studies with multiple assessments can be considered of substantial benefit in cost and time although the challenge of estimating the variance-covariance matrix remains. Also, to be noted that the distance between the timepoints is not taken into consideration by the sample size with multiple timepoints derivation method which could be a topic of further research.

Software availability

Software available from: The Comprehensive R Archive Network (https://cran.r-project.org/)

Source code available from: https://github.com/Sarfaraz-Sayyed/Sample-Size-Variation.

Archived source code at time of publication: https://zenodo.org/badge/latestdoi/547747570.¹⁴

Archived source code at time of revision: Sample Size Variation (zenodo.org)

License: MIT License.

Data availability

No data are associated with this article.

Acknowledgement

We would like to thank Novartis Healthcare Pvt. Ltd. and Manipal Academy of Higher Education for their support in carrying out this research and all the reviewers for providing their valuable feedback and suggestions.

References

1. Biemer PP, Lyberg LE: Introduction to Survey quality. Hoboken, New Jersey: John Wiley and Sons; 2003.
2. Nisbet R, Miner G, Yale K: Model Evaluation and Enhancement. Handbook of Statistical Analysis and Data Mining Applications. Elsevier; 2018. Publisher Full Text
3. Khelif A, Saleh MN, Salama A, et al.: Changes in Health-Related Quality of Life with Long-term Eltrombopag Treatment in Adults with Persistent/Chronic Immune Thrombocytopenia: Findings from the EXTEND Study. Am. J. Hematol. 2019; 94: 200–208. PubMed Abstract | Publisher Full Text
4. Palileo-Villanueva LM, Dans AL: Composite endpoints. J. Clin. Epidemiol. 2020; 128: 157–158. Publisher Full Text
5. Bloch DA: Sample size requirements and the cost of a randomized clinical trial with repeated measurements. Stat. Med. 1986; 5(6): 663–667. PubMed Abstract | Publisher Full Text
6. Hedekar D, Gibbons RD, Waternaux C: Sample size estimation for longitudinal designs with attrition: Comparing time-related contrasts between two groups. J. Educ. Behav. Stat. 1999; 24(1): 70–93. Publisher Full Text
7. Basagana X, Liao X, Spiegelman D: Power and sample size calculations for longitudinal studies estimating a main effect of a time-varying exposure. Stat. Methods Med. Res. 2011; 20(5): 471–487. PubMed Abstract | Publisher Full Text
8. Pourhoseingholi MA, Vahedi M, Rahimzadeh M: Gastroenterol Hepatol Bed Bench. Winter. Sample size calculation in medical studies. 2013; 6(1): 14–17.
9. Hajian-Tilaki K: Sample size estimation in epidemiologic studies. Caspian J. Intern. Med. 2011; 2(4): 289–298. PubMed Abstract
10. Veena Manja MD, Satyan Lakshminrusimha MD: Epidemiology and Clinical Research Design, Part 1: Principles. NeoReviews. 2015; 16(2): e94–e108. PubMed Abstract | Publisher Full Text
11. Manja V MD, Lakshminrusimha S MD: Epidemiology and Clinical Research Design, Part 1: Study Types. Neoreviews. NeoReviews. 2014; 15(12): e558–e569. PubMed Abstract | Publisher Full Text
12. Diggle PJ, Heagerty P, Liang K-Y: Analysis of Longitudinal Data. Second EditionNew York: Oxford Statistical Science Series; 2002.
13. Black DM, Reid IR, Cauley JA, et al.: The Effect of 6 Versus 9 Years of Zoledronic AcidTreatment in Osteoporosis: A Randomized SecondExtension to the HORIZON-Pivotal Fracture Trial (PFT). J. Bone Miner. Res. 2014; 30(5): 934–944. Publisher Full Text
14. Sarfaraz-Sayyed: Sarfaraz-Sayyed/Sample-Size-Variation: Sample Size variation (V1.0.0). Zenodo. Software.2022. Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 21 Dec 2022

Author details Author details

Sarfaraz Sayyed
Roles: Formal Analysis, Investigation, Visualization, Writing – Original Draft Preparation

Ashwini Mathur
Roles: Conceptualization, Supervision, Writing – Review & Editing

Asha Kamath
Roles: Conceptualization, Supervision, Writing – Review & Editing

Competing interests

The author (Sarfaraz Sayyed) and the co-author (Ashwini Mathur) are employed by Novartis Healthcare Pvt. Ltd.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (2)

version 2

Revised

Published: 05 Sep 2024, 11:1550

https://doi.org/10.12688/f1000research.124917.2

version 1

Published: 21 Dec 2022, 11:1550

https://doi.org/10.12688/f1000research.124917.1

© 2024 Sayyed S et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Sayyed S, Mathur A and Kamath A. Sample size variation in single-time post-dose assessment vs multi-time post-dose assessment [version 2; peer review: 1 approved, 2 not approved]. F1000Research 2024, 11:1550 (https://doi.org/10.12688/f1000research.124917.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 2

VERSION 2

PUBLISHED 05 Sep 2024

Revised

Views

Reviewer Report 21 Nov 2024

Simon Vandekar, Vanderbilt University Medical Center, Tennessee, USA

Not Approved

https://doi.org/10.5256/f1000research.170509.r333804

The paper doesn’t reference the huge body of literature on power analysis in longitudinal studies. There are 14 references and only of them is a classical text on longitudinal data analysis (Diggle et al). For example, the book by Ahn, Heo, Zhang “Sample Size Calculations for Clustered and Longitudinal Outcomes in Clinical Research.” There are numerous R package for longitudinal power analyses, but these were not discussed (e.g. https://journal.r-project.org/articles/RJ-2022-022/). The results and topic are not novel and overlook critical references that have studied the topic. Given the very general topic, I would have expected more references to the existing literature.

There need to be more references for the mathematics in the paper, such as “Method for Sample size with single time point assessment analysis.” Grammar in the paper needs to be reviewed. There are some notation errors, e.g. in equations (8) and (9) Var(\Gamma) doesn’t make sense because Gamma is defined as a parameter (so it has zero variance).

Is the rationale for developing the new method (or application) clearly explained?

Yes
Is the description of the method technically sound?

Partly
Are sufficient details provided to allow replication of the method development and its use by others?

Yes
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

References

1. Ahn C, Heo M, Zhang S: Sample Size Calculations for Clustered and Longitudinal Outcomes in Clinical Research. 2014. Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Biostatistics

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

CITE

Report a concern

Respond or Comment

Views

Reviewer Report 23 Oct 2024

Kiranmoy Das, Indian Statistical Institute, Kolkata, West Bengal, India

Approved

https://doi.org/10.5256/f1000research.170509.r320982

I have seen the revised version of the paper and the author's comments.
I ... Continue reading

CITE

Report a concern

Respond or Comment

Views

Reviewer Report 28 Sep 2024

Ronald Geskus, Oxford University, Oxford, England, UK

Not Approved

https://doi.org/10.5256/f1000research.170509.r320983

The article still contains several typos, sentences that abruptly end and capital letters that are misplaced. I suggest the authors do a careful check of typos. Also, I still found a couple of errors, as well as a few statements that are not clear to me.

1. "It affects the estimator’s precision, which is a metric that is important for the chosen target population of all professional athletes". Precision is a characteristic of the data. Explain why precision is an important metric for the target population.

2. The authors found only two papers (references 3 and 4) in their literature review on longitudinal studies? There must be many more.

3. Spiegelma -> Spiegelman

4. Page 4: Make clear that the normal distribution of the sample mean is an approximation, which becomes more accurate with increasing sample size (based on the central limit theorem)

5. The authors replied: "Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion." I doubt whether "H_a is true" and formula (2) avoid confusion. To my opinion, it is clearer to change "H_a is true" and "\delta > 0" by a specific value for \delta.

6. Page 5: H_a -> H_0 for null hypothesis, and \psi_c=C^t \Lambda, with "t" meaning transpose.

7. Formula (8) is not a test statistic, \psi_c is a property of the population not based on random variables. Also, \Lambda is a population property, not determined by random variables.

8. covar(Y_2-X_2,Y_1-X_1)=covar(Y_2,Y_1)+covar(X_2,X_1). I don't see where the value 4 comes from in (9).

9. I still find the section "Calculation of sample size" not very clearly phrased. Specify what is meant with "the last time point" (later it is written to be 3 years). What time trend does it imply? Replace "allowing" by "choosing"

10. Explain how the sample size of 287 is obtained. How large is the assumed difference under the null hypothesis, i.e. what is the chosen time point?

11. The rationale for the mean over time contrast chosen is still not clear to me. Why would one choose 1/4 at later time points, given that a linear trend is assumed?

12. Can the authors explain i) why the sample size for correlation=0.5 (compound symmetry) coincides with single time point assessment and ii) why the required sample size is larger if correlation<0.5?

13. I do not understand why with low correlation the single time point assessment performs best. Please explain.

14. I do not understand why the sample size for AR(1) mean over time contrast increases with increasing number of time points included if correlation > 0.35. Please explain.

15. Page 11: "Whereas for the mean diff contrast the sample size required with fewer visit is lower as compared to the sample size required with higher number of visits". It's not the number of visits that matters (it is always based on two time points only), but the distance between the visits. The required sample size for AR(1) mean difference contrast increases if the second time point is further away from the baseline. I don't think this is a general property. It will depend on the combination of serial correlation and difference in mean at the chosen time point. This needs further investigation.

16. The authors give a detailed introduction for readers who do not have a statistical background. It would be very useful to provide some guidance how the results can directly be used by them.

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: biostatistics

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 21 Dec 2022

Views

Reviewer Report 10 Jul 2024

Ronald Geskus, Oxford University, Oxford, England, UK

Not Approved

https://doi.org/10.5256/f1000research.137160.r294678

The authors describe some formulas for sample size calculations with longitudinal data with a continuous outcome. The authors only consider one specific approach, namely generalized least squares and comparing changes from baseline. This latter approach is discouraged, see e.g. https://www.bmj.com/content/323/7321/1123.

More general methods to perform sample size and power calculations with longitudinal data have been described and implemented in software (e.g. https://journal.r-project.org/articles/RJ-2022-022/).

Also, there are some descriptions that need further attention.

1. The introduction describes the basic principles of hypothesis testing. This can be assumed known and therefore I suggest to leave it out of the paper.

2. Page 3: The authors describe sampling error as the uncertainty arising from: "the limitation to examine every situation, the absolute precision of the effect of protein shake on the athlete's performance would be hard to measure." I am not sure whether I understand what is written here, but I think it is not correct.

3. Page 3: Sampling error decreases with increasing sample size, but it is not true that " there comes a point where increasing the sample size has no further effect on the sampling error".

4. Formula (2): power is typically calculated for a specific choice of the effect size, not for all delta>0 at once.

5. Page 7: "Appropriate sample size was calculated for multi-time and single time cases with different scenarios to achieve an overall mean treatment difference (0.9 points)." What is meant with mean difference in the situation with more than 2 time points? The difference could vary per time point but still give a mean difference of 0.9.
The last sentence of that paragraph is irrelevant information.

6. How was \sigma_{ij} chosen?

7. Page 8: make more explicit that a discrete AR(1) is assumed. There also exitsts a continuous AR(1)

8. Page 9: The authors write: "For correlation = 0.1 to 0.35 the sample size coincides with that of single time point assessment." I don't see this in the figure

9. Page 9: I don't understand the rationale "effect remains only for some time and then the disease condition reverses back' and "each dose reduces the disease severity and the over the course it is totally removed from the patient’s body" for the two contrasts.

10. Explain why the sample size goes down to zero if the correlation approaches 1. A correlation of 1 does not make sense. It implies that the second measurement is completely determined by the first, and the intervention cannot have any impact.

11. Legends Fig 1 and 2 have been mixed up.

Is the rationale for developing the new method (or application) clearly explained?

No
Is the description of the method technically sound?

Yes
Are sufficient details provided to allow replication of the method development and its use by others?

Yes
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Biostatistics

CITE

Report a concern

Author Response 05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

05 Sep 2024

Author Response

Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to ... Continue reading Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to propose a framework, but we have modified it now to make it more precise.

- Regarding methods to perform sample size calculation, we agree with you, but our intention here is to highlight using simple methods to the clinical researchers, the difference in samples required when we have longitudinal information available under 2 different correlation structures.

Comment 1: We kept this as this would be helpful to new researchers not having statistical background.
Comment 2: We have modified it for better understanding.
Comment 3: We have modified and added a note to have more clarity.
Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion.
Comment 5: We have modified the sentence for better clarity and also removed the last sentence of the paragraph.
Comment 6: Sigma was taken from a trial data (reference given) and was assumed to be same over all the time periods.
Comment 7: Thanks for bringing this up, we have updated this in the manuscript now.
Comment 8: This was wrongly written under figure 1, it was supposed to be under figure 2 conclusion. we have made the necessary changes now.
Comment 9: The first rationale refers to chronic diseases where the disease can usually be controlled but not cured. So here we are trying to see that the average effect from the treatment is the one which is expected to control the disease. Have changed the wordings for better understanding. The second rationale is for infectious diseases where with the right treatment regimen the disease will be cured. Have now tried to put in better words.
Comment 10: Agree with your comment on both the extreme cases that a correlation of 1 would imply all the effect is due to intervention not leaving any space for error and also that correlation 0 would mean no effect from the intervention. We have now revised the plots to start from 0.05 till 0.9
Comment 11: Have made the necessary changes.

- Regarding clarifying the rationale, basically in clinical trials the use of all time point data is not generally used. The effect at intermittent time points and the covariance structure between time points would be helpful which otherwise not taken into consideration results in large sample size.

- Regarding conclusion on the methods, we have updated the conclusion to have more insights, how taking into correlation would benefit and also how the effects diminish even if too many timepoints are added.
Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to propose a framework, but we have modified it now to make it more precise.

- Regarding methods to perform sample size calculation, we agree with you, but our intention here is to highlight using simple methods to the clinical researchers, the difference in samples required when we have longitudinal information available under 2 different correlation structures.

Comment 1: We kept this as this would be helpful to new researchers not having statistical background.
Comment 2: We have modified it for better understanding.
Comment 3: We have modified and added a note to have more clarity.
Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion.
Comment 5: We have modified the sentence for better clarity and also removed the last sentence of the paragraph.
Comment 6: Sigma was taken from a trial data (reference given) and was assumed to be same over all the time periods.
Comment 7: Thanks for bringing this up, we have updated this in the manuscript now.
Comment 8: This was wrongly written under figure 1, it was supposed to be under figure 2 conclusion. we have made the necessary changes now.
Comment 9: The first rationale refers to chronic diseases where the disease can usually be controlled but not cured. So here we are trying to see that the average effect from the treatment is the one which is expected to control the disease. Have changed the wordings for better understanding. The second rationale is for infectious diseases where with the right treatment regimen the disease will be cured. Have now tried to put in better words.
Comment 10: Agree with your comment on both the extreme cases that a correlation of 1 would imply all the effect is due to intervention not leaving any space for error and also that correlation 0 would mean no effect from the intervention. We have now revised the plots to start from 0.05 till 0.9
Comment 11: Have made the necessary changes.

- Regarding clarifying the rationale, basically in clinical trials the use of all time point data is not generally used. The effect at intermittent time points and the covariance structure between time points would be helpful which otherwise not taken into consideration results in large sample size.

- Regarding conclusion on the methods, we have updated the conclusion to have more insights, how taking into correlation would benefit and also how the effects diminish even if too many timepoints are added.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

05 Sep 2024

Author Response

Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to ... Continue reading Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to propose a framework, but we have modified it now to make it more precise.

- Regarding methods to perform sample size calculation, we agree with you, but our intention here is to highlight using simple methods to the clinical researchers, the difference in samples required when we have longitudinal information available under 2 different correlation structures.

Comment 1: We kept this as this would be helpful to new researchers not having statistical background.
Comment 2: We have modified it for better understanding.
Comment 3: We have modified and added a note to have more clarity.
Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion.
Comment 5: We have modified the sentence for better clarity and also removed the last sentence of the paragraph.
Comment 6: Sigma was taken from a trial data (reference given) and was assumed to be same over all the time periods.
Comment 7: Thanks for bringing this up, we have updated this in the manuscript now.
Comment 8: This was wrongly written under figure 1, it was supposed to be under figure 2 conclusion. we have made the necessary changes now.
Comment 9: The first rationale refers to chronic diseases where the disease can usually be controlled but not cured. So here we are trying to see that the average effect from the treatment is the one which is expected to control the disease. Have changed the wordings for better understanding. The second rationale is for infectious diseases where with the right treatment regimen the disease will be cured. Have now tried to put in better words.
Comment 10: Agree with your comment on both the extreme cases that a correlation of 1 would imply all the effect is due to intervention not leaving any space for error and also that correlation 0 would mean no effect from the intervention. We have now revised the plots to start from 0.05 till 0.9
Comment 11: Have made the necessary changes.

- Regarding clarifying the rationale, basically in clinical trials the use of all time point data is not generally used. The effect at intermittent time points and the covariance structure between time points would be helpful which otherwise not taken into consideration results in large sample size.

- Regarding conclusion on the methods, we have updated the conclusion to have more insights, how taking into correlation would benefit and also how the effects diminish even if too many timepoints are added.
Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to propose a framework, but we have modified it now to make it more precise.

- Regarding methods to perform sample size calculation, we agree with you, but our intention here is to highlight using simple methods to the clinical researchers, the difference in samples required when we have longitudinal information available under 2 different correlation structures.

Comment 1: We kept this as this would be helpful to new researchers not having statistical background.
Comment 2: We have modified it for better understanding.
Comment 3: We have modified and added a note to have more clarity.
Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion.
Comment 5: We have modified the sentence for better clarity and also removed the last sentence of the paragraph.
Comment 6: Sigma was taken from a trial data (reference given) and was assumed to be same over all the time periods.
Comment 7: Thanks for bringing this up, we have updated this in the manuscript now.
Comment 8: This was wrongly written under figure 1, it was supposed to be under figure 2 conclusion. we have made the necessary changes now.
Comment 9: The first rationale refers to chronic diseases where the disease can usually be controlled but not cured. So here we are trying to see that the average effect from the treatment is the one which is expected to control the disease. Have changed the wordings for better understanding. The second rationale is for infectious diseases where with the right treatment regimen the disease will be cured. Have now tried to put in better words.
Comment 10: Agree with your comment on both the extreme cases that a correlation of 1 would imply all the effect is due to intervention not leaving any space for error and also that correlation 0 would mean no effect from the intervention. We have now revised the plots to start from 0.05 till 0.9
Comment 11: Have made the necessary changes.

- Regarding clarifying the rationale, basically in clinical trials the use of all time point data is not generally used. The effect at intermittent time points and the covariance structure between time points would be helpful which otherwise not taken into consideration results in large sample size.

- Regarding conclusion on the methods, we have updated the conclusion to have more insights, how taking into correlation would benefit and also how the effects diminish even if too many timepoints are added.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 30 May 2024

Kiranmoy Das, Indian Statistical Institute, Kolkata, West Bengal, India

Not Approved

https://doi.org/10.5256/f1000research.137160.r273771

Reviewer’s comments on the manuscript “Sample-size variation in single-time post-dose assessment vs multi-time post-dose assessment”

The manuscript attempts to address a very important and interesting problem. However, the presentation is not quite clear, there are lots of typos and grammar issues here and there. The methodology developed in the manuscript can be improved. My comments are given below. After addressing these comments and after a substantial revision, I can make some positive comment on the manuscript.
1. Introduction: Please provide some good motivation and nice description of the problem. There are several typos. Please avoid the phrase For e.g., this should be For example, and also “The size of the sample and the effect size are major factors which affects”, this should be like “The sample size and effect size are major factors that affect”.
2. Methods: Equation (1), you are assuming here that the variance sigma^2 is known. Please specify it, and also how appropriate this assumption is in real application?
3. For the longitudinal studies, your method depends on the auto-correlation \rho. You need to estimate \rho before calculating the sample size. Please clarify that and also discuss how to estimate \rho. Maybe you run a pilot trial and get an estimate?
4. Both the figures 1 and 2 focus on how the sample size varies with correlation \rho. For a longitudinal trial, it maybe more important to know how it covaries with the number of time points. Correlation is something that cannot be adjusted, but the number of time points can be adjusted as per the requirement. I think this issue has been ignored in the manuscript.

Is the rationale for developing the new method (or application) clearly explained?

No
Is the description of the method technically sound?

Yes
Are sufficient details provided to allow replication of the method development and its use by others?

No
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Biostatistics, Longitudinal data analysis, Bayesian modeling

CITE

Report a concern

Author Response 05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

05 Sep 2024

Author Response

Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. ... Continue reading Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. 1.
Comment 3: An explanation of rho estimation is now added.
Comment 4: We have added a paragraph under the conclusion section to address this.

We have added a few lines in the Introduction section to clarify the rationale.
Regarding details for replication of the method, details on assumptions are specified in the section “Calculation of sample size” in the manuscript. The software availability section provides the link to the related R program residing in Github.
Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. 1.
Comment 3: An explanation of rho estimation is now added.
Comment 4: We have added a paragraph under the conclusion section to address this.

We have added a few lines in the Introduction section to clarify the rationale.
Regarding details for replication of the method, details on assumptions are specified in the section “Calculation of sample size” in the manuscript. The software availability section provides the link to the related R program residing in Github.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

05 Sep 2024

Author Response

Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. ... Continue reading Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. 1.
Comment 3: An explanation of rho estimation is now added.
Comment 4: We have added a paragraph under the conclusion section to address this.

We have added a few lines in the Introduction section to clarify the rationale.
Regarding details for replication of the method, details on assumptions are specified in the section “Calculation of sample size” in the manuscript. The software availability section provides the link to the related R program residing in Github.
Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. 1.
Comment 3: An explanation of rho estimation is now added.
Comment 4: We have added a paragraph under the conclusion section to address this.

We have added a few lines in the Introduction section to clarify the rationale.
Regarding details for replication of the method, details on assumptions are specified in the section “Calculation of sample size” in the manuscript. The software availability section provides the link to the related R program residing in Github.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 21 Dec 2022

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2	3
Version 2 (revision) 05 Sep 24	read	read	read
Version 1 21 Dec 22	read	read

Kiranmoy Das, Indian Statistical Institute, Kolkata, India
Ronald Geskus, Oxford University, Oxford, UK
Simon Vandekar, Vanderbilt University Medical Center, Tennessee, USA

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

3 Views

21 Nov 2024 | for Version 2

Simon Vandekar, Vanderbilt University Medical Center, Tennessee, USA

3 Views Cite this report Responses(0)

Not Approved

Is the rationale for developing the new method (or application) clearly explained?

Yes
Is the description of the method technically sound?

Partly
Are sufficient details provided to allow replication of the method development and its use by others?

Yes
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

References

1. Ahn C, Heo M, Zhang S: Sample Size Calculations for Clustered and Longitudinal Outcomes in Clinical Research. 2014. Publisher Full Text

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Biostatistics

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

4 Views

23 Oct 2024 | for Version 2

Kiranmoy Das, Indian Statistical Institute, Kolkata, West Bengal, India

4 Views Cite this report Responses(0)

Approved

I have seen the revised version of the paper and the author's comments.
I am quite satisfied and I have no additional comments.
I recommend the acceptance of the manuscript

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Biostatistics, Longitudinal data analysis, Bayesian modeling

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

10 Views

28 Sep 2024 | for Version 2

Ronald Geskus, Oxford University, Oxford, England, UK

10 Views Cite this report Responses(0)

Not Approved

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

biostatistics

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

15 Views

10 Jul 2024 | for Version 1

Ronald Geskus, Oxford University, Oxford, England, UK

15 Views Cite this report Responses(1)

Not Approved

Is the rationale for developing the new method (or application) clearly explained?

No
Is the description of the method technically sound?

Yes
Are sufficient details provided to allow replication of the method development and its use by others?

Yes
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Biostatistics

Respond to this report

Responses (1)

Author Response

05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

Thanks Dr. Ronald Geskus for your review and advice.
Please find below our responses.
- Regarding analyzing change from baseline, we completely agree with you. Our intention was just to propose a framework, but we have modified it now to make it more precise.

- Regarding methods to perform sample size calculation, we agree with you, but our intention here is to highlight using simple methods to the clinical researchers, the difference in samples required when we have longitudinal information available under 2 different correlation structures.

Comment 1: We kept this as this would be helpful to new researchers not having statistical background.
Comment 2: We have modified it for better understanding.
Comment 3: We have modified and added a note to have more clarity.
Comment 4: We fully agree with you, this was a generic equation to show we are looking positive change in response and then further with the examples we have assumed for certain fixed delta how the sample size varies. But have modified it to avoid confusion.
Comment 5: We have modified the sentence for better clarity and also removed the last sentence of the paragraph.
Comment 6: Sigma was taken from a trial data (reference given) and was assumed to be same over all the time periods.
Comment 7: Thanks for bringing this up, we have updated this in the manuscript now.
Comment 8: This was wrongly written under figure 1, it was supposed to be under figure 2 conclusion. we have made the necessary changes now.
Comment 9: The first rationale refers to chronic diseases where the disease can usually be controlled but not cured. So here we are trying to see that the average effect from the treatment is the one which is expected to control the disease. Have changed the wordings for better understanding. The second rationale is for infectious diseases where with the right treatment regimen the disease will be cured. Have now tried to put in better words.
Comment 10: Agree with your comment on both the extreme cases that a correlation of 1 would imply all the effect is due to intervention not leaving any space for error and also that correlation 0 would mean no effect from the intervention. We have now revised the plots to start from 0.05 till 0.9
Comment 11: Have made the necessary changes.

- Regarding clarifying the rationale, basically in clinical trials the use of all time point data is not generally used. The effect at intermittent time points and the covariance structure between time points would be helpful which otherwise not taken into consideration results in large sample size.

- Regarding conclusion on the methods, we have updated the conclusion to have more insights, how taking into correlation would benefit and also how the effects diminish even if too many timepoints are added.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

18 Views

30 May 2024 | for Version 1

Kiranmoy Das, Indian Statistical Institute, Kolkata, West Bengal, India

18 Views Cite this report Responses(1)

Not Approved

Is the rationale for developing the new method (or application) clearly explained?

No
Is the description of the method technically sound?

Yes
Are sufficient details provided to allow replication of the method development and its use by others?

No
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Biostatistics, Longitudinal data analysis, Bayesian modeling

Respond to this report

Responses (1)

Author Response

05 Sep 2024

Sarfaraz Sayyed, Analytics, Novartis Healthcare Pvt. Ltd., Hyderabad, 500080, India

Thanks Dr. Kiranmoy Das for your review and advice.
Please see our response below.
Comment 1: We have updated the introduction section.
Comment 2: Justification now added above equation no. 1.
Comment 3: An explanation of rho estimation is now added.
Comment 4: We have added a paragraph under the conclusion section to address this.

We have added a few lines in the Introduction section to clarify the rationale.
Regarding details for replication of the method, details on assumptions are specified in the section “Calculation of sample size” in the manuscript. The software availability section provides the link to the related R program residing in Github.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Biemer PP, Lyberg LE: Introduction to Survey quality. Hoboken, New Jersey: John Wiley and Sons; 2003.

[2] 2. Nisbet R, Miner G, Yale K: Model Evaluation and Enhancement. Handbook of Statistical Analysis and Data Mining Applications. Elsevier; 2018. Publisher Full Text

[3] 3. Khelif A, Saleh MN, Salama A, et al.: Changes in Health-Related Quality of Life with Long-term Eltrombopag Treatment in Adults with Persistent/Chronic Immune Thrombocytopenia: Findings from the EXTEND Study. Am. J. Hematol. 2019; 94: 200–208. PubMed Abstract | Publisher Full Text

[4] 4. Palileo-Villanueva LM, Dans AL: Composite endpoints. J. Clin. Epidemiol. 2020; 128: 157–158. Publisher Full Text

[5] 5. Bloch DA: Sample size requirements and the cost of a randomized clinical trial with repeated measurements. Stat. Med. 1986; 5(6): 663–667. PubMed Abstract | Publisher Full Text

[6] 6. Hedekar D, Gibbons RD, Waternaux C: Sample size estimation for longitudinal designs with attrition: Comparing time-related contrasts between two groups. J. Educ. Behav. Stat. 1999; 24(1): 70–93. Publisher Full Text

[7] 7. Basagana X, Liao X, Spiegelman D: Power and sample size calculations for longitudinal studies estimating a main effect of a time-varying exposure. Stat. Methods Med. Res. 2011; 20(5): 471–487. PubMed Abstract | Publisher Full Text

[8] 8. Pourhoseingholi MA, Vahedi M, Rahimzadeh M: Gastroenterol Hepatol Bed Bench. Winter. Sample size calculation in medical studies. 2013; 6(1): 14–17.

[9] 9. Hajian-Tilaki K: Sample size estimation in epidemiologic studies. Caspian J. Intern. Med. 2011; 2(4): 289–298. PubMed Abstract

[10] 10. Veena Manja MD, Satyan Lakshminrusimha MD: Epidemiology and Clinical Research Design, Part 1: Principles. NeoReviews. 2015; 16(2): e94–e108. PubMed Abstract | Publisher Full Text

[11] 11. Manja V MD, Lakshminrusimha S MD: Epidemiology and Clinical Research Design, Part 1: Study Types. Neoreviews. NeoReviews. 2014; 15(12): e558–e569. PubMed Abstract | Publisher Full Text

[12] 12. Diggle PJ, Heagerty P, Liang K-Y: Analysis of Longitudinal Data. Second EditionNew York: Oxford Statistical Science Series; 2002.

[13] 13. Black DM, Reid IR, Cauley JA, et al.: The Effect of 6 Versus 9 Years of Zoledronic AcidTreatment in Osteoporosis: A Randomized SecondExtension to the HORIZON-Pivotal Fracture Trial (PFT). J. Bone Miner. Res. 2014; 30(5): 934–944. Publisher Full Text

[14] 14. Sarfaraz-Sayyed: Sarfaraz-Sayyed/Sample-Size-Variation: Sample Size variation (V1.0.0). Zenodo. Software.2022. Publisher Full Text

Sample size variation in single-time post-dose assessment vs multi-time post-dose assessment

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Background

Objective

Notation and framework

Methods

Method for Sample size with single time point assessment analysis

(1)

(2)

(3)

(4)

(5)

(6)

Method for Sample size with multiple time point assessment analysis

(7)

(8)

(9)

(10)

(11)

(12)

(13)

Calculation of sample size

Sample size (single time point case)

Sample size (longitudinal case)

Figure 1. Simulation results with compound symmetry correlation structure.

Figure 2. Simulation results with discrete auto regressive of order 1 correlation structure.

Contrast for repeated measures

CS variance structure with ‘mean over time’ and ‘mean difference’ contrasts

AR (1) covariance structure with ‘mean over time’ and ‘mean difference’ contrasts

Results

Under CS type of variance covariance structure (Figure 1)

Under AR(1) type of variance covariance structure (Figure 2)

Discussion

Conclusion

Software availability

Data availability

Acknowledgement

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated