Cystic fibrosis (CF) is a multi-system genetic condition but the two main affected organs are lungs (resulting in recurrent infections and respiratory failure) and gastrointestinal tract (resulting in fat malabsorption and poor growth) ¹ . Median survival has improved to 45 years, in part because of improvement in care quality ² . An important quality improvement initiative is benchmarking, which involves identifying high-performing centres and the practices associated with outstanding performance ^{3– 5} . Since forced expiratory volume in one second (FEV ₁) is an important CF prognostic marker ^{6– 9} , it is often used as an outcome measure for benchmarking ^{3– 5, 10} .

Different statistical methods of analysing FEV ₁ data can yield different results ¹¹ , but there is scant attention paid to the methods of processing FEV ₁ data. We previously reported a statistically significant reduction in year-to-year %FEV ₁ fall for our CF centre from 2013–2016 ¹² . We now set out to understand the impact of using different FEV ₁ data processing methods on our CF centre’s outcome.

This is a single-centre retrospective analysis of routinely collected clinical data from 2013–2016. Regulatory approval for the analysis was obtained from NHS Health Research Authority (IRAS number 210313). All adults with CF diagnosed according to the UK CF Trust criteria aged ≥16 years were included, except those with lung transplantation or on ivacaftor. These treatments have transformative effects on %FEV ₁ ^{13– 15} , thus may affect the interpretation of year-to-year variation in %FEV ₁.

Demographic data (age, gender, genotype, pancreatic status, CF related diabetes, Pseudomonas aeruginosa status), body mass index (BMI) and FEV ₁ data were collected by two investigators (HZH and RC / HZH and MEG) independently reviewing paper notes and electronic records. Where data from the two investigators differ, the original data from paper notes or electronic records were reviewed to by both investigators to ensure the accuracy of abstracted data. This process ensures the accuracy of abstracted data and helps avoid potential bias from inaccurate or inconsistent data collection ¹⁶ . FEV ₁ data were processed with three different methods prior to analysis. First, %FEV ₁ readings (calculated with Knudson equation ¹⁷ and available in whole numbers) were directly extracted from spirometer machines. Second, FEV ₁ volumes (in litres, to two decimal places) were extracted and clean height data were used to calculate %FEV ₁ (as whole numbers) with Knudson equation ¹⁷ . Third, FEV ₁ volumes (in litres, to two decimal places) were extracted and clean height data were used to calculate %FEV ₁ with GLI equation ¹⁸ using an Excel Macro (Microsoft Excel 2013).

Best %FEV ₁, i.e. the highest %FEV ₁ reading in a calendar year for each study subject was used for analysis since it is most reflective of the true baseline %FEV ₁ ¹⁹ . Year-to-year %FEV ₁ change was calculated by subtracting best %FEV ₁ at Year 1 from Year 2 (i.e. negative values indicate fall in %FEV ₁ and positive values indicate increase in %FEV ₁). In addition to calculating year-to-year %FEV ₁ change using three different FEV ₁ data processing methods, %FEV ₁ change calculated with GLI equation was also adjusted for baseline %FEV ₁ using reference values from Epidemiologic Study of CF (ESCF) ²⁰ . The ESCF study found median %FEV ₁ change of –3%/year, –2%/year and –0.5%/year for baseline %FEV1 ≥100%, 40–99.9% and <40% respectively ²⁰ . Adjusted %FEV ₁ change was calculated by subtracting median ESCF %FEV ₁ change from actual %FEV ₁ change. Thus, an adjusted %FEV ₁ change >0 meant the subject’s year-to-year change in %FEV1 was less than expected (indicating better health outcome) whilst an adjusted %FEV ₁ change <0 meant the subject’s year-to-year change in %FEV ₁ was more than expected (indicating worse health outcome).

%FEV ₁ change from 2013–2014 to 2015–2016 calculated using different FEV ₁ data processing methods were compared using Friedman test. Bland-Altman analyses ²¹ were also used to compare year-to-year variation in FEV ₁ as calculated with Knudson equation against year-to-year variation in FEV ₁ as calculated with GLI equation, to understand the impact of using different reference equations. Analyses were performed using SPSS v24 (IBM Corp) and Prism v7 (GraphPad Software). P-value <0.05 was considered statistically significant.

This analysis included 208 adults, with 147 adults providing data for all four years. Overall, the cohort was ageing but baseline %FEV ₁ increased from 2014 onwards (see Table 1).

The %FEV ₁ increase was in part due to younger adults with higher %FEV ₁ transitioning from paediatric care because %FEV ₁ tended to decline from year to year (see Table 2). However, different year-to-year change in %FEV ₁ results were obtained with different FEV ₁ data processing methods. There was statistically significant reduction in year-to-year fall in %FEV ₁ using %FEV ₁ readings as recorded in spirometer machines ( p=0.016). Cleaning of height data and standardisation of %FEV ₁ calculation with Knudson equation ¹⁷ did not alter the magnitude of year-to-year variation in %FEV ₁, but the p-value was no longer statistically significant ( p=0.062). The use of GLI equation altered the magnitude of year-to-year variation in %FEV ₁ although the trend of reduced year-to-year fall in %FEV ₁ persisted ( p=0.135). Adjustment for baseline %FEV ₁ further increased the p-value ( p=0.210).

Similar results were obtained when restricting the analyses to those aged ≥18 years (see Table 3). Bland-Altman analyses comparing year-to-year variation in %FEV ₁ calculated from clean FEV ₁ data using Knudson equation ¹⁷ vs year-to-year variation in %FEV ₁ calculated from clean FEV ₁ data using GLI equation ¹⁸ indicate the tendency for Knudson equation ¹⁷ to over-estimate the magnitude of year-to-year fall in %FEV ₁ by a mean difference of 0.1–0.4% (see Figure 1).

We demonstrated that different centre-level year-to-year variation in %FEV ₁ results were obtained using different FEV ₁ data processing methods. In particular, year-to-year fall in %FEV ₁ was smaller in magnitude when %FEV ₁ was calculated using GLI equation ¹⁸ instead of Knudson equation ¹⁷ . This is in part due to the demographic of our centre which has a relatively young adult population. A previous study found a near-linear %FEV ₁ decline from childhood to adulthood with GLI equation, whereas there was accelerated %FEV ₁ decline during adolescence and young adulthood when %FEV ₁ was calculated with Knudson equation ²⁴ . One advantage of using the GLI equation, which is seamless across all ages, is that it improves the interpretation of %FEV ₁ decline ^{24, 25} . Another advantage is that %FEV ₁ decline can be adjusted for baseline %FEV ₁ using ESCF reference values (since the ESCF values for %FEV ₁ decline were calculated using the GLI equation ²⁰ ).

The limitation for all single-centre analysis is the potential lack of generalisability. Another limitation of our analysis is that the ESCF reference values used to adjust year-to-year variation in %FEV ₁ were derived using a cohort from around 15 years ago ²⁰ , and may not represent the current population. Our results nonetheless highlighted that year-to-year variation in %FEV ₁ can be extremely sensitive to the FEV ₁ data processing methods. This is one of the challenges of using year-to-year variation in %FEV ₁ to infer quality of care. Another challenge is that %FEV ₁ lacks sensitivity as an outcome measure. A recent sample size estimation using the UK CF registry data suggests that 273 adults per centre are needed to detect a 5% FEV ₁ difference at the 95% significance level ²⁶ . The sensitivity of measures used to detect variations in care quality is particularly pertinent to CF because a relatively small population is spread across many centres. Indeed, only 6/28 (21.4%) of all UK adult CF centres have ≥273 adults. That means process measures, e.g. medication adherence, is important to detect variations in quality of CF care. Mant & Hicks previous demonstrated that measuring processes of care proven in randomised controlled trials to reduce death allows detection of meaningful differences in care quality for myocardial infarction with just 75 cases, whereas 8179 cases would be needed if mortality was used as the quality indicator ²⁷ .

Given the limitations of FEV ₁ as an outcome measure in CF, results of centre comparisons based on FEV ₁ data should be carefully interpreted. Observational studies with year-to-year variation in %FEV ₁ as an outcome measure should carefully consider and clearly specify the data processing methods used.

Regulatory approval for the analysis was obtained from NHS Health Research Authority (IRAS number 210313).

The data referenced by this article are under copyright with the following copyright statement: Copyright: � 2018 Hoo ZH et al.

Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Dataset 1: Sheffield forced expiratory volume in one second (FEV ₁) data 10.5256/f1000research.14981.d205603 ²⁸