maxRatio improves the detection of samples with abnormal amplification profiles on QIAgen’s artus HIV-1 qPCR assay

Background: Accurate viral load (VL) determination is paramount to determine the efficacy of anti-HIV-1 therapy. The conventional method used, fit-point (FP), assumes an equal efficiency in the polymerase chain reaction (PCR) among samples that might not hold for low-input templates. An alternative approach, maxRatio, was introduced to compensate for inhibition in PCR. Methods: Herein, we assessed whether maxRatio could improve VL quantification using 2,544 QIAgen artus HI virus-1 RT-PCR reactions. The assay’s standard dilutions were used to build external standard curves with either FP or maxRatio that re-calculated the VLs. Results: FP and maxRatio were highly comparable (Pearson’s ρ=0.994, Cohen’s κ=0.885), and the combination of the two methods identified samples (n=41) with aberrant amplification profiles. Conclusions: The combination of maxRatio and FP could improve the predictive value of the assay.


Introduction
Infection with HIV-1 accounts for a global prevalence of 38 million cases and a one million deaths yearly 1 . An accurate viral load (VL), typically carried out by quantitative polymerase chain reaction (qPCR), is pivotal for addressing the efficacy of antiviral therapies 2 . The threshold level of detection for the HIV-1 VL has been reported in the range 20-44 viral genomic copies per milliliter (c/mL) 3,4 .
qPCR data are usually analyzed by the fit-point (FP) method, which assumes equal amplification efficiency between samples 5 . However, anomalies in the background fluorescence at low template input, can affect the quantification 6-8 . An alternative method, maxRatio, was introduced to overcome these issues 9 . It has been reported that maxRatio conferred a marginal increase in assay accuracy over FP 10,11 .
FP provides only a quantification cycle (Cq) value, which is then used to calculate VL. MaxRatio, instead, gives two parameters: one associated with the reaction's efficiency (MR) and one equivalent to, albeit distinct from, Cq. These two parameters can be linked to bestow a quantitative cycle (FCNA) compensated for inhibition.
In the present work, we aimed to determine whether maxRatio could improve the determination of HIV-1 VL. We compared the quantification of HIV-1 VL computed by FP and maxRatio on a dataset generated with the QIAgen artus HIV assay, which has a reported limit of detection of 35.5 c/mL, and we showed that maxRatio could pinpoint samples with abnormal amplification profiles.  405, 4,050, 40,500, and 405,000 c/mL. Each reaction also contained a primer set targeting an internal control (IC) to assess the proper extraction of the samples.

Data analysis
The FP method generated the Cq by registering the fractional cycle where the fluorescence passed the threshold of 0.2 units. The maxRatio transform of the amplification data and determination of the cut-offs were computed as previously described 9,10 . Different operators visually inspected the reaction's profiles and classified each reaction as either passed or failed. Using R v.3.6, linear models (standard curves, SC) were built on the CDs and applied to calculate the copy numbers according to the formula 10 (xb)/m where x is the quantitative cycle (either Cq or FCNA), b and m are the intercept and slope, respectively, of the linear models 13 . Testing the difference between the expected and the calculated copy numbers was carried out with an unpaired t-test. VL correlation was obtained with the Pearson product-moment coefficient ρ 14 and agreement between methods was tested with the Cohen's κ 15 ; both are reported with their 95% confidence interval (CI).

Results
The present dataset was derived from 122 individual artus HIV-1 runs, corresponding to 2,544 reactions (480 CDs, 122 NTCs and 1,931 clinical samples). The cut-offs obtained by expectation-maximization analysis were multiplied by 2.7 to generate the values used to filter the maxRatio data, as depicted in Figure 1.
FP quantified 22 reactions below the detection limit of 20 c/mL that were interpreted as non-reactive by maxRatio. Conversely, maxRatio identified 18 reactions below 20 c/mL while FP quantified them above this level. Visual inspection of the amplification profiles of the reactions failed by FP showed that 10 of them (45.5%) had a proper sigmoid shape for the IC signal that, however, was discarded by the FP (as exemplified in Figure. 5A). In contrast, the others had a low signal for either HIV-1 or IC recovered by maxRatio ( Figure 5B). Conversely, 15 (83.3%) of the reactions failed by maxRatio showed either a low IC or HIV-1 input ( Figure 6A), whereas the FCNA of the remaining reactions produced fractional VL that were rounded to 0 c/mL ( Figure 6B).

REVISED Amendments from Version 2
1. We re-arranged the acronyms in the paper to make the labelling clearer, specifically by clarifying the use of internal control (IC) in the method section.
2. We corrected a wrong proportion to 45.5%.
3. We encountered no issues in downloading fully functional comma-separated files. We recommend using the "original file format" option provided by Harvard Dataverse when downloading the datasets. We added a dictionary file explaining the fields' names used in the datasets associated with the present work.

Discussion
The purpose of the present work was to assess the potential benefits of maxRatio in determining HIV-1 VL given its inherent compensation of PCR inhibition. Contrary to our expectation, the SCs we built with either FP or maxRatio were virtually the same. Both methods gave VLs significantly divergent from the expected copy numbers at the lower and upper CDs, and maxRatio was, in general, more discrepant from the expected copy numbers than FP. Even concerning the samples' quantification, the two methods produced essentially the same VLs. The main difference between the two methods was in terms of sample's reactivity. By accepting the reactions identified as non-reactive by FP, but reactive by maxRatio, there would have been 18 false-negative results. Conversely, 15 samples identified as non-reactive by maxRatio showed aberrant IC that raised quality control, rather than false-positive, issues overlooked by FP.
The current use of maxRatio is to confirm the reactivity determined by FP on the Abbott m2000rt platform. Our data supported this combination as the most effective approach for screening purposes. Samples in disagreement between FP and maxRatio would require further assessment that could reduce the workload involved in issuing the results and minimize the risk of providing false results.
The present work had some limitations. Firstly, the sample size was small. Since the correlation between the two methods was high (99.4% overall and 84.4% below the quantification limit of 405 c/mL), a more extensive sample set would provide only a marginal improvement in comparing the two algorithms. However, more samples will provide more instances of amplification profiles that are processed differently by FP and maxRatio. In the present study, 40 reactions out of 1931 samples (2.07%) showed discrepancies in quantification at the clinical threshold of 20 c/mL. Expanding such a subset of discrepant reactions to, say, 4000 could provide a database of profiles that can facilitate (perhaps using machine learning approaches) identifying the characteristics that led to the failure in quantification. Moreover, analysis of qPCR chemistries other than artus HI virus-1 might determine whether such  characteristics are common to all reactions or peculiar to the kit used herein. Secondly, the CDs were prepared by diluting the control samples provided in the kit, but the actual concentration was not measured. Finally, we did not have access to the actual issued results; thus, we could not confirm the official VL values.
In conclusion, we compared FP and maxRatio in providing HIV-1 VL. Contrary to our expectations, maxRatio did not give a better quantification than FP, but combining the two methods could minimize issuing false results.

Open Peer Review Joel Tellinghuisen
Department of Chemistry, Vanderbilt University, Nashville, TN, USA The authors use a method apparently first described in their ref 9 (2008), which was coauthored by the 2 nd author of this work. This maxRatio method seems to be a useful tool for automated screening of very large numbers of qPCR reaction profiles for non-detects, though it is not clear to me that it is much better than, for example, just setting a low limit on acceptable plateau levels, perhaps combined with an upper limit on acceptable C q values, beyond which the template number would round to zero. I did not see any comparisons of this sort either here or in ref 9. And I am not well enough acquainted with the various instruments to know what other possible tools there are for this kind of discrimination. However, it does appear to me that the title is not quite a truthful description of the work, since the authors note in Results that the MR method and the "fixed point" (FP) method with which they compare it gave nearly identical "stratification of the reactions into reactive and non-reactive." And then early in the Discussion (and again in the final paragraph of the Conclusion), they acknowledge that the two methods gave virtually identical standard curves (SC). On the other hand, it is not clear to me how the discrimination was done in the FP method. This may have been by manual inspection (bottom of 1 st column, p 3), in which case the MR method would still be advantageous. (But, as noted, the "inspection" could also be automated.) The near-agreement in the 2 standard curves is not surprising, because the FCNA is essentially a first-derivative maximum (FDM), shifted slightly from the use of finite differences and even more by the correction -log2(MR) (which is not mentioned here but is given in eq 2 in ref 9). Incidentally, this correction must only apply for a particular instrument and its fluorescence axis scale, because it clearly changes as the data are scaled. In any event, both C t (used here for C q in the FP method) and the FDM are legitimate C q markers, so they should indeed give similar SCs. In fact, C t is arguably the poorest C q marker, for reasons Spiess and I have explained, most recently in Biomol. Detect. Quantif. 17 (2019) 100084. 1 Thus, the use of, for example, a relative threshold obtained from a fit model that includes the baseline function, should give even better FP results.
I wanted to check some of the authors' results but unfortunately could not easily get the data from their referenced source (p 9 and ref 12). The Excel files seem not to be .csv, as labeled; the .tab files are text, but the application for opening them is not given (GitHub perhaps?). Nor is their content explained. I did attempt to reproduce their SCs in Figure 2, by digitizing the plotted data (WebPlotDigitizer, https://apps.automeris.io/wpd/). My results more or less confirmed theirs with one exception: My uncertainties for the back-calculated N 0 values for FP in Table 1 were ~10% smaller than theirs for the first 3 concentrations but ~40% larger for the highest. The relative errors here depend on only the structure of the data, so the last should also have been small by ~10% (this discrepancy from digitization limitations). The larger error makes the discrepancy < 2s for this case, removing one of the two values judged to be discordant.

Some specific problems:
Some labels are not defined: EM (1 st para of Results), SD (5 th line of Discussion). And IC appears in Fig. 1 but isn't defined until several paragraphs after Fig. 1 is mentioned. 1.
I don't understand the introductory sentence in the last para of Results. And if the 10 which had sigmoidal appearance were of 22, that would be 45.4%, not 43.4%.

2.
While it is clear that the reaction profiles in Figs 5B and 6A should not give results, I don't understand why those in 5A do not. 3.
paragraph of the results section. We agree with the reviewer that the slope of the standard curves built with either FP or maxRatio should be similar. The focus of the work was not on the subtle differences between the standard curves based on the two methods but their applications.
The driving force of this and the previous works on the application of maxRatio done by Shain and Marongiu was to avoid the definition of a baseline threshold level. Firstly, the guidelines for the threshold value are too generic to give universal outcomes. Secondly, the threshold value might not be the same for each PCR assay. Thirdly, in our experience, the threshold level is set without much mathematical consideration. Conversely, maxRatio provides an objective way to quantify the reactions that dispense the threshold level altogether. As the reviewer remarked in his cited paper (Biomol Detect Quantif 2019;17:100084) the threshold level does not hold throughout the PCR while "most workers have taken [it] as a level near baseline where it is hoped that eq. 2 [associating the copy number to the efficiency of the reaction] remains valid". We agree with the conclusion of the author in his paper --that is, that the absolute threshold is a poor choice --because we believe that maxRatio allows overcoming its use. We disagree with the reviewer when he stated, "it is not clear to me that [maxRatio] is much better than, for example, just setting a low limit on acceptable plateau levels". If the reviewer meant to lower the threshold, there would be firstly the problem mentioned above of where to set the cut-off level, and secondly, there would be a trade-off between picking up more true positives at the expense of more false positives. This trade-off would require further determination of the impact of the analytical method on the sensitivity and specificity of the assay that (a) was beyond the scope of our work and (b) is skipped altogether by using maxRatio.
Furthermore, we disagree with the reviewer when he remarked, "FCNA is essentially a firstderivative maximum (FDM), shifted slightly from the use of finite differences and even more by the correction -log2(MR)". In fact, maxRatio is not the same as FDM. maxRatio does not use differences. maxRatio consistently identifies a cycle number earlier and closer to the exponential portion of the amplification curve. In addition, maxRatio provides a second measure (MR) which is highly useful in evaluating reactive-nonreactive status. FCNA helps correct the quantitation of PCR results where the response is partially inhibited. FDM cannot do this. We also disagree with his statement, "This correction must only apply for a particular instrument and its fluorescence axis scale because it clearly changes as the data are scaled." maxRatio works by calculating ratios. It is entirely unaffected by multiplicative scaling effects.
We loaded the datasets following the Harvard Dataverse's guidelines, essentially uploading a comma-delimited table. We quickly downloaded the original data simply by clicking on the "download" icon and selecting the download option. On downloading the data saved on Harvard Dataverse, the comma-separated values original file format)" can be opened directly with a spreadsheet application (in our case: LibreOffice Calc). On downloading the alternative version "tab-delimited", Harvard Dataverse automatically adds the extension .tab, but the file's layout remains comma-delimited, implying that the spreadsheet applications do not appropriately recognize such file format. We recommend downloading the dataset only in its original format. Nevertheless, the procedure of data downloading is under the domain of Harvard Dataverse.
significance of the data contained in Table 1 (i.e paired or unpaired, with or without other corrections such as Welch's or Bonferroni's) and which software was used to perform the statistical analysis.
In the Results paragraph it is suggested to mitigate the following statement: "…the agreement in the stratification of the reactions into reactive and non-reactive was almost perfect (κ = 0.885, 95% CI:0.863-0.907). ". Instead, "noticeably robust" can be a good alternative.