Measurement uncertainty interval in case of a known relationship between precision and mean

Experimental design and statistical model for an in-house validation study with samples with known concentrations

It is assumed that an in-house validation study has been conducted with samples with known concentrations. In the simplest case, the samples with known concentrations are obtained by diluting certified reference material. For each concentration level, several measurement results are obtained under in-house reproducibility conditions. This is best achieved via a factorial design, such as described in ISO/TS 23471.³

The following table (Table 1) provides an example of a factorial design with 7 factors and 8 factor level combinations.

More generally, the number of factor level combinations is denoted $\mathit{n}$. With $\mathit{m}$ (known) concentration levels and $\mathit{p}$ replicates per concentration level, a total of $\mathit{m}\bullet \mathit{p}\bullet \mathit{n}$ measurements are performed.

The statistical model (mixed linear model) is as follows:

${\mathit{Y}}_{\mathit{ijk}}$ denotes the measurement result at concentration level $\mathit{i}=1,\dots ,\mathit{m}$ for factor level combination $\mathit{j}=1,\dots ,\mathit{n}$ and replicate $\mathit{k}=1,\dots ,\mathit{p}$

${\mathit{x}}_{\mathit{ij}}$ denotes the known concentration,

$\mathit{\alpha }$ and $\left(\mathit{\beta }-1\right)$ denote the absolute and the relative components of method bias. The $\mathit{\alpha }+\mathit{\beta }\bullet \mathit{x}$ curve represents the expected measured concentration at known concentration $\mathit{x}$ and will be referred to in the following as the mean curve.

$\mathit{A}\left(\mathit{j}\right)$ and $\mathit{B}\left(\mathit{j}\right)$ denote the absolute and the relative components of the effect of factor level combination $\mathit{j}$ (each of these terms results from summing effects of the individual factors).

${\mathit{A}}_{\mathit{j}}$ and ${\mathit{B}}_{\mathit{j}}$ denote the absolute and the relative components of block effect $\mathit{j}$

${\mathit{a}}_{\mathit{ijk}}$ and ${\mathit{b}}_{\mathit{ijk}}$ denote the absolute and the relative components of the repeatability error for measurement result ${\mathit{Y}}_{\mathit{ijk}}$.

In the following, two in-house validation data sets will be considered: Thiamphenicol in milk (Example 1) and Clopidol in egg (Example 2).

The following table (Table 2) provide test results for Example 1 ($\mathit{m}=4$ known concentration levels, $\mathit{p}=1$ replicate).

The test results for Example 1 are displayed in Figure 1.

Figure 1. Example 1 - test results.

The following table (Table 3) provides test results for Example 2 ($\mathit{m}=6$ known concentration levels, $\mathit{p}=1$ replicate).

The test results for Example 2 are displayed in Figure 2.

Figure 2. Example 2 - test results.

For each example, reproducibility standard deviation values are calculated by means of Equation 1. With the exception of the two fixed effects $\mathit{\alpha }$ and $\mathit{\beta }$, the terms on the right side of Equation 1 are modelled as random variables. The corresponding variance components are denoted as follows:

${\mathit{A}}_1\left(\mathit{s}\right),\dots ,{\mathit{A}}_{\mathit{q}}\left(\mathit{s}\right)$ are random variables with zero mean and variances ${\mathit{\sigma }}_{\mathit{A},1}^2,\dots ,{\mathit{\sigma }}_{\mathit{A},\mathit{q}}^2$, respectively

${\mathit{B}}_1\left(\mathit{s}\right),\dots ,{\mathit{B}}_{\mathit{q}}\left(\mathit{s}\right)$ are random variables with zero mean and variances ${\mathit{\sigma }}_{\mathit{B},1}^2,\dots ,{\mathit{\sigma }}_{\mathit{B},\mathit{q}}^2$, respectively and where

$\left({\mathit{s}}_1\left(\mathit{j}\right),\dots ,{\mathit{s}}_{\mathit{q}}\left(\mathit{j}\right)\right)$ denotes the vector of factor levels for the $\mathit{q}$ factors and for factor level combination $\mathit{j}$

Moreover,

${\mathit{A}}_1,\dots ,{\mathit{A}}_{\mathit{n}}$ are random variables with zero mean and variance ${\mathit{\sigma }}_{\mathit{A}}^2$

${\mathit{B}}_1,\dots ,{\mathit{B}}_{\mathit{n}}$ are random variables with zero mean and variance ${\mathit{\sigma }}_{\mathit{B}}^2$.

${\mathit{a}}_{111},\dots ,{\mathit{a}}_{\mathit{mnp}}$ are random variables with zero mean and variance ${\mathit{\sigma }}_{\mathit{a}}^2$

${\mathit{b}}_{111},\dots ,{\mathit{b}}_{\mathit{mnp}}$ are random variables with zero mean and variance ${\mathit{\sigma }}_{\mathit{b}}^2$

The estimation of variance components in mixed linear models is described, for example, in Searle et al.,⁴ McCulloch et al.⁵ and Clarke.⁶ Estimates for Example 1 are provided Table 4.

The in-house precision parameters for Example 1 for four different concentrations are provided in Table 5. At a given known concentration $\mathit{x}$, the in-house reproducibility standard deviation is calculated as follows:

The following two tables (Table 6 & Table 7) provide variance and precision estimates for Example 2.

Calculation of the $\mathit{y}\mathit{\pm }\mathit{U}$ measurement uncertainty

The estimate of reproducibility at concentration $\mathit{x}$ (see Equation 2) can be used to derive an estimate of the standard measurement uncertainty for the measurand $\mathit{Y}$ as a function of $\mathit{x}$:

The expanded measurement uncertainty is then obtained as follows:

where $\mathit{k}$ denotes the coverage factor.

It should be noted that it may be necessary to include the uncertainty of bias correction and the uncertainty of the (certified reference) values for the known concentrations used in the validation study, as appropriate. (see step 3 of Asymmetric measurement uncertainty approach).

Based on the precision values from the previous section, and using $\mathit{k}=2$ (for the sake of simplicity), one obtains the following expanded uncertainty values (Table 8 & Table 9):

The inconsistency of the y ± U approach in the case of heteroscedasticity

Typically, the expanded measurement uncertainty is used to construct a measurement uncertainty interval of the form $\mathit{y}\pm \mathit{U}$. Such symmetric measurement uncertainty intervals are not always appropriate. This will be demonstrated on the basis of a theoretical example.

In this example, it is assumed that the relative measurement uncertainty is already known and is constant at 35% across the applicable range of concentrations as specified in the scope of the analytical method. If the measurement result is $\mathit{y}=10$ (in this theoretical example, the unit plays no role and is therefore suppressed), then, applying the $\mathit{y}\pm \mathit{U}$ approach with $\mathit{k}=2$ (for the sake of simplicity) and with the standard uncertainty obtained from the known 35% relative standard deviation (RSD) value, a measurement uncertainty interval of [3, 17] is obtained. According to the JCGM definition of measurement uncertainty discussed in the introduction, this means that the value $\mathit{y}=3$ could reasonably be attributed to the measurand. However, if the measurement result had been $\mathit{y}=3$ (instead of $\mathit{y}=10$), applying the same 35 % relative measurement uncertainty would result in an uncertainty interval of [0.9, 5.1]. Since the original value of $\mathit{y}=10$ does not lie within this second interval, it can be seen that the $\mathit{y}\pm \mathit{U}$ approach is inconsistent in such a situation.

The two uncertainty intervals just discussed are summarized in the following table (Table 10).

The inconsistency of the $\mathit{y}\pm \mathit{U}$ approach discussed above is illustrated in the following figure (Figure 3). Due to the constant RSD of 35 %, the width of the uncertainty interval (in the diagram: the height of the uncertainty interval) depends on the measurement result. Considered on its own, each uncertainty interval characterizes the dispersion of values that could be reasonably be attributed to the measurand on the basis of the measurement result. However, considered together, the different uncertainty intervals display inconsistencies. In particular, the value $10$ does not lie within the uncertainty interval for the measurement result $3$, even though the latter lies within the uncertainty interval constructed around the former.

Figure 3. Uncertainty intervals (vertical axis) following the $y\pm U$ approach for a constant RSD of 35%, for two measurement results (horizontal axis).

(In this theoretical example, the unit plays no role and is therefore suppressed).

The inconsistency of the $\mathit{y}\pm \mathit{U}$ approach will now be shown on the basis of the uncertainty intervals for Example 2 provided in Table 9. At a Clopidol concentration of 4 μg/kg Clopidol, the $\mathit{y}\pm \mathit{U}$ approach yields the uncertainty interval y ± 1.19 μg/kg. In routine testing, if the measurement result is $\mathit{y}=4$ is obtained, it will be concluded that values above 5.19 μg/kg cannot reasonably be attributed to the measurand. However, from the same Table 9, we know that the lower limit of the uncertainty interval for a sample with 5.5 μg/kg Clopidol concentration is 3.88 μg/kg (or 3.98 μg/kg if bias is considered). In other words, we know that a measurand value of 5.5 μg/kg is perfectly compatible with a measurement result of 4 μg/kg. It follows that the value 5.5 μg/kg should lie within the measurement uncertainty interval for a measurement result of 4 μg/kg. This is not the case for the uncertainty interval obtained via the $\mathit{y}\pm \mathit{U}$ approach. This is illustrated in the following figure (Figure 4).

Figure 4. Uncertainty intervals (vertical axis) following the $y\pm U$ approach on the basis of the data from Example 2 (Clopidol in egg) for two measurement results (horizontal axis).

For a measurement result of 4 μg/kg (say in routine testing), values above 5.19 do not lie within the uncertainty interval. However, according to the evaluation of the data from the in-house validation study, measurement results $\le$ 4 μg/kg were consistent with a measurand value of 5.5 μg/kg.

As has just been seen, in the presence of heteroscedasticity, the $\mathit{y}\pm \mathit{U}$ approach leads to inconsistencies. For this reason, a different approach for the determination of measurement uncertainty in the case to heteroscedastic data is required. Such an approach will now be presented.

Description of the approach

Step 1: prediction range

For each known concentration, the distribution of test results can be characterized in terms of a prediction interval. This interval reflects the degree to which the test results agree with one another, at the given concentration level and under the specified conditions (e.g. repeatability or in-house reproducibility). For a chosen prediction probability level (e.g. ${\mathit{p}}_{\mathit{\text{pred}}}=95\%$), a subsequent test result will lie inside the prediction interval with probability ${\mathit{p}}_{\mathit{\text{pred}}}$.

If measurements are performed at different known concentrations, then it is possible to construct a prediction range, rather than individual prediction intervals. This step involves applying one statistical model to all the data, as described the previous section. The construction of a prediction range is described in the following figure (Figure 5).

Figure 5. Construction of a prediction range.

For each known concentration, the diamonds represent the measurement results and the solid vertical line represents the prediction interval. The prediction range is represented by the dashed lines.

For further information regarding the computation of a prediction range, the reader is referred to the discussion of variance functions in ISO 5725-2⁷ and to the in-house validation approach described in Gowik et al.⁸ and Jülicher et al.⁹ [1].

Step 2: measurement uncertainty interval

Once a prediction range has been calculated, the measurement uncertainty interval for a given test results (obtained e.g. in routine testing) can be determined. Step 2 no longer involves the data from the validation study. Rather, for Step 2, it is assumed that a prediction range has previously been calculated and is thus available.

Accordingly, the meaning of the axes is now different. The vertical axis now represents the measurement result obtained, say, in routine testing and denoted ${\mathit{y}}_{\mathit{m}}$, while the horizontal axis now represents the measurand (which is to be characterized via the measurement) denoted $\mathit{y}$ (This notation is chosen so as to be consistent with JCGM 106, though, strictly speaking, ${\mathit{y}}_{\mathit{m}}$ is one realization of a random variable ${\mathit{Y}}_{\mathit{m}}$ and $\mathit{y}$ is one realization of a random variable $\mathit{Y}$. However, a distinction between random variables and their realizations is not required in this paper.).

The starting point for Step 2 is a test result, displayed on the ${\mathit{y}}_{\mathit{m}}$-axis. The intersection of the ${\mathit{y}}_{\mathit{m}}$-value with the upper prediction curve is projected onto the $\mathit{y}$-axis to obtain the lower measurement uncertainty limit. Indeed, for measurand values below this $\mathit{y}$-value, measurement results can be expected to be less than the ${\mathit{y}}_{\mathit{m}}$-value. Secondly, the intersection of the ${\mathit{y}}_{\mathit{m}}$-value with the lower prediction curve is projected onto the $\mathit{y}$-axis to obtain the upper measurement uncertainty limit. Indeed, for a measurand values above this $\mathit{y}$-value, measurement results can be expected to be greater than the ${\mathit{y}}_{\mathit{m}}$-value. The resulting $\mathit{y}$-axis interval (grey horizontal band hugging the y-axis) thus corresponds to the values which could “reasonably be attributed to the measurand.”

This procedure is illustrated in Figure 6.

Figure 6. Procedure for obtaining a measurement uncertainty interval from a measurement result.

Step 3: best estimate for the measurand

The “best estimate of the measurand” is the projection onto the $\mathit{y}$-axis of the intersection of the measurement result (${\mathit{y}}_{\mathit{m}}$-value) with the mean curve $\mathit{\alpha }+\mathit{\beta }\bullet \mathit{y}$ (see previous section with the measurand $\mathit{y}$ replacing the known concentration $\mathit{x}$). If the bias is negligible (i.e. $\mathit{\alpha }$ is close to zero and $\mathit{\beta }$ is close to 1), then this $\mathit{y}$-value will be close to the ${\mathit{y}}_{\mathit{m}}$-value. However, if a bias is present, then the two values will differ, and taking the $\mathit{y}$-axis projection corresponds to a bias or recovery correction. For this reason, the $\mathit{y}$-axis projection is denoted ${\mathit{y}}_{\mathit{\text{corr}}}$. If such a bias or recovery correction is performed, it is recommended to include the “uncertainty of the bias correction” (see Ref. 1) as an additional variance component in the calculation of the prediction intervals at each known concentration $\mathit{x}$ during the validation study. The uncertainty of bias correction, in turn, may consist of various sources of uncertainty such as the statistical uncertainty of the parameters $\mathit{\alpha }$ and $\mathit{\beta }$ and the uncertainty of reference values used as known concentrations.

Finally, it should be noted that the symmetry of the measurement uncertainty interval is determined in relation to ${\mathit{y}}_{\mathit{\text{corr}}}$.

Calculation of the measurement uncertainty intervals

In this section, it is shown how to calculate the lower and upper limits of asymmetric measurement uncertainty intervals.

It is assumed that there is no bias and that, for each concentration level in the recovery experiment, the measurement results follow a normal distribution.

Let ${\mathit{y}}_{\mathit{L}}$ denote the lower limit of the expanded uncertainty interval and ${\mathit{y}}_{\mathit{U}}$ the corresponding upper limit. Furthermore, let ${\mathit{f}}_{\mathit{U}}\left(\mathit{y}\right)$ denote the upper limit of the prediction interval at measurand value $\mathit{y}$, and let ${\mathit{f}}_{\mathit{L}}\left(\mathit{y}\right)$ denote the lower limit of the prediction interval at $\mathit{y}$. For a given measurement result ${\mathit{y}}_{\mathit{m}}$, the two limits ${\mathit{y}}_{\mathit{L}}$ and ${\mathit{y}}_{\mathit{U}}$ can then be computed iteratively, using the implicit relationships

In the case of a constant relative standard deviation ${\mathit{\sigma }}_{\mathit{rel}}$ across all concentration levels, and in the absence of bias, explicit expressions for ${\mathit{y}}_{\mathit{L}}$ and ${\mathit{y}}_{\mathit{U}}$ can be provided. Indeed, in such a situation, a given measurement result ${\mathit{y}}_{\mathit{m}}$ obtained to characterize the (unknown) measurand $\mathit{y}$ will lie with 95 % probability [2] in the interval $\left[\mathit{y}-2\bullet \mathit{y}\bullet {\mathit{\sigma }}_{\mathit{rel}},\mathit{y}+2\bullet \mathit{y}\bullet {\mathit{\sigma }}_{\mathit{rel}}\right]$, i.e.

The above inequality can be rewritten as

Thus, the measurand $\mathit{y}$ will lie in the interval $\left[\frac{{\mathit{y}}_{\mathit{m}}}{1+2\bullet {\mathit{\sigma }}_{\mathit{rel}}},\frac{{\mathit{y}}_{\mathit{m}}}{1-2\bullet {\mathit{\sigma }}_{\mathit{rel}}}\right]$ with 95% probability.

For example, for a constant relative standard deviation of 40% (i.e. ${\mathit{\sigma }}_{\mathit{rel}}=0.4$) and for the measurement result ${\mathit{y}}_{\mathit{m}}=100$, the lower and upper uncertainty limits are computed as:

As long as the exact quantile is used and the assumptions are valid, this uncertainty interval is statistically exact in the sense that the coverage probability is exactly 95%.

Symmetric versus asymmetric intervals

When is the proposed uncertainty interval symmetric?

If the prediction limits run parallel to the mean curve $\mathit{\alpha }+\mathit{\beta }\bullet \mathit{x}$ (see Section 0), the uncertainty interval is perfectly symmetric. This is the case when it is the absolute rather than the relative standard deviation which is constant across concentration levels. It should also be noted that, when the prediction limits run parallel to the mean curve $\mathit{\alpha }+\mathit{\beta }\bullet \mathit{x}$ and when there is no bias (100 % recovery), then the prediction and measurement uncertainty intervals are identical. This is illustrated in the following figure (Figure 7).

Figure 7. Uncertainty intervals (A) following the $y\pm U$ approach and (B) following the asymmetric approach.

If there is no bias and if the prediction limits run parallel to the mean curve $\alpha +\beta \bullet x$, then the intervals obtained from the two approaches are identical.

The assumption that an uncertainty interval is symmetric is justified if the following two conditions are met:

Condition 1: Measurement results are distributed symmetrically around the corresponding mean value at a given known concentration in the validation study.

Condition 2: Heteroscedasticity is negligible.

When is the proposed uncertainty interval asymmetric?

In the field of analytical chemistry, in most cases, the distribution of results obtained from measurements performed on one and the same sample is more or less symmetric, so that symmetry condition 1 above is met. Condition 2, however, is almost never met. In the case of weakly heterogeneous variances, uncertainty intervals may still be approximately symmetric. Similarly, if the spread of measurement results at any given concentration level remains small (i.e. the relative standard deviations are small), uncertainty intervals may still be approximately symmetric. However, if the dispersions vary considerably across concentration levels and the corresponding relative standard deviations are large (say, greater than 10 %), then it is necessary to take the asymmetry of the uncertainty intervals into account. Restricting the concentration range under consideration in order to avoid variance heterogeneity is common analytical practice. In some cases, this expedient may allow the symmetry assumption to be applied.

The following figure (Figure 8) illustrates the relationship between magnitude of dispersion and symmetry of the uncertainty interval.

Figure 8. Measurement uncertainty intervals constructed following the asymmetric approach, starting from the same measurement result $y_m=5$.

The degree of asymmetry depends on the magnitude of the variance.