What does it mean if a patient is positive for anti-Jo-1 in routine hospital practice? A retrospective nested case-control study

Introduction

Diagnosis of a connective tissue disease (CTD) may be considered in many clinical situations because of the diverse clinical manifestations of these diseases. Diagnosing a specific condition may be facilitated by a variety of immunological tests, including auto-antibodies to anti-nuclear antibodies (ANA) or extractable nuclear antigens (ENA)^1,2. Anti-ENA antibodies consist of a panel including auto-antibodies associated with systemic lupus erythematosus, scleroderma, mixed connective tissue disease, myositis and Sjogren’s syndrome.

Data from the manufacturers of anti-ENA testing kits indicate that the diagnostic precision of anti-Jo-1 (sensitivity and specificity) exceeds 95%³. The sensitivity and specificity of a test does not tell us about the probability of disease. The sensitivity and specificity data of many diagnostic tests are calculated from case-control studies, in which test positivity is compared in patients with a known connective tissue disease with healthy controls or controls with other diseases. These methods greatly over-estimate the positive predictive value (PPV) of a diagnostic test, because disease prevalence is increased in the assembled population by the methods used for calculation⁴. Clinicians find diagnostic test accuracy data confusing and tend to over-estimate the probability of disease following a positive test result⁵. Disease prevalence in populations tested for anti-ENA in routine care is rarely known because testing may be requested and performed for any patient and without constraints in most hospitals. Thus, the prevalence of relevant diseases is likely to be considerably lower. Positive test results in general hospital practice may thus lead to over-diagnosis, especially in sick hospital patients with multiple morbidities, and may lead to additional expensive testing, patient and clinician anxiety and overtreatment.

In this study we focus on anti-Jo-1 (histidyl-tRNA synthetase). This autoantibody is associated with inflammatory muscle diseases and interstitial lung diseases (ILD). We retrospectively identified a cohort of patients tested for the panel of anti-ENA autoantibodies in our hospital. Using a nested case-control design we identified patients who tested positive for anti-Jo-1 and compared them with controls, from the same source population, who tested negative for anti-Jo-1. We sought to determine the accuracy and value of this test for hospital medical practice.

Methods

Results

A total of 4014 samples from 3584 patients were tested for ENA during the 2-year study period. There were 3 patients excluded because their samples originated from outside our hospital, leaving 4009 samples and 3581 patients. The first sample tested chronologically was designated the test of interest. Positive ENA tests occurred in 616 (17.2%) patients, 2965 (82.8%) patients were negative. The study flow diagram is shown in Figure 1. The frequency of positives at levels of ≥20 and ≥40 units are shown in Table 1. SSA, SSB, combined SSA and SSB, including patients with high concentrations of auto-antibodies, were commonly found in this population.

Figure 1. Study flow diagram.

We studied the anti-Jo-1 positive patients (40, 1.1%) in detail and selected 80 ENA negative patients, randomly, as controls. Key clinical characteristics of these two populations are shown in Table 2.

A total of 350 patients (9.8%; 428 samples or 10.7% of all samples) were tested on more than one occasion, 55 patients were tested on 3 or more occasions and 10 patients were tested on 4 or more occasions. Of the 40 (1.1%) anti-Jo-1-positive samples, 7 had the test done on more than one occasion. The median interval between the first and second test request for 350 patients was 124 days (interquartile range, 233 days; mean 168 days).

The stability of Jo-1 values between the first and second test, where a second test was requested later and where numerical data (ENA screen positive) were available, are shown in Figure 2. This illustrates that values remain stable between repeat tests over a median of 124 days, but that stability appears to lessen as anti-Jo-1 values rise. The likelihood of tests being repeated was greater in ENA-positive patients (18.8%) than in ENA-negative patients (7.8%), p<0.0001, Fisher’s exact test.

Figure 2. Bland–Altman plot of extractable nuclear antigen-screen-positive patients with at least two Jo-1 measurements (n=109 patients).

Jo-1 is considered positive if >20 AU/ml. The bold horizontal line shows the bias and dotted lines show upper and lower limits of agreement. Narrow limits of agreement are shown, but note that agreement appears to deteriorate at higher values of Jo-1. The bias or mean difference value (bold horizontal line) is small.

Finally, using available stored samples from Jo-1-positive patients (n=33), we checked whether repeat measurement on the same sample using the same Quanta Lite® platform yielded consistent value (Figure 3). In three cases the first test result was reported as >50 U/ml (per laboratory practice at that time), whereas on retesting, more precise values were reported. For analysis, we assumed, in these strongly positive cases, that the original test results were identical to the more precise values reported on repeat testing. The degree of agreement between these repeat measures on the same sample is visualized using a Bland–Altman plot (Figure 3), the bias (average difference between repeat measures) was 5 AU/ml (95% CI, −8.0–18.7). We were concerned that auto-antibody levels in stored samples would have degraded over time; however, results from stored samples were not consistently lower or higher than the index test result (Figure 3).

Figure 3. Bland–Altman plot of anti-Jo-1-positive (>20 AU/ml when first tested) patients compared with re-test result on stored serum re-tested using Quanta Lite assay for anti-Jo-1 (n=33).

The bold horizontal line shows the bias and dotted lines show upper and lower limits of agreement.

Discussion

Studies such as this one, which report on the diagnostic value of a test in routine care and where patients have a wide variety of clinical features, are uncommon. Often, academic estimations of the diagnostic value of a test are assessed by comparing positivity in patients with confirmed disease against those with no disease or another disease. This approach increases the prevalence of diseased subjects in the population used to make calculations of diagnostic accuracy. Diagnostic sensitivity and specificity of a test may vary with disease prevalence and with disease spectrum, for example disease severity and the presence of co-morbidity^4,11. Thus, a test may perform better where selected cohorts of patients with known disease and more severe disease are included, as seems likely to occur in cohorts of patients from referral centers, where much of the data on the relationship between Jo-1 and disease originate. The true value of a test only becomes apparent when it is used in usual clinical settings where diagnosis may be uncertain, other serious illnesses present, symptoms are unexplained or where there is multi-morbidity.

The PPV of anti-Jo-1 for ILD in our study was 12.5%. This figure is well below that produced by another study, in which anti-Jo-1 was concluded to be highly specific for autoimmune myositis but not thought to be sensitive¹². Our patients were tested using a commercial ELISA for anti-ENA and anti-Jo-1. These assays have been compared with other methods and other manufacturers’ immunoassays, and are believed to be reliable¹³. Nevertheless, anti-Jo-1 detected by ELISA may lack specificity^13,14. Older techniques are believed to be more specific and use native antigens in soluble form, whereas ELISAs and other assays, such as addressable laser bead immunoassays, use antigen-coated surfaces⁹. The use of a solid phase in ELISA could cause conformational changes to antigens or antigen denaturation, influencing test performance.

A plea for greater clinical correlation and consideration before introducing new assays, particularly in the context of local test usage, was made over a decade ago¹⁵. Indeed, the UK National External Quality Assessment Service (UKNEQAS) recommends that a less sensitive or more specific test, such as indirect immunofluorescence (IIF), should be used to screen for samples prior to testing by more sensitive methods¹⁵. However, UKNEQAS reports considerable differences in the interpretation of IIF. Regarding ENA testing, UKNEQAS note: “a more worrying observation is the continuing complete lack of homogeneity and agreement of reported ENA specificities across the various manufacturers for almost any ENA type”¹⁶.

Anti-Jo-1 is grouped with autoantibodies believed to be specific for myositis¹⁷, and occurs very rarely in normal populations¹⁸. Around 22% of patients with polymyositis diagnosed in specialist centers had anti-Jo-1¹⁹ In addition when clinical data was reviewed for anti-Jo-1 positive patients, identified from a laboratory database, 70% had ILD²⁰. This compares with 12.5% ILD in our anti-Jo-1-positive cohort. An association of myositis and ILD has led to coining of the term ‘anti-synthetase syndrome’. The notion of a unique syndrome attributable to the presence of auto-antibodies to synthetases has been challenged by a review of cohort studies⁹; our data add to this doubt. We showed that 17% of our population screened positive for anti-ENAs, most commonly SSA and or SSB. Anecdotally, we know that these positive tests result in referral from primary care to secondary care. Anti-Jo-1 occurred in 1.1% of our patients, including many without a recognized CTD, myositis or ILD.

Our study has notable limitations. First, its retrospective design means that systematic data collection, for example specialist review and specific steps to determine the presence of Raynaud’s, ‘mechanics hands’, myositis or ILD, were not planned from the outset. Second, patients found to be positive for anti-Jo-1 could have been investigated more diligently than those found to be negative: an example of work up or verification bias. Thus, there could have been a greater propensity to diagnose ILD and use diagnostic tests such as chest CT scans. We identified two patients with high titer anti-Jo-1 who had such scans motivated purely by a positive anti-Jo-1 test.

We relied on routinely collected clinical data recorded in hospital electronic records and depended on clinical diagnoses rather than applying disease classification criteria. We did not access primary care records and may also have missed data if patients had been seen at other hospitals or in private health clinics. However, a strength of our study was the inclusion of a large cohort of patients seen in a variety of clinical settings. A retrospective design allowed our study to be completed with limited resources and in a timely manner. The inclusion of a heterogeneous population provides more reliable estimates of the value of anti-ENA and anti-Jo-1 in hospitals, the setting where these tests are likely to be ordered. We judged that a follow-up period of 1 year after the first anti-ENA test was done, provided sufficient time for key clinical features to emerge, especially ILD. We recognize, though, that auto-antibodies may precede CTD by many years and acknowledge that more prolonged follow-up may have given added value.

We show that anti-Jo-1 occurs in patients with varied illnesses, including cancer and other autoimmune diseases, raising the possibility that seriously ill patients with multiple morbidities may develop auto-antibodies at significant levels without relevant auto-immune diseases, but perhaps because of tissue damage. Auto-antibodies found in myositis have been classified into those believed to be specific for myositis (for example anti-Jo-1) and those that are associated with myositis. This distinction implies a special role for certain autoantibodies in disease pathogenesis. For anti-Jo-1, this view may be justified since this autoantibody can cause lung and muscle disease in animal models²¹. Yet, our data suggest that caution is necessary in according anti-Jo-1 a prominent role in disease pathogenesis. Recent research indicates that histidyl-tRNA synthetases have a role beyond protein synthesis, such as maintenance of immune homeostasis and stimulating immunological responses^22,23. This suggests that much more needs to be learnt about the potential immunopathogenic role of anti-Jo-1 and other anti-synthetases.

The development, evaluation and dissemination of diagnostic tests is not standardized. Additionally, methods of test evaluation are often sub-optimal²⁴. Test usage may become widespread and clinicians may not fully appreciate the limitations of a test in practice. A test believed to be highly specific for a disease owing to published or manufacturer data may lead to additional confirmatory diagnostic testing, including invasive testing, with the risk of over-diagnosis and patient harm.

Of the anti-synthetase auto-antibodies, only anti-Jo-1 is tested routinely. A plea for wider serological testing, especially in patients with ILD, was made recently²⁵. It is believed that patients with ILD who have auto-antibodies other than anti-Jo-1 have a worse prognosis. However, we believe that robust evaluations of the precision of newer auto-antibodies for diagnosis and determining prognosis, in heterogeneous populations likely to be tested in secondary care, are necessary before wider dissemination.

We found that 10.7% (428) of the samples tested were repeat requests for anti-ENA, in some cases on multiple occasions. The ‘Choosing Wisely’ initiative, supported by the American College for Rheumatology, recommends restraint and careful patient selection when first testing for ANA. Testing for anti-ENA is not recommended when the ANA is negative²⁶. It appears, therefore, that widespread inappropriate laboratory testing is occurring. Determining the appropriateness of testing, however, is problematic²⁷, but examples of good practice, at least for restricting repeated test requests, have been described by the UK Academy of Medical Colleges²⁸. We did not attempt to ascertain reasons for repeated requests, but such testing may be done where there is diagnostic uncertainty, borderline results or simply because of inadequate review of available data or not knowing that samples had recently been submitted. For example, we show that 17.3% of anti-ENA repeat requests were made within 7 days of the first request.

Restricting test requests or ‘gating’ has been implemented successfully for anti-neutrophil cytoplasm antibody (ANCA) in an English hospital²⁹, though such a policy requires clinician and laboratory staff time, which may mean that such a strategy is not cost effective. However, a policy to decline repeat requests within specified time periods could be implemented more readily and could be automated. Our hospital is in the process of implementing this for selected immuno-diagnostic tests.

Limited data suggests that anti-Jo-1 levels reflect disease activity³⁰. Our data shows that anti-Jo-1 levels measured by ELISA vary at higher concentrations when re-checked at a later point (Figure 2). We also noted variation in test results when the same positive sample was re-tested (Figure 3) after storage for many months.

In summary, we show that many hospital patients negative for ANA are tested for anti-ENA, often repeatedly. We show that positivity for anti-Jo-1 is uncommon and that positive results are poorly predictive for interstitial lung disease and clinically diagnosed autoimmune myositis.

Data availability

Dataset 1. Complete data for all participants regarding disease status and results of extractable nuclear antigen (ENA) testing. Data are presented as ENA-negative controls, Jo-1-positive patients and data pooled from all patients. DOI: 10.5256/f1000research.14834.d204254¹⁰.