Keywords
adverse drug reactions, adverse drug events, drug safety, patient safety, Medicaid, pharmacoepidemiology, post-marketing surveillance, propensity scores, acetylcholinesterase inhibitors
adverse drug reactions, adverse drug events, drug safety, patient safety, Medicaid, pharmacoepidemiology, post-marketing surveillance, propensity scores, acetylcholinesterase inhibitors
Despite its limitations, the Food and Drug Administration’s (FDA) Adverse Drug Event Reporting System (FAERS) has successfully identified rare and unexpected adverse events1–3. In many previous studies, administrative data sources have been used to estimate the extent of the problem or confirm safety signals identified from AERS4,5. However, fewer studies have demonstrated the potential of administrative data for first-line adverse drug reaction (ADR) surveillance6. In this pilot study, we present a framework for directed discovery of possible ADRs using population-based administrative data sources, an approach intended to complement the FDA’s adverse reporting system. We describe our approach as directed because we target specific health outcomes of interest instead of simply mining the data for statistical associations.
We examined the associations between drug use and possible ADRs resulting from treatment of dementia with acetylcholinesterase inhibitors (AChEIs), namely, donepezil hydrochloride, rivastigmine tartrate, and galantamine hydrobromide. We measured associations for four classes of ADEs—established reactions (gastrointestinal and psychological disturbance), potential reactions based on drug pharmacology (respiratory disturbance), novel unexpected reactions (hepatic and hematological disturbance), and death. Hepatic and hematologic syndromes were evaluated because they are two examples of potentially fatal reactions that have been found in post-marketing surveillance of drug-induced disease7.
The directed discovery framework consists of clinical framing, data preparation, event detection, and hypothesis generating and testing. The first three components are described in the Methods; hypothesis generating and testing are explored in the Discussion.
Clinical framing consisted of reviewing the medical literature and consulting clinical experts to define the treatment groups, inclusion criteria, drug courses, outcomes and covariates.
Sources. Data consisted of pharmacy and medical claims and enrollment status from Utah Medicaid recipients in the fee-for-service program between 1/01/2003 and 12/31/2005. We linked Utah death-certificate data to Medicaid recipients by a deterministic method using a social security number. To protect patients’ privacy, all potentially traceable personal identifiers were removed. The University of Utah Institutional Review Board approved this study (IRB_00016984).
Subjects. We studied Utah Medicaid recipients’ aged 50 and older with a dementia-type diagnosis (Table 1). As Medicaid enrollment occurs on a monthly basis, we tracked membership enrollment and de-enrollment and censored the patients whose enrollment was terminated and not re-established within the study period. Because of the relatively high rate of sustained enrollment, approximately 99% of the cohort was enrolled for at least 80% of the months from their first until their last month of eligibility or until the study period ended. We did not limit inclusion to continuously enrolled recipients.
Treatment Groups. We inferred patient AChEI use by reconstructing courses of AChEI therapy from pharmacy claims data. To achieve a greater homogeneity among users’ disease stage and risk of adverse reactions8, we restricted the AChEIs cohort to the first incident course of AChEI therapy, which was defined as their first course with at least a 180-day drug-free period. To ensure that patients were receiving medical care during the 180-day drug-free period and were not receiving the drug elsewhere, recipients’ had to be enrolled and to have at least one medical claim during the 180-day drug-free (baseline) period. We defined a course of AChEI therapy as beginning on the week the drug was first dispensed and ending on day 60 after a continuous gap in the drug supply of ≥ 60 days (Figure 1).
AChEI = Acetylcholinesterase inhibitors. Rx = Dispensed Prescription.
The untreated comparison group consisted of Medicaid recipients 50 years and older with a dementia-like diagnosis who did not receive AChEI therapy. We established a 180-day baseline period during which recipients were enrolled and had at least one medical claim. The index date for individuals in the untreated group began at the first dementia-related outpatient visit that allowed for a 180-day baseline period. Starting time zero with a dementia-related outpatient visit established an indicated population that was engaging the health care system.
Outcomes. As noted earlier, our primary clinical outcomes were gastrointestinal, psychological, respiratory, hematological and hepatic conditions, and death. We identified health care visits related to each clinical outcome in professional and facility claims using Healthcare Cost and Utilization Project (HCUP) Clinical Classification Software (CCS) codes (documented in Table 1). As a primary diagnosis typically indicates the reason for seeking medical care or the most important problem at the visit, we limited the outcome detection to the primary diagnosis codes. We tailored outcome classifications for each study design (described under Event Detection). Our analysis also measured the association of AChEI use with death.
Potential confounding. We assessed demographic variables, comorbidities, drug therapy, and indicators of health care utilization as potential confounders. Comorbidity indices included HCUP comorbidity software version 3.2 and the modified RxRisk-V (RxRisk-Vm) score, which infers comorbidity using pharmacy claims9. We measured health care utilization by considering the number of outpatient visits, hospitalizations, and emergency department (ED) visits, and we also accounted for use of hospice services and nursing home care.
We considered specific classes of medications as potential confounders—specifically, antianxiolytics, anticonvulsants, Parkinson’s treatment, antidepressants, antipsychotics, steroids, narcotics, respiratory agents, anticoagulants, corticosteroids, and sedatives. We treated the use of statin drugs as an indicator of health status because they are preferentially prescribed to healthier, less frail patients who are not at the end of life10.
Person time unit. We constructed the final analytic table using 1-week discrete time intervals; i.e., changes in covariate status, medication use and outcomes are captured weekly. This interval maximizes efficiency without omitting clinically important changes in patient outcome and covariate status. All database manipulation was conducted in SAS 9.2.
Cohort design. We used an open cohort design with propensity score matching to explore associations between data on drug utilization and possible ADRs. We used propensity scores to address covariate imbalance using logistic regression models to predict AChEI treatment. We included confounders and risk factors in the propensity score models11. Because we included risk factors along with confounders, we built separate propensity score models and matched cohorts for each study outcome. Two physicians who routinely treat patients with dementia independently selected variables to construct propensity score models. They discussed disagreements to arrive at consensus. Variables for each model are listed in Table 3.
Our analyses included propensity score matching followed by additional matching on key prognostic covariates12. For example, we performed propensity matching with covariate matching whether an individual had a gastrointestinal visit during the baseline period when evaluating the gastrointestinal outcome. Analysis of death consisted of propensity score matching and covariate matching for baseline age and hospice care.
Clinical endpoints were intended to measure increased health care utilization associated with specific diagnoses. We defined episodes of care to differentiate clusters of events and to reduce the impact of immediate clinical exuberance associated with a new episode of care. A 4-week gap in claims for each clinical outcome was required to initiate a new episode. For each study endpoint, we calculated the incidence densities per 100 patient-years.
We established matched untreated cohorts using Mahalanobis metric matching13. Baseline characteristics of patients in the AChEI-treated and matched untreated cohorts were compared using Student’s t-tests and chi-square tests. We used conditional multivariable Cox-proportional hazard models that allowed for recurrent events to assess the effect of AChEI on specific clinical endpoints14. All statistical analyses were performed with Stata MP 9.2 for Windows.
Case-crossover design. We established three 6-week time-windows (pre-treatment, first treatment, second treatment window) to assess acute and transient effects of AChEI treatment (Figure 1). The index week for the pre-treatment window was the week following the most recent clinic visit for any condition during the baseline period.
To capture acute effects of AChEI treatment, we used the week the AChEI was first dispensed as the index week for the first treatment window. We compared the odds of experiencing an event during that window with the odds of experiencing an event during the pre-treatment window to identify acute treatment effects. To evaluate the transience or stability of possible ADRs, we compared the odds of experiencing an event during the second treatment window to the odds of experiencing an event during the pre-treatment window. Patients were noted as having an event if they had a medical claim with the primary clinical diagnosis code of interest; we used only one event per time-window. Odds ratios between the referent and treatment windows were computed using conditional logistic regression. See Figure 2 for a summary of the two designs.
Of the 29,046 eligible patients in the study populations, 4,109 had a medical claim with a dementia diagnosis between 1/01/2003 and 12/31/2005. The AChEI-treated cohort consisted of 976 total users and 332 users with incident courses; of the latter, 224 were started on donepezil, 59 on rivastigmine, and 49 on galantamine. Because the numbers of incident users of specific AChEIs were small, we did not assess potential ADRs for individual drugs. In the AChEI-treated group the median duration of incident courses was 33.4 weeks with an interquartile range (IQR) from 15 to 68.5 weeks. The median proportion of weeks for which the AChEI-treated group was estimated to have access to the medication at least 1 day during the week was 100%, with an IQR of 95%–100%. The untreated cohort consisted of 2,968 patients who were diagnosed with dementia but did not receive medication to treat the disorder during the study period (Figure 3).
Note: Groups highlighted in blue met the inclusion criteria for this study and were the primary comparison.
Basic characteristics of the study population during the 6-month baseline period are presented in Table 2. Compared with the untreated population, incident AChEI users were slightly younger, had fewer HCUP comorbidities, fewer clinic visits, and a lower frequency of hospice care. Incident AChEI users also had a higher frequency of statin use and nursing home care. RxRisk-Vm scores and the average numbers of hospitalizations and ED visits were similar for AChEI users and non-users (untreated patients).
After propensity score matching for each clinical endpoint, the two groups were similar on all variables for each outcome-based cohort, except the average number of ED visits, which was slightly higher in the untreated matched groups for the evaluation of respiratory and hepatic episodes (Table 3). In general, the lack of statistically significant differences between the AChEI-treated and untreated groups on propensity-adjusted variables suggests a balance in measured covariates between treatment groups.
Table 4 presents the incidence densities per 100-person years and 95% confidence intervals for the complete untreated population and propensity-matched comparisons. Table 5 presents the hazard rates for all unadjusted and matched comparisons.
Crude analyses. In bivariate analysis (Table 5) we did not observe a higher rate of gastrointestinal episodes in the group treated with AChEIs compared to the untreated group. The rates of psychological episodes, respiratory episodes, hematological episodes, and hepatic episodes were slightly higher, but not statistically significantly, in the group treated with AChEIs compared to the untreated group. The rate of death in the group treated with AChEIs was significantly lower than in the untreated group.
Propensity-matched analyses. We observed significantly higher rates of gastrointestinal episodes (Hazard Ratio [HR]: 2.02; 95% CI: 1.28 - 3.2) and hematologic episodes (HR: 2.32; 95% CI: 1.47 - 3.67) in the AChEI-treated group than in the propensity-matched untreated group (Table 5). For psychological episodes, respiratory episodes, and hepatic episodes, we observed higher, but not statistically significant, rates in the AChEI-treated group than in the propensity-matched untreated group. We observed a weak and non-significant association between AChEI treatment and mortality.
Case-crossover analysis. In crossover analysis we did not observe increased odds of experiencing gastrointestinal events during either the first or second treatment windows. We observed an acute, but non-significant, effect of AChEI treatment on the odds of experiencing a psychological event during the first-treatment window; this was not sustained during the second-treatment window. We observed acute, but non-significant, effects of AChEI treatment on the odds of experiencing respiratory event and hematological events during the first-treatment window; both rates appeared to decrease during the second-treatment window. The acute effect of AChEI treatment on the odds of experiencing a hepatic event during the first-treatment window was imprecise and appeared to decrease during the second-treatment window (Table 6).
We developed a cohort-based framework for using population-based administrative data to identify known ADRs and to discover ADRs that may have gone unnoticed during clinical trials. We evaluated AChEI therapy in people with dementia, considering a composite of possible ADRs─ i.e., expected, suspected, unexpected reactions, and death ─to demonstrate that our analytic techniques produced expected results. We used propensity score matching and a within-subject design in an attempt to handle confounding. Our pilot study examined data from patients diagnosed with dementia for both cumulative effects of AChEI treatment and acute effects following initiation of AChEI therapy. We demonstrated this approach with Medicaid data from the state of Utah; nonetheless, the framework presented here can be transferred for use with other health insurer databases, including the Medicare Parts A, B, and D data now available.
A pervasive issue in pharmacoepidemiologic studies is confounding by indication15. This problem arises because factors that influence treatment choices made by clinicians also influence outcomes. Confounding by indication can bias the crude association between drug treatment and outcomes in either direction and with unknown magnitude. Propensity score models are one method used in pharmacoepidemiologic studies to balance measured confounders with the goal of making the treatment groups exchangeable.
In this study, we addressed confounding by indication by developing propensity score models for each study outcome. Theoretical confounders available in the data were included in each model to reduce bias. Before matching, the untreated group appeared to be frailer than the treated group; they had a higher proportion of hospice care, more comorbidity, and a lower proportion of statin users, which suggested less aggressive care because of poorer health. As one would expect, the unadjusted analysis made AChEI treatment appear protective against mortality when compared with the untreated group (HR: 0.66; 95% CI: 0.52 - 0.82), which is not supported by clinical trials or other observation studies16,17. After propensity and covariate matching we found no difference between the AChEI-treated and untreated groups (HR: 1.07; 95% CI: 0.74 - 1.54). This illustrates the importance of addressing confounding by indication when designing ADR surveillance systems.
An alternative approach to addressing confounding is to use inverse probability weighting (IPW) methods to model time-varying treatments and confounders. In simulation studies, these methods were less biased than conventional methods when time-varying confounding was present18. When allowing treatment to be time-varying, we observed gastrointestinal disturbance and discovered hematological disturbance; we noted the same findings as if follow-up began at initiation of drug treatment (data not shown). Future work should explore the presence of time-varying confounding and the benefits of using IPW methods to discover novel ADRs associated with drug treatments.
To evaluate possible acute and transient effects of AChEI treatments, we employed a type of case-crossover analyses. Typically in case-crossover analyses, events are compared between event and control time-windows for each individual. A major benefit of this within-subject design is that each person acts as his or her own control19,20. It also accounts for confounding by indication and other time-invariant and difficult-to-measure confounders. The drawback of such designs involves changes in treatment utilization that are influenced by health status or the study endpoints in question21. For example, when day-level drug utilization data are inferred from dispensing history, determining whether adverse effects are truly transient or the result of a decrease or discontinuation of drug treatment is difficult. Ultimately, we deemed the within subject analysis to be an excellent complement to the propensity score approach because of its ability to discover acute and transient effects and for its simplicity and ability to remove time-invariant confounding by indication.
The framework described here provides a structured approach for confirming expectations by evaluating known ADRs and discovering new ADR safety signals, such as the association we found between AChEI use and hematological disturbance. In support of the analytical effectiveness of these procedures, our approach confirmed an association with an expected reaction, gastrointestinal disturbances. The findings from the two study designs, however, were not consistent. Our inability to find an acute increase of gastrointestinal events in the first-treatment time window may be attributable to insensitivity of claims-based coding to identify symptoms of gastrointestinal disturbance.
Despite the fact that our approach detected a significant association with one expected reaction, gastrointestinal disturbance, it failed to identify a strong positive association with the second expected reaction, psychological disturbance. We did show a higher rate of psychological episodes in the propensity-matched analysis; nevertheless, the association was not statistically significant. We did; nonetheless, observe higher odds of experiencing psychological events in the first-treatment time window than in the pretreatment time window using the within subject design. Even though the higher odds was expected, it was not statistically significance. This result can likely be attributed to a combination of factors. First, is the low power in the within subject design and second may be insensitivity of claims-based coding to identify symptoms of psychological disturbance.
We discovered no clear associations between AChEIs and respiratory disturbance or death. In a recent sequence symmetry analysis, the initiators of AChEI had no detectable increased rate of complications of chronic airway disorders25. We found no clear evidence of an increase or decrease in mortality associated with AChEI treatment in published studies or meta-analysis to which to compare our results16.
Our analysis of unexpected reactions discovered a statistically significant positive association between AChEI treatment and hemotological episodes. Hematological events also appeared to be positively associated with early AChEI treatment. A detailed review of results with hematological event subcategories (not reported here) found that the rate of anemia was much higher in the AChEI-treated group than in the untreated group during the first 6 weeks of drug treatment. Further analysis is required to determine if this higher rate is causally associated with initiating anti-dementia drug treatment. At present, no known pharmacologic or empirical reasons can explain how AChEI drugs cause hematological toxicity.
The incidence of hepatic disturbance appeared to be higher in the treated group, although non-significant, in both the within subject and propensity matched design. Hepatotoxicity was a major safety concern with tacrine, which is the reason why it is no longer a commonly used; hepatotoxicity has not been reported for other AChEIs26. Larger observational studies are needed to determine whether an association between AChEIs and hepatotoxicity exists.
The results from this study are considered hypothesis generating rather than identifying causal treatment effects. Causal studies require validation of treatments, outcomes, and covariate classifications. Furthermore, causal studies require a stronger theoretical understanding and explication of the underlying causal relationships between the treatment and outcomes.
We compared AChEI-treated patients with an incident AChEI course of therapy, to an untreated cohort of patients with a dementia diagnosis. Other options were to compare directly the safety of AChEI products with one another or to compare the safety of AChEI therapy with the safety of other classes of medications used to treat such patients’ dementia. We were not able to compare individual drug products. Treatment with AChEIs is not directly comparable to treatment with memantine, a glutamaterginc N-methyl D-aspartate (NMDA) receptor antagonist, because memantine is typically not the first-line treatment for dementia; rather it is used in addition to an AChEI therapy, complicating any comparison.
In pharmacoepidemiologic studies, an untreated referent group can also be defined as patients with an incident course of a medication that is not associated with the indication or evaluated outcomes. This type of “active control group” is likely to be more similar to the treated group in regard to the activation of the health care system than the indicated but untreated group17. Drug dispensing indicates that the patient has activated the health system. In addition, prescription of a new medication is likely to result in closer monitoring and evaluation of an individual’s health status. The primary concern when comparing treated with untreated groups is under-recording of health conditions, making the members of the comparison group seem healthier than they really are, which can lead to overestimation of the effects of drug treatment.
Because of the multiple outcomes in this study, we were unable to identify a single medication that could yield comparable cohorts for all events. Instead, we used a dementia-related visit, not drug dispensing, as the index date for the untreated group. For both cohorts, the median amount of time to a clinic visit following the index date was 3 weeks, and the longitudinal visit process was also similar. These patterns suggest that health care access and followup may have been similar for the two groups.
Another limitation of this study is the small number of subject in the AChEI treatment group. This markedly limited our ability to confirm the expected adverse effects of AChEI treatment and discover adverse events that may have gone undetected in clinical trials.
The discovery of an association between a drug treatment and a theoretical reaction, an idiosyncratic reaction, or death is considered hypothesis generating or signal detection. Confirmation requires additional observational and possibly experimental studies. Ideally, discovered associations would first be confirmed or further characterized in large, disparate data sources to reproduce evidence of the association across different populations. In May 2008, the FDA published The Sentinel Initiative report to present the national strategy for monitoring medical product safety27. Their approach primarily establishes a nationwide health information network for confirmation of safety signals across multiple large databases. Additional observational studies along with richer clinical information such as electronic health records or prospectively designed studies, however, may be needed to characterize the causal relationship between a drug treatment and the adverse outcome.
The raw data are available upon request. IRB approval and signed Data Use Agreements with the Utah Department of Health. For more information, please contact the corresponding author Brian Sauer.
The primary ethical consideration is the privacy and confidentiality of patient data. Limited datasets were used that restricted the use of direct patient identifiers. Data are stored on secured servers and only shared according to IRB policy and state data use agreements.
Concept and Design: BCS, JRN, RR, MS
Acquisition of Data: WX, BCS
Analysis and interpretation of data: BCS, JRN, SS, SLW, JAS, WX, RR, KNL, MS
Drafting Manuscript: BCS, WX, JAS
Critical Revision of Manuscript: BCS, SLW, KNL
Statistical Analysis: BCS, SS
Obtaining funding: KNL, WX, MS
Administrative, technical or material Support: KNL
Supervision: MS, KNL
The original research was supported by an award from the Agency for Healthcare Research and Quality to RTI International, Contract No. HHSA 290 2005 0036 I. Brian C. Sauer was also funded by the Veterans Affairs Health Services Research and Development Career Development Award (RCD 06-300-2).
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
The authors thank the Utah Medicaid Program for providing the data and Zhiwei Liu for extracting the data and providing expert SAS consultation. We also extend our appreciation to Linda Lux, M.P.A., Jacqueline Amoozegar, B.A., and Loraine Monroe, all of RTI International, who provided dedicated support and assistance to the original project on which this work is based. The authors of this paper are responsible for its content. Statements in the paper should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services of a particular drug, device, test, treatment, or other clinical service.
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Competing Interests: No competing interests were disclosed.
Competing Interests: No competing interests were disclosed.
Competing Interests: No competing interests were disclosed.
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | |||
---|---|---|---|
1 | 2 | 3 | |
Version 1 29 Oct 14 |
read | read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)