The Blow Up—A Meta-analysis by Nissen and Wolski
The Blow Up—A Meta-analysis by Nissen and Wolski
In May 2007, an important meta-analysis by Nissen and Wolski was published in the New England Journal of Medicine in terms of cardiovascular (CV) safety of rosiglitazone.1 The article examined studies of rosiglitazone using published articles and the database of the manufacturer (GlaxoSmithKline [GSK]). The authors included studies of more than 24 weeks duration with a randomized control group not receiving rosiglitazone. To be included, studies needed outcome data for myocardial infarction (MI) and for CV death. Of 116 possible studies, 42 trials met their inclusion criteria.
Using a fixed effects model (Peto), the authors estimated that the odds ratio (OR) for MI was 1.43 (95 % confidence interval [CI] 1.03–1.98; p=0.03). The OR for CV death was 1.64 (95 % CI 0.98–2.74; p=0.06). Understandably, the study created a furore as it appeared to show an increase in MI risk for one of the more popular drugs used to treat diabetes. Given that earlier studies had shown favorable effects on multiple surrogate markers of CV risk, this was especially noteworthy. The increase in CV risk is especially important as rosiglitazone was popular and diabetes continues to explode in prevalence worldwide and has high CV risk.2
The authors note several limitations in their study. Among them was the absence of adjudicated endpoints for MI and CV death, except in the Diabetes reduction assessment with ramipril and rosiglitazone medication (DREAM) study.3 Moreover, time-to-event data were unavailable for these trials, preventing a hazard ratio (HR) calculation. As only summary data were available, it was impossible to know whether one patient had more than one event (i.e. MI and CV death) so no composite endpoint could be determined. Additionally, the authors acknowledged that these studies were not designed to study CV endpoints. Pooled data limited the kinds of analysis that could have been performed with source data. Definitions of events such as MI were not available for events that were reported primarily as serious adverse events. Most of the 42 studies analyzed were of short duration and small in size. These kinds of matters led to considerable uncertainty, which the authors acknowledged in terms of the scale of their effects. There are accordingly large CIs in this study. They also acknowledged that meta-analytic approaches are less convincing than large, prospective, randomized trials designed to focus on the events of interest such as MI and CV death. Partial Confirmations, Comparison Studies, and Analytical Problems
One early response to this study was an unplanned interim analysis of an ongoing randomized controlled trial of rosiglitazone and its CV effects by Home and colleagues to see if this trial should be stopped for safety reasons.4 The Rosiglitazone evaluated for cardiovascular outcomes (RECORD) trial used an open label design to determine the non-inferiority for CV safety when rosiglitazone was added to 2,220 patients with inadequate control of type 2 diabetes while on a sulfonylurea or metformin versus the combination of these two latter agents in 2,227 patients. The primary endpoint was a composite of hospitalization or death from CV causes. The analysis was performed after 3.75 years of follow-up. Of those studied with adjudicated primary endpoints, 217 patients in the rosiglitazone add-on arm and 202 patients in the control arm reached the adjudicated endpoint. This resulted in a HR of 1.08 (95 % CI 0.89–1.31). Similar data were found by including subjects with events pending adjudication (HR 1.11, CI 0.93–1.32). One secondary endpoint in the interim RECORD analysis was significantly positive and that was congestive heart failure (CHF). In the rosiglitazone arm, 47 patients were hospitalized for CHF versus 22 for the control group with a HR for both adjudicated and pending adjudication that was significant (HR for both combined 2.15, CI 1.30–3.57). The RECORD trial itself had some potential problems with its own analysis as the expected proportion of subjects with primary endpoint events (11 %) for adequate power was considerably higher than the actual proportion during the study (closer to 2.5 %), making the study have a higher likelihood of an indeterminate result for the primary endpoint.
Other papers have been published that have attempted to use alternative analytical approaches and see if they could confirm the Nissen paper or not. Concerns about the number of short-term studies incorporated into the original Nissen paper model approach used led Diamond and colleagues5 to analyze the statistical methods used by Nissen. They found additional problems that led them to conclude there was no ability to declare whether rosiglitazone had increased or decreased CV death or MI risk. Some of Diamond’s critique is related to the exclusion of trials (n=6) with no events in an unknown number of subjects, the lack of focus on trials that were rosiglitazone monotherapy versus control (which may be considered the cleanest kind of an analysis), the use of studies with active comparators with potential benefit to CV death or MI risk (e.g. metformin), the absence of data on stroke or non-CV death, and inadequate detail about the literature search methods used by Nissen. The primary focus of the critique by Diamond et al. was first on the use of a fixed model (Peto method) for analysis. The second statistical focus was on the choice of a Cochran Q test to justify pooled data from many heterogeneous studies and populations studied—in effect to show that they were nonetheless statistically not too heterogeneous to be lumped together.
This second critique can be exemplified by asking whether it was acceptable to pool rosiglitazone studies of patients with Alzheimer’s disease or psoriasis with those of patients with pre-diabetes and diabetes and tentatively concludes that these were potentially improper methods. Similarly, it was questioned whether pooling data from diabetes patients with and without heart disease was analytically incorrect as the diagnosis might interact with the endpoints of interest, such as CV death or MI.
The first critique in terms of the use of the Peto method may seem picky and arcane, but it has considerable relevance to whether their analysis supported the Nissen endpoints of MI and CV death. Therefore, in terms of the Diamond paper, by choosing six alternative model methods, none of the ORs for MI was increased and for the “nearly significant” (perhaps better termed not significant) increased risk for CV death, no other methods employed even came close to significance. This strong methodological critique suggested that while the matter of rosiglitazone and CV death risk was compelling, it was one that the Nissen analysis did not present a compelling case for increased risk for rosiglitazone. Diamond suggested that a return to the methods of randomized controlled trials would be far superior and felt the call for rapid regulatory action by Nissen was misplaced in emphasis because of uncertainty about the validity of the analytical approach. A subsequent meta-analysis6 of rosiglitazone and CV death risk examined similar data, but tried to avoid concerns raised about the Nissen paper. It restricted itself to longer-term studies, perhaps recognizing most CV endpoint studies (e.g. statin trials) do not show significant endpoint differences before 12–24 months. It also limited its analysis to patients with diabetes or pre-diabetes and thus minimized heterogeneity to enhance the valid pooling of data. Singh et al.6 included in their meta-analysis studies that were randomized controlled trials of rosiglitazone for prevention or treatment of diabetes. Studies in their analysis needed to be at least 12 months in follow-up and to have monitored adverse CV events with numerical data on all adverse events, not only MI and CV death. Only four studies out of 140 trials were selected after screening. These included the DREAM trial, the RECORD and A Diabetes Outcome Progression Trial (ADOPT) studies and one additional study. It is important to point out that the final results of RECORD had not yet been published or presented. In what appears to be a partial confirmation of the results of the Nissen study, they found a 43 % (CI 1.06–1.91) increased risk for MI, although it should also be noted that the numbers of events were small (94 out of 6,421 or 1.46 % as was absolute risk for rosiglitazone versus 83 out of 7,870, or 1.05 % absolute risk for controls). This mirrored the situation in the Nissen original analysis, which also had small numbers of events and low absolute risk. There was no confirmation of the trend toward increased CV death in this study (relative risk [RR] 0.9, CI 0.63–1.26) and again the number of events was small: 59 versus 72. However, there was an increased risk for CHF 2.09 (CI 1.52–2.99) again with small numbers and low absolute risks (102 versus 62 and 1.6 versus 0.8 %). It is important to draw a distinction between absolute and RR in these studies. A neglected area of discussion with these meta-analyses and other studies in the presentation of CV risk results is the overuse and emphasis on RRs and the minimization of absolute risks. The latter gives a much more balanced picture of risks and benefits than the RRs do. This problem was particularly egregious in the publicity that reached the general public. The headlines about the original Nissen analysis trumpeted summaries of 43 % increased risk for heart attack and 64 % increased risk for heart disease death. The absolute number of MI events was quite small between the original meta-analysis groups—MI was 86 out of 15,560 or 0.55 % for rosiglitazone and 72 out of 12,283 or 0.59 %. Such an inappropriate emphasis magnifies the perceived risk to an unwarranted degree. In a reductionist analogy, one could say that the risk with one treatment was doubled, but on the understanding that it went, for example, from one in a million to two in a million gives a profoundly different sense of the risk. Nonetheless, given that heart disease is the primary cause of death, disability and cost in type 2 diabetes, any increase in that risk is clearly highly undesirable and if correctly estimated should lead to avoidance of a therapy that increases such risk.
Other Randomized Trials Rosiglitazone Evaluated for Cardiovascular Outcomes
The final report of the RECORD trial7 was similar to that of the earlier interim analysis. The primary endpoint of this study was time to the first CV hospitalization or death secondary to CV causes. Hospitalizations were defined as admission for HF, cerebrovascular disease, cardiac revascularizations, amputation secondary to peripheral arterial disease (PAD), thrombotic events, and acute MIs (AMIs). These events were isolated by questionnaires and adverse event reporting at subject follow-up visits. Of note, the study was underpowered to determine non-inferiority of rosiglitazone. The event rate per year reached was only 2.5 % (not the expected 11 %), significantly reducing the power of the study’s conclusions. RECORD found no difference in CV deaths between the rosiglitazone group and the comparator arm (rosiglitazone group HR 0.84, 95 % CI 0.59–1.18; p=0.32). There were also no differences in all-cause mortality between the two groups (rosiglitazone group HR 0.86; 95 % CI 0.68–1.08; p=0.19).
There was no increase in the primary endpoint in the TZD group (HR 0.99, 95 % CI 0.85–1.16; p=0.93). The HR for MI was slightly higher in the rosiglitazone group but was not statistically different (HR 1.14, 95 % CI 0.80–1.63; p=0.47). A secondary composite endpoint of stroke, MI, and CV death showed no difference between rosiglitazone and the comparator arm (rosiglitazone group HR 0.93, 95 % CI 0.74–1.15; p=0.50). There was a statistically significant increase for HF in the rosiglitazone group. There were 60 CVDs in the rosiglitazone group (out of 2,220 subjects) and 71 deaths in the metformin/sulfonylurea group (out of 2,227 subjects). These results must be viewed in light of the reduced power of the trial, as participants in the rosiglitazone arm did not reach the pre-defined event rate. As reviewed by this author,8 this may have been due to increased use of statins in the rosiglitazone group (55 versus 46 % in the comparator group). With this caveat in mind, the RECORD trial did not show an increased risk for MI and total CV death as had been suggested by the meta-analyses released two years earlier.2,6 Bypass Angioplasty Revascularization Investigation 2 Diabetes
The first Bypass angioplasty revascularization investigation (BARI) study, released in 1996, found that patients with diabetes treated with coronary artery bypass grafting (CABG) had a statistically significant improvement in five-year survival compared with those treated with percutaneous coronary intervention (PCI) (80.6 versus 65.5 %; p=0.003).9 These findings led to the six-year, multicenter BARI 2D trial, released in June 2009 (see Table 1).10 In BARI 2D, 2,368 subjects with type 2 diabetes and angiographically documented CHD were randomized to undergo coronary revascularization (PCI or CABG) or intensive medical therapy (IMT). Furthermore, patients received insulin-sensitization (IS) therapies (metformin, TZDs) or insulin-provision (IP) therapies (insulin, sulfonylureas), with a goal of attaining an glycated hemoglobin (HbA1c) level of less than 7 %. The primary endpoint in BARI 2D was all-cause mortality, and a secondary endpoint included major CV events (death, MI, and stroke).
Events were adjudicated by a clinical events committee. A total of 82 % of patients had symptomatic myocardial ischemia, and the average duration of diabetes was approximately 10 years. Recruitment took longer than expected, and the study was extended by 1.5 years, increasing the average follow-up to 5.3 years. Those receiving IS medications (n=977) were using metformin (74.6 %) and TZDs (62.1 %). A total of 55.1 % of all patients were taking rosiglitazone (with the remaining 7 % of patients taking, presumably, pioglitazone, although this is not explicitly stated). Subjects receiving IP agents (n=967) most frequently used insulin (60.7 %) and sulfonylureas (52 %). Throughout the study, those receiving IS medications had statistically significant lower HbA1c compared with those on IP drugs (7 versus 7.5 %; p<0.001). The body mass index at three-year follow-up was significantly lower in those on sensitizers as well (31.7 versus 32.5kg/m2; p=0.003), and their high-density lipoprotein cholesterol concentrations were significantly higher (42 versus 40 mg/dl; p<0.001). All-cause mortality, the primary endpoint, did not differ between those on IS agents and those taking IP drugs. Five-year survival was 88.2 % among patients on sensitizers versus 87.9 % in the provision group (difference of 0.3 %, 95 % CI -2.2 to 2.9; p=0.89). Rate of freedom from major CV events (death, MI, and stroke) was also not significantly different (difference of 2.4 %, 95 % CI -1.2–6.0; p=0.13). As expected, there was less severe hypoglycemia in the IS group (5.9 versus 9.2 %; p=0.003). In the patients randomized to early revascularization by CABG who were on IS therapies, there were significantly fewer major CV events at five years (using Kaplan-Meier estimates) versus those receiving IMT (18.7 versus 32 %; p=0.002). This difference was not seen in those on IP agents. There was borderline significance between the sensitization and provision groups, with the former having a non-significantly = fewer number of major CV events when undergoing revascularization (p=0.07). In summary, the BARI 2D study showed no significant differences in the rates of major CV events and all-cause mortality between patients with type 2 diabetes randomized to early revascularization or IMT, and to those using an IS strategy (metformin, TZDs) for glucose control, and those using an IP strategy (insulin, sulfonylureas). However, the study findings did suggest that individuals undergoing early revascularization with CABG who were also taking IS agents had fewer adverse cardiac outcomes compared with those taking IP therapies. Although we cannot generalize these results to the effects of TZDs alone on CV morbidity and mortality, they should be taken into account when determining the safety and clinical use of this drug class. It is hoped that future analyses of the BARI 2D data will tease apart the effects of the TZDs, specifically on CV risk. The Action to Control Cardiovascular Risk in Diabetes Study and the Veterans Administration Diabetes Trial It should be briefly noted that both the Action to control cardiovascular risk in diabetes (ACCORD) study11 and the Veterans Administration diabetes trial (VADT)12 tested whether CV death risk was reduced by intensive glycemic control. In their protocols there was heavy use of rosiglitazone with intensive glycemic treatment. The 22 % increase in mortality in the ACCORD trial and no benefit to CV death risk in either study raised questions about whether the failure to show benefit might be explained by heavy use of rosiglitazone. Neither study could confirm this by post hoc statistical analyses. The ability to do so in such complex treatment studies is difficult at best and perhaps unachievable.
New Meta-analyses and Other Studies
A recent retrospective analysis cohort study13 of 227,571 Medicare beneficiaries over 65 years of age who started treatment with rosiglitazone or pioglitazone from July 2006 to June 2009 was followed for up to three years after TZD initiation to compare the effects of pioglitazone and rosiglitazone on AMI, stroke, CHF and death, and a composite of these. The assessment used incidence rates by TZD, attributable risk, and numbers needed to harm, as well as Kaplan-Meier plots of time to event, and Cox proportional HRs for time to event, adjusted for potential confounding factors, with pioglitazone as reference. The results of this study found 8,667 endpoints in the study period. The HR for rosiglitazone compared with pioglitazone was 1.06 (95 % CI 0.96–1.18) for AMI, 1.27 (95 % CI 1.12–1.45) for stroke, 1.25 (95 % CI 1.16–1.34) for heart failure, 1.14 (95 % CI 1.05–1.24) for death, and 1.18 (95 % CI 1.12–1.23) for the composite of AMI, stroke, heart failure, or death. The attributable risk for the composite endpoint was 1.68 (95 % CI 1.27–2.08) excess events per 100 person-years of treatment with rosiglitazone. The numbers needed to harm was 60 (95 % CI 48–79) treated for one year. While AMI was not significant, the composite endpoint including it was significant for increased risk. As is shown in Table 1, Nissen and Wolski have recently updated their prior meta-analysis14 from 2007. With the inclusion of 56 studies, they found a marginally significant although smaller OR (1.28 versus 1.43; p=0.04). The previous study had detected a strong trend to increased CV death risk (1.6; p=0.06), but the updated meta-analysis14 did not detect a significant trend. In order to improve their methods in response to critiques, they performed their analysis with and without use of the RECORD trial, which had a small impact on their findings. They also carried out an analysis with and without studies with no events and again found no significant differences in overall assessment of MI risk.
Mannucci and colleagues have performed meta-analyses of rosiglitazone and pioglitazone and written reviews about these two agents.15,16 Contrary to other such analyses they have argued that studies of neither drug have convincingly shown either benefit or harm to the CV system. In their analysis they studied 164 trials with a duration of more than four weeks (retrieved from Medline, and identified unpublished studies on www.clinicaltrials.gov). The trials found 42,922 and 45,483 patient-years for rosiglitazone and comparators, respectively, to analyze. Their results shows that the OR for all-cause and CV death mortality was 0.93 (0.76–1.14) and 0.94 (0.68–1.29). The risk for non-fatal MI and CHF were 1.14 (0.90–1.45) and 1.69 (1.21–2.36). Therefore, CHF risk was increased, but no other endpoints including ischemia or mortality were increased, and they also note that increased CHF risk occurs when rosiglitazone was combined with insulin.
In a retrospective analysis that sought to compare the relative CV safety of several diabetes drugs (sulfonylureas, metformin, and the two glitazones). Pantalone and colleagues17 sought to examine the safety of these agents for prevalence of coronary heart disease, CHF, and allcause mortality in a database of 20,450 from an electronic health record in the Cleveland Clinic database. In their study they found that there was a reduced risk for CHF and mortality for metformin, respectively, Cox proportional HR of 0.76 (CI 0.64–0.91) and 0.54 (CI 0.46–0.64) when compared with sulfonylureas. Pioglitazone also was associated with reduced mortality risk compared with sulfonylureas with HR of 0.59 (CI 0.43–0.81). No other differences in risk for CAD, CHF, or mortality were noted in this analysis. Certainly, one point that should be raised by such an analysis is whether the increase in events with glitazones is significantly different from one another. Furthermore, the comparators or other combined therapies such as sulfonylureas and metformin need to be considered as not neutral but possibly beneficial or harmful for CV death risk in themselves. For example, comparison of a TZD to metformin that are unfavorable may be due to the benefit of metformin and not harm from TZD.
Erdmann et al. published a review in 2009 in which they compared the evidence for and against rosiglitazone in light of the evidence for and against pioglitazone for CV death safety,18 including their own earlier analyses of the Prospective pioglitazone clinical trial In macrovascular events (PROactive) trial.19–20 Their analysis emphasizes the differences between these two agents and also the unknowns or partial data in terms of other diabetes oral medications (including the dipeptidyl peptidase-4 [DPP4] inhibitor incretins sitagliptin and vildagliptin), α-glucosidase inhibitors, metformin, and sulfonylureas. One of the figures in their review (see Figure 1) illustrates the differences between pioglitazone and rosiglitazone in terms of CV death risk and also illustrates the overlap between studies showing benefit or harm and those revealing uncertainty about CV death risk, which is a recurring theme of this review.
In a retrospective inception cohort study published recently by Wertz et al.21 directly comparing rosiglitazone and pioglitazone with a propensity score-matched population that included mortality data, there were no differences found between these two agents in 36,628 patients in AMI, CHF, or death. For rosiglitazone, 602 subjects (4.16 %) had an AMI, CHF, or death compared with matched pioglitazone patients of whom 599 (4.14 %) had any of these CV death events. The HR in this observational study was 1.03 (CI 0.91–1.15; p=0.666) from the index date January 1, 2001 until December 12, 2005. The Food and Drug Administration Situation The US Food and Drug Administration (FDA) held hearings on July 13–14, 2010, which generated much publicity and left many feeling unclear in terms of what the effective outcome of the review of rosiglitazone and its safety were. Clearly based on recent evidence, some of which is given above, the committee of 33 people of different persuasions formulated what seemed to be a mixed message as reviewed by Cliff Rosen in a recent editorial in the New England Journal of Medicine.22 Of the committee members, 12 voted to remove rosiglitazone from the market, 10 (including Dr Rosen) voted for much stricter control over prescriptions, seven voted for further warnings, three felt no changes were necessary, and one abstained. Although the message of the committee may seem mixed and ambiguous, Dr Rosen’s narrative suggests it is clear that more action needs to be taken because of the persistent concern raised by the remaining apparent increased risk for ischemic heart disease related to rosiglitazone. He argues that neither the marketplace nor FDA warnings have markedly influenced the sales of rosiglitazone, which remain steady, although substantially less than pioglitazone.
Several matters of ethics and scientific uncertainty are intertwined in the debate over rosiglitazone and its CV safety. The science suggests residual uncertainty whether rosiglitazone increases the risk for MI events despite credible concerns. The FDA mandated Thiazolidinedione Intervention With Vitamin D Evaluation (TIDE) study with not be completed or powered to provide the certainty needed. What action should be taken in the absence scientific certainty? Decision-makers should consider that RR compared with other therapies seems strongly suggestive of residual problems. It may be best to take a primum non nocere (first do no harm) approach. Do we harm patients more by leaving the drug on the market in the face of uncertain risk or more by removing an effective drug that may have more risk than seems reasonable, but less worrisome than alternatives? It seems necessary to act despite the scientific uncertainty, while recognizing that weak science makes for problematic policy decisions. Comparing rosiglitazone and pioglitazone one would say that there are fewer concerns about the latter drug for CV risk but has concerns about CHF and bone fracture risk probably equal to rosiglitazone. Whether it is appropriate to start new patients on rosiglitazone is questionable. The FDA (so far in its stance on the enrolment of patients in the TIDE trial) appear to be saying no unlike their stance in terms of pioglitazone and other non- TZD alternatives. No final decision from the FDA has yet been taken but inevitably it seems that rosiglitazone does not have a rosy future and that a preference for alternatives in the form of safer TZDs or peroxisome proliferator-activated receptors (PPARs) can be seen. Selective modulators seem to be the way of the future.