* Most papers presenting the benefits of statins have been done by the Oxford Cholesterol Treatment Trialists (CTT) team. They tend to present benefits of statins in relative risk terms – with claims of benefits of approximately 20% being typical.
* This week’s paper set out to evaluate the absolute risk reduction for relationships between statins, LDL-cholesterol and deaths/disease.
* The authors used the same trials that the CTT use and found potential for bias in a number of areas, not least that all of the included trials were funded, in part or wholly, by the pharmaceutical industry.
* The study examined all-cause mortality, heart attacks and strokes for all trials together and then separately examined trials that included people who had previously had a heart event or not. The relative risk reductions ranged between 9% and 38%, which seems large. The absolute risk reductions ranged between 0.3% and 2.2%, which doesn’t seem large.
* The authors conducted a number of tests that led to concerns about the reliability of pooling these trials given their differences and inconsistent definitions of key terms.
* The authors also tried to establish what role reductions in LDL-Cholesterol played in any benefits from statins. They found no consistent impact of reducing LDL-cholesterol.
* Patients are presented with the information that taking a statin (for 4.4 years on average) would reduce their risk of having a heart attack by 29% (relative risk). The same information could be presented as 77 people would need to take statins for approximately 4.4 years on average to prevent 1 heart attack (absolute risk).
* This paper was simple and powerful and should change patient-doctor consultations, but I suspect that it won’t.
I need to declare an interest for this week’s note. I know the second author, Dr Maryanne Demasi. We have met at a couple of conferences, and we email each other with research queries from time to time. I don’t know the lead author, Dr Paula Byrne, or any of the other authors, but I’m impressed by whoever thought of this study. It’s one of those where you wonder why it hadn’t been done sooner. The paper was called “Evaluating the Association Between Low-Density Lipoprotein Cholesterol Reduction and Relative and Absolute Effects of Statin Treatment. A Systematic Review and Meta-analysis” (Ref 1).
The aim of the study was brilliant in its simplicity – to evaluate the absolute risk reduction for relationships between statins, LDL-cholesterol and deaths/disease. The benefits of statins and LDL-cholesterol reduction are habitually presented in relative risk terms. For example, this paper called “Efficacy and safety of statin therapy in older people: a meta-analysis of individual participant data from 28 randomised controlled trials” reported in the abstract “Overall, statin therapy or a more intensive statin regimen produced a 21% proportional reduction in major vascular events per 1·0 mmol/L reduction in LDL cholesterol” (Ref 2).
Later in the same paper, a side effect of statins was reported as follows: “Statins have been estimated to increase the risk of myopathy (defined as muscle pain or weakness combined with large increases in blood concentrations of creatine kinase) typically by one case per 10,000 patients treated with statins per year.” The benefits of statins were thus presented in relative risk terms (21%). One side effect was reported in absolute risk terms (1 in 10,000 in a year). That’s clearly disingenuous. The side effect was also carefully worded to be an extreme one (myopathy) rather than a common one (muscle pain). The patient information leaflet reported “joint pain, muscle pain and back pain” as a common side effect, which was defined as “may affect up to 1 in 10 people” (Ref 3). The patient leaflet reports absolute risk.
Byrne et al searched academic databases for large randomised clinical trials that examined the effectiveness of statins in reducing total mortality and cardiovascular outcomes. Inclusion criteria included: a planned study duration of two or more years (long enough); more than 1,000 participants (large enough); and the study needed to report absolute changes in LDL-Cholesterol levels. The trial interventions were statins vs placebo or usual care. Participants were men and women older than 18 years.
Maryanne has worked at the Nordic Cochrane Collaboration – the one centre that produced unbiased research until that too got corrupted. The research team clearly had Cochrane level discipline, as research protocols were thoroughly followed and documented throughout the study and paper. The protocol had originally intended to extract data on major vascular events and major coronary events. However, the researchers found that these outcomes were not consistently defined across trials, leading them to conclude that producing a meta-analysis of differently defined terms would be inappropriate. They found similar difficulties with the definitions of cardiovascular death and cardiovascular events and thus focused their analyses on all-cause mortality, spontaneous myocardial infarction (heart attack) and spontaneous stroke. The word spontaneous excluded procedure related events. E.g., someone can have a stroke following an operation to prevent a stroke (this is more common than people might realise – it happened to a close relative of mine). Excluding these focuses analysis on objective events, independently of procedures.
The database search returned 21 trials to be included in the analysis. The trials were listed in the supplemental online content (Table e1 in the supplemental is an excellent summary of the large enough and long enough statin trials and the headline details on each). For those of you who know your statin trials, the 21 were: 4S 1994; WOSCOPS 1995; CARE 1996; LIPID 1998; AFCAPS/TexCAPS 1998; LIPS 2002; HPS 2002; PROSPER 2002; ALLHAT-LLT 2002; ASCOT-LLA 2003; ALERT 2003; CARDS 2004; 4D 2005; ASPEN 2006; MEGA 2006; SPARCL 2006; CORONA 2007; JUPITER 2008; GISSI-HF 2008; AURORA 2009; and HOPE-3 2016. The Oxford Cholesterol Treatment Trialists (CTT) are the main producers of meta-analyses of statin trials. These are the same trials that they commonly use in their papers to make relative risk claims.
The 21 trials included similar numbers of primary prevention trials (7), secondary prevention trials (6), and trials that included participants from primary and secondary prevention populations (8). Primary prevention refers to participants who have not had a cardiovascular event and the trial is trying to prevent the first (primary) event. Secondary prevention refers to participants who have had a cardiovascular event and the trial is trying to prevent the second (secondary) event.
The 21 trials achieved LDL-Cholesterol differences between the statin and placebo groups ranging from 17mg/dL (0.44 mmol/L) to 68 mg/dL (1.75 mmol/L) for each individual trial. The CTT papers usually make their relative risk reduction claims on the basis of each 1 mmol/L reduction in LDL-C. Of the 21 trials, 12 achieved a reduction of 1 mmol/L or greater, so this level of reduction is not achieved in all trials. The average trial follow-up period was 4.4 years, ranging from 1.9 to 6.1 years. The number of trial participants ranged from 1,255 (trial 4D 2005) to 20,536 (trial HPS 2002).
A key part of the process for meta-analysis is to examine possible bias in the trials being pooled. It’s a way of reviewing how robust the pooled findings are. If the original trials are of poor quality, then pooling them together amplifies that poor quality. The bias assessment in the Byrne et al paper reported some concerns related to the 21 trials. In the 4D trial, it was unclear from the protocol how the authors planned to measure or analyse fatal stroke. In the ASPEN trial, there were changes to the inclusion criteria for participants after two years. (However, it was unclear if this would have introduced bias.) Four trials (JUPITER, CARDS, AFCAPS/TexCAPS, and ASCOT-LLA20-23) were terminated early, and this may have been a source of bias. The researchers also noted that all of the included trials were funded, in part or wholly, by the pharmaceutical industry.
A meta-analysis was conducted on 19 of the 21 trials that reported data on all-cause mortality and 18 trials reporting data on heart attacks and strokes. The absolute risk reduction (ARR) was 0.8% for all-cause mortality, 1.3% for heart attacks, and 0.4% for stroke. The relative risk reduction (RRR) for all-cause mortality was 9%, 29% for MI, and 14% for stroke.
Byrne et al then did separate meta-analyses for primary and secondary trials to see if the results were different for those two groups of people. They found that the ARR was 0.6% for all-cause mortality, 0.7% for heart attack, and 0.3% for stroke in primary prevention. The ARR was 0.9% for all-cause mortality, 2.2% for heart attack, and 0.7% for stroke in secondary prevention. The RRR was 13% for all-cause mortality, 38% for heart attack, and 24% for stroke in primary prevention and 14%, 27%, and 13%, respectively, in secondary prevention. I’ve presented these data in a table below.
Examining the results
The researchers undertook further assessments of the findings:
1) Testing true effects.
There is an output of meta-analysis called I2. It measures something called heterogeneity. Think of homogeneity as ‘same’ and think of heterogeneity as ‘different.’ (Homosexual meaning attraction to the same sex and heterosexual meaning attraction to the different sex helps us to remember this).
A certain amount of the variation we find must be due to sampling error because we didn’t sample the whole population. We don’t care about sampling error. We want true error. Heterogeneity aims to measure that true variation.
We don’t want to find evidence of heterogeneity i.e. we want heterogeneity to be low. If there is little heterogeneity, what we observe is likely the effect that we would observe in the general population. If there is much heterogeneity, what we observe is far less likely to be reflective of the general population. We want I2 to be close to zero.
The Cochrane handbook advises that an I2 of 30-60% may represent moderate heterogeneity and 50-90% may represent substantial heterogeneity (Ref 4). There are 18 results in the table above. Byrne et al reported that 8 of these had an I2 above 50%, while 6 had an I2 below 30%. For this reason, their conclusions contained the caveat that “the presence of significant heterogeneity reduces the certainty of the evidence.” (The researchers also used another test for heterogeneity – I said this was thorough).
2) Testing the role of LDL-cholesterol.
The researchers undertook a statistical technique called meta-regression to assess the impact of reductions in LDL-cholesterol on relative and absolute effects. Findings were inconclusive with the confidence intervals all indicating that the results could have happened by chance. The proportion of between-study differences explained by LDL-C ranged from 0% to 14%. i.e., there was very little, if any, association between the size of LDL-C reduction and the size of the statin effect.
The researchers then conducted further meta-regressions, which adjusted for things like the number of events, or study population, and length of trial. These tests revealed inconsistent results. Some association was found for the relative risk effects on all-cause mortality and stroke, but not for heart attacks. Some association was found between the size of LDL-C reduction and the absolute risk effect on stroke, but not for all-cause mortality or heart attacks.
The Cholesterol Treatment Trialist papers consistently conclude that taking statins has large (c. 20%) relative risk benefits and that these benefits are per 1·0 mmol/L reduction in LDL-Cholesterol. Byrne et al found very small absolute risk benefits and no consistent impact of reducing LDL-cholesterol.
The researchers set out to compare the absolute and relative risk reductions for total mortality and cardiovascular events achieved with statins and to explore whether differences were explained by the amount of LDL-C lowering. They found that there were benefits, albeit the absolute benefits being a fraction of the relative numbers. However, they also found that the heterogeneity analysis suggested that pooling the studies may have been ill advised and the results from pooling statin trials may not be reliable. This is an important finding in itself.
Examining primary and secondary trials separately might have explained the trial differences. The researchers might have found that when they looked at primary trials alone and secondary trials alone, the differences and heterogeneity fell away. But this didn’t happen. The differences stayed – hence the concern about the reliability of any findings.
The researchers found weak and inconsistent relationships between LDL-Cholesterol reductions and outcomes (deaths or events). They made the point that if the magnitude of LDL-C reduction is associated with reduction in outcomes, it should be consistent. If reducing LDL-C reduces the relative risk of all-cause mortality, it should reduce the absolute risk of all-cause mortality and it should reduce the risk of heart attacks too. These relationships were inconsistent, which undermined LDL-C as the mode of claimed benefit. Given that the claimed relationships between LDL-C reduction and health benefit underpin current cholesterol clinical guidelines, this is a very important finding.
As regards the original aim – to understand the difference between absolute and relative risk reductions, the researchers spelled out what the difference could mean in a patient-doctor conversation. Taking heart attacks in all studies as an example, the absolute risk difference was 1.3%, the relative risk difference was 29%. The absolute risk number translates to a Number Needed to Treat of 77. i.e., 77 people would need to take statins for approximately 4.4 years on average to prevent 1 heart attack. If that were weighed up in a patient-doctor conversation with the patient leaflet information, the patient may come to a different choice about statins. The doctor should say “you have a 1 in 77 chance that you will be the person who avoids a heart attack if you take statins for approximately 4.4 years. You have a 1 in 10 chance of the side effects in the patient leaflet extract above.” (And there are many more side effects listed in the patient leaflet – less common but more serious).
This paper should change the patient-doctor discussion in consulting rooms world-wide. Sadly, I suspect that it won’t.
Ref 1: Byrne et al. Evaluating the Association Between Low-Density Lipoprotein Cholesterol Reduction and Relative and Absolute Effects of Statin Treatment. A Systematic Review and Meta-analysis. JAMA Internal Medicine. 2022. https://jamanetwork.com/journals/jamainternalmedicine/article-abstract/2790055
Ref 2: Armitage J, Baigent C, Barnes E, et al. Efficacy and safety of statin therapy in older people: a meta-analysis of individual participant data from 28 randomised controlled trials. The Lancet 2019. https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(18)31942-1/fulltext
Ref 3: https://www.medicines.org.uk/emc/files/pil.5241.pdf