Imagine that a doctor said: “I’d like you to take this pill. It will reduce your risk of dying by 30%.” You’d pretty seriously consider it, if not jump at the chance…
I happened to be in the car, with Radio 4 tuned in, last Wednesday (3rd Feb 2016) at 3.30pm. Dr. Mark Porter’s programme, Inside Health (the transcript is below the recording), had just started and I heard a researcher saying that his trial had been stopped just over three years into a five year study because the benefit was already so dramatic. This is the holy grail of randomised controlled trials – ‘we just had to stop the trial because it was unethical not to treat the control group because the intervention group was doing so well’. Having seen this happen more than once, it can also happen, funnily enough, when gaps that had emerged between the two groups start to narrow…
The researcher being interviewed was Dr. Paul Whelton. Whelton is the chair of the SPRINT (Systolic blood PRessure INtervention Trial) steering committee. The full paper for the trial can be seen here. (In Figure 3, the data available at four year shows the intervention and control group lines closer together than at the time the trial was stopped; funnily enough).
The trial enrolled over 9,000 people in the US between the dates of November 2010 and March 2013. There were many inclusion and exclusion criteria. These are very important, because they define the profile of people to which the results can apply. All participants had to be over the age of 50. And they had to have systolic blood pressure (the first number of the two e.g. the 140 in 140/90) between 130 and 180. And they had to have had a significant risk factor for cardiovascular disease (already had cardiovascular disease, already got chronic kidney disease and/or had a 10 year risk of 15% of higher on the Framingham scale). Two exclusion criteria were listed – the participant must NOT have diabetes and they must NOT have already had a stroke.
The aim of the trial was quite interesting – it was not to see if blood pressure medications were better than not taking medications. It was to see if what they called “intensive” blood pressure (BP) lowering treatment (to medicate people to get them below 120 systolic BP) was better than “standard” BP lowering treatment (to medicate people to get them below 140 systolic BP). The goal of this trial, therefore, was to see if giving more meds was better than giving fewer. The idea of not giving meds at all was not part of this trial.
The trial participants were also interesting – older people and sicker people – designed to ensure that there would be a decent number of deaths in a short period of time. This is not underhand – if the trial outcome is deaths, you need people to die!
4,678 were analysed for the intensive treatment group; 4,683 were analysed for the standard treatment group. At the time the trial was stopped, 155 people in the intensive treatment group had died and 210 people in the standard treatment group had died. Table 2 in the full paper has the statistics on this. This was presented as a Hazard (risk) Ratio of 0.73, which can be presented as a 27% difference between the two interventions. Indeed, Whelton presented this on Inside Health as “In fact their total mortality was reduced by close to 30% and I’ve never been in a trial where we’ve seen that before.”
That’s relative risk, which I have frequently explained as highly misleading. The absolute risk of dying in any 1 year of the study was approximately 1 in 100. In the intensive treatment group 1.03 people in 100 died; in the standard treatment group 1.37 people in 100 died. That’s 10 people in 1,000 vs. fewer than 14 people in 1,000. That’s hardly “Stop the trial! People are dropping like flies!”
Who didn’t benefit
Then is when it gets even more fun. Figure 4 in the paper is called a forest plot or a blobbogram. I kind of like the second name. All you need to look for in a blobbogram is whether each horizontal line touches or crosses the vertical line, which is drawn at 1.0. Any horizontal line that touches or crosses that vertical line is NOT significant. It could have happened by chance. You should ignore it.
The blobbogram in this paper tells us that, the intensive treatment made a significant difference for:
– all participants grouped together;
– people who didn’t have previous chronic kidney disease;
– people 75 years old or older;
– non black people;
– people who didn’t have previous cardiovascular disease; and
– people whose starting blood pressure was under, or equal to, 132.
The intensive treatment made NO significant difference for:
– people who did have previous chronic kidney disease;
– people under 75 years old (that’s 82% of participants for starters);
– black people;
– people who did have previous cardiovascular disease;
– people whose starting blood pressure was greater than 132 (that was 66% of people).
For the majority of people, therefore, there was no benefit from intensive treatment. (As an aside, is it surprising that lowering BP to below 140 makes no difference to people whose starting BP is already below 132?!)
Thanks to Dr. Margaret McCartney on Inside Health, the programme discussed the numbers needed to treat to impact one person. In McCartney’s words: “you had to treat 90 people in order to delay one death over the three year period of the trial. Now that is important but it means that 89 out of those 90 people did not get benefit from having their death delayed and were at risk of just getting the side effects from the intervention. Is that worth it or is it not? That’s for the patient to decide, not for me.”
Which brings us nicely on to…
The down sides
The side effects were serious. A Serious Adverse Event is defined as a “Fatal or life threatening event, resulting in significant or persistent disability, requiring or prolonging hospitalization, or judged important medical event.”
The paper reported: “Serious adverse events of hypotension, syncope [fainting], electrolyte abnormalities, and acute kidney injury or acute renal failure… occurred more frequently in the intensive-treatment group than in the standard-treatment group.” “A total of 220 participants in the intensive-treatment group (4.7%) and 118 participants in the standard-treatment group (2.5%) had serious adverse events that were classified as possibly or definitely related to the intervention (hazard ratio, 1.88; P<0.001).”
If you want to report deaths rates as 30% better than the standard treatment group, you should also report side effects as double that of the standard treatment group. Presenting benefit as 30% (relative) and the side effects as 4.7% (absolute) is naughty at best and deliberately misleading at worst. Some of those side effects are serious and nasty as well – acute kidney injury or acute renal failure? What would happen if the trial were kept to five years long, or extended for longer? Would significant numbers of people in the intensive treatment group die of kidney failure? How many will end up on lifelong dialysis as a result of this trial?
The honest patient conversation
This kind of trial informs medical policy – it is intended to. That’s why drug companies spend so much money funding research, researchers and research institutions. The messages that are best left in the minds of the medical world are 1) the trial stopped early i.e. you must get your patients on this treatment asap or you will be unethical and 2) there was a 30% difference in mortality i.e. you must get your patients on this treatment asap or you will be unethical.
Then the doctor conversation can be: “I’d like you to take this pill. It will reduce your risk of dying by 30%.”
As McCartney suggested, the good doctor sets out the facts in an understandable way and lets the patient decide if absolute benefit is worth the known (side effect) risk. The honest doctor conversation in this case – only even to be considered with patients over the age of 50 – would be:
“If you are male, or over the age of 75, or have no previous chronic kidney disease, or are not black, or have no previous cardiovascular disease, or have current systolic blood pressure under 132, AND over the age of 50 (because we didn’t test younger people), AND don’t have diabetes (because we didn’t test for people with that), AND haven’t had a stroke (because we didn’t test for people with that) … if you then take 3 pills a day (that was the actual number of pills administered) every day for 3.26 years, you have a 1 in somewhere up to 90 chance (Note 1) of delaying death i.e. you might not die within the 3.26 years – you might die the day after, or sometime after, we don’t know. Oh, and by the way, you double your (relative) risk of some serious side effects including acute kidney failure and you might faint here and there, so don’t idle at the top of staircases.”
Now who would jump at the meds?!
One of the key findings from this meta-analysis was that you can’t take six studies of 2,500 sick men and set dietary guidelines for entire populations. (Except that this is what happened – oh – and the studies themselves did not support the guidelines introduced). Research is said to lack “generalisability” if it is undertaken on, and/or shows results for a small or specific part of the population. Put simply, you can’t study people over 50, find significance for men or over 75s only and then apply guidelines to entire populations…
When McCartney joined the debate, so did Tony Heagerty, introduced as “Professor of Medicine at University of Manchester, who has a special interest in treating high blood pressure.”
Closing the debate, Porter asked Heagerty the key question: “Well this is the big question isn’t it Tony – does this one trial, which has come up with this finding and suggestion that we should be aiming at 120 – is that enough to change our current guidance and practice on it?”
Heagerty replied “I believe it does actually… the National Institute for Health and Clinical Excellence scoped out or looked for a reason to change guidelines in 2015 and the guideline writing committee felt there was no new evidence. I think that in 2016 the British guidelines will have to be readdressed with regard to targets for looking at effective blood pressure control.”
The paper’s own blobbogram confirmed that there was no significant difference for people under 75 years old (82% of the study participants). There was no significant difference for people with previous chronic kidney disease, or people with previous cardiovascular disease. There was no significant difference for women. There was no significant difference for black people and, perhaps most incredibly of all, there was no significant difference for people whose starting blood pressure was greater than 132 (66% of participants).
So there are calls for UK wide guidelines to be changed for all people when absolute risk benefit vs. Serious Adverse Effect cost barely holds up for white men over 75 with no previous CVD and already low blood pressure.
This is when John McEnroe’s legendary comments springs to mind: “You cannot be serious”!
(Note 1) the number needed to treat included women/under 75 year olds etc, who we now know will not benefit, so the number needed to treat will change with the caveats spelled out. Even if you leave out the caveats and tell any non-diabetic/non-stroke patient over 50 they have a 1 in 90 chance of benefit and likelihood of side effects – some serious – I suspect they would decline the kind offer.