These were the headlines that we woke up to on Thursday 29th September 2016. More specifically, it was claimed that a Mediterranean diet could prevent 20,000 deaths in Britain each year. That’s an important clarification, as we’re all going to die.
The original study can be seen here and it’s on open view.
There were three really interesting learnings from this study: i) we get a detailed definition of what researchers think the Mediterranean Diet is (as opposed to what Mediterranean people actually eat); ii) we get a new (and incomprehensible) way of guessing (estimating) the impact of this made up diet on deaths; and iii) we get an example of the new way of reporting studies/grabbing headlines, which I forecast will end the “20% greater risk “ coverage we have suffered to date.
The data source
The study is one of personal interest to me, as the data came from the European Prospective Investigation into Cancer study (EPIC) and I’m a member of the EPIC study. This article came from the Norfolk part of the EPIC study (this is called a cohort). The Norfolk cohort included 25,639 men and women, who were aged between 40 and 79 years, when they were recruited into the study (between 1993 and 1997). I was some way off this age between 1993 – 1997, so I shouldn’t be part of this particular EPIC cohort. However, it does mean that I am familiar with the EPIC dietary questionnaire and you can see the questionnaire here.
I recall completing this questionnaire when I was recruited in the 90s and I’ve had one or two questionnaires since (I can’t remember). I was recruited through the Vegetarian Society, of which I was a member at the time. EPIC wanted more vegetarians to assess, so the Vegetarian Society was a natural recruitment avenue.
Talking of veggies, look at the meat part of the questionnaire on page 3. The questionnaire asks how often we had a “medium serving” (whatever that is) of each meat on average over the past year (good luck remembering!) There are nine options from “never/less than once a month” to “6+ times a day”. That would be easy as a vegetarian i.e. never; not so easy for someone who consumes meat.
Plus, how is the baseline questionnaire balanced with any follow-ups? I was veggie when I first completed the questionnaire, but not veggie at follow-up. If I die now of a heart attack, am I a vegetarian death? Had I been the biggest processed meat consumer in the world in the 1990s and then saw the error of my ways the day after filling in the survey, my heart attack could be attributed to meat consumption, despite 20 years of being veggie. That’s how compromised these long term epidemiological studies are.
Of the original 25,639 participants in the EPIC Norfolk cohort, some were excluded for not having Food Frequency Questionnaires and/or for having cardiovascular disease (CVD) at baseline. This left 23,902 participants, with an average age of approximately 59 at baseline (baseline was between 1993 and 1997). CVD cases were examined in 2009 (after an average 12.2 years of follow-up) and CVD deaths were examined in 2013 (after an average 17 years of follow-up). Participants were, on average, 76 years old when CVD deaths were being assessed, so we would expect many to have died.
In fact, not so many people died from CVD during the average 17 years of follow up. To be precise, there were 1,714 CVD deaths per 382,765 person years studied in total. This gave a death rate of 448 per 100,000 people. Well done to the authors for reporting 448 per 100,000 people in the abstract of the article; the absolute death rate is rarely given.
The characteristics of the 23,902 participants were set out in Table 1 and the key differences highlighted in the text. Here’s the key bit, which we’ve seen before: “Participants with high adherence [to the Mediterranean Diet] were less likely to be current smokers, and more likely to be physically active and have a college education and higher social status compared to participants with low adherence.” There’s that ‘healthy people eat healthily’, rather than ‘eating healthily makes you healthy’ observation, which crops up time and again. Whatever the authors think a Mediterranean Diet is, it is as likely a marker, as a maker, of a healthy person.
What the article did
In simple steps, this is what the article did (because the headlines were about deaths, I’m only going to look at deaths, so as not to confuse numbers with CVD events):
Step 1) Make up a Mediterranean Diet (we’ll come back to this);
Step 2) Assign a score to this made up Mediterranean Diet (we’ll come back to this too);
Step 3) Use highly inaccurate Food Frequency Questionnaires (FFQs) from many years ago (1993-1997 and/or 1998-2000) to work out how participants scored against this made up Med Diet;
Step 4) Estimate how many CVD deaths were estimated (they do use the estimate word twice in this part of the paper) to be attributable to low (vs. high) adherence to this Mediterranean Diet;
Step 5) Apply this estimate to UK deaths, so that it looks huge and grabs headlines.
That was basically what happened, but there was much fun along the way. Let’s take a quick look at the best bits:
Step 1: Make up a Med Diet
If you go to the end of the article and look at “Additional File 1”, this is where we find out how the Mediterranean Diet has been made up. To be fair, four different models of the Med Diet from other articles were used. (The relevant models are fully detailed in references 4, 5, 17 and 28 of the main paper, if you’re interested). The study that forms the major part of the five steps above is the so-called “Mediterranean Diet pyramid”, so we’ll stick with the analysis related to this one. (You can find out more about “Mediterranean Diet Pyramid” from reference 4 at the end of the paper – or just Google those words).
Step 2: Assign a score
The fabulous Additional File 1 tells us that each participant Food Frequency Questionnaire (FFQ) is assigned a score out of 15 as follows (one point possible per 15 lines):
Table S2: Pyramid based Mediterranean diet score (PyrMDS) scoring criteria.
Score of 0
Score of 1
2/d for men,
≥4/d for men,
1.5-2.5/d for men,
Principal source of
(ZH – The notes to this table have been left out for simplicity. You can see them in the Additional File. I don’t think they add anything – the columns show really well what gets 0 points vs. 1 point.)
You can see from the right hand column how you get the maximum 15 points for high adherence to this made up Med Diet: vegetables 6 or more times a day; legumes (beans, pulses, tofu, soya, vegeburgers) 2 or more times a week; 3-6 pieces of fruit a day; nuts 1-2 times a day; cereals 3-6 times a day (WHAT?!); and so on…
There were three interesting details: i) Dairy was described as single cream, double cream, clotted cream, low or full fat yoghurt, dairy desserts, cheese, milk etc – by no means low-fat-dairy; ii) potatoes were not seen as healthy – being given a maximum of 3 per week; and iii) moderate alcohol consumption scored higher than non-consumption (there is evidence to support this).
This diet is not what people eat in the Mediterranean therefore, but it is consistent with the made up Med Diet (veg, fruit, cereals, pulses, olive oil), which is increasingly being presented as a bona fide diet. The most powerful aspect of this diet may not be what is eaten, but what isn’t: sweets, crisps, biscuits, snacks and junk etc.
3) Use the FFQs to see how closely people adhered to this Med Diet
Three groups were then created by distributing the participants into three evenly sized groups (tertiles). (This is the fairest way to break up people into groups to be compared, so well done again). The lowest third scored 3.2-8 out of 15; the middle group scored 8-9.1 out of 15 and the highest group scored 9.1-13.1 out of 15.
The study then ignored the middle third and compared low adherence to the Med Diet with the high adherence group. So people scoring 3-8 out of 15 were compared with those scoring 9-13.
As a matter of interest, I’m in the bottom group. I would score 5 out of 15 and I have a super nutritious real food diet with the main staples being: grass fed meat; oily fish; eggs; farm sourced dairy products and veg from the man with the allotment in the village. I get points for ≥6/d vegetables, ≥2/wk fish, ≤1/wk processed meat, ≤3/wk potato and ≤2/wk sweets. Everything else I’m too low (cereals/nuts) or too high (eggs/dairy/red meat). I don’t even get a point for alcohol as I don’t like the taste, so I’m a non-consumer!
4) Estimate how many CVD deaths were estimated to be attributable to low (vs high) adherence to this Med Diet
This is the second interesting learning, as mentioned in the introduction. The researchers guess/estimate what impact the diet had on deaths. This is how the estimate of the estimate was done:
“We calculated PAF [Population Attributable Fraction] based on the formula of rate difference: PAF = (I0 − Ii)/I0, equivalent to I0 − HR × dMDS/I0, where HR was estimated continuously with adjustment for potential confounders as aforementioned, I0 is observed incidence per 10,000 person-years, and Ii represents a hypothetical, ideal incidence if the population achieved high MDS (95th percentile) (dMDS=MDSideal – MDSobserved). The CI of the PAF was derived from bootstrapping [I love that bit!] to estimate HR and PAF iteratively (n resampling = 100, after confirming no difference in results between n = 100 and 1000).”
Ho ho ho. If you understand a word of that, you’re a better estimation evaluator than I am.
Whatever the heck went on there, the number that magically popped out of the estimation machine was 12.5%.
5) Apply this estimate to UK deaths, so that it looks huge and grabs headlines
The discussion part of the paper said: “CVD accounts for approximately 155,000 deaths [in the UK] each year… If we assume causality [which is not a reasonable assumption] and generalizability of our findings to the general UK population [also not reasonable – the UK average age is nearer 40, than 59, for starters, Norfolk is nothing like Merthyr or Kensington etc], a PAF of 12.5% would have equated to 19,375 cases of CVD deaths preventable each year.”
And that’s where the headline comes from.
You cannot be serious!
You simply cannot do what these researchers have done. Steps 4 and 5 are complete nonsense. The researchers have taken 155,000 CVD deaths p.a. and (however they got the magic number of 12.5%) they try to claim that 12.5% of all those 155,000 UK deaths were attributable to people not being on this made up Med Diet. We must take a step back and return to common sense.
The number one reason for those 155,000 CVD deaths, by a margin, was age. This short post should help. CVD is so massively determined by age: 1 in 166,667 children aged 5-14 are likely to die from heart disease in any one year vs. 1 in 22 people aged 85 or older.
As the ”Heart Disease” post explains, the other two major causes of CVD (which we can do nothing about) are gender (don’t be male) and genetics (don’t have dodgy parents). I would wager that well over 80% of CVD deaths can be explained by age/gender and genetics. Of whatever small proportion is left, smoking will be the next major risk factor and the first that we can do something about. Then I would put sedentary behaviour, as a risk factor, above some made up diet any day. I would then expect being normal/overweight, rather than underweight/obese, to be important and this would go hand in hand with not eating sugar/white flour/a highly processed diet. Where will legumes and nuts come in? Nowhere at all my friends.
Watch the headlines
I praised the abstract for including the absolute death rate from CVD during the study – 448 per 100,000 person years. Another number presented in the abstract was that the hazard ratio for CVD mortality was 0.91 (0.87-0.96). This could not be found in the body of the paper, but I assume it was comparing the high Med Diet adherence with the low adherence once more. (0.91 is a 9% relative difference, by the way. Bradford Hill will be turning in his grave that this could even get off the starting blocks.)
Put these two numbers together and we have a CVD death rate of 448 per 100,000 person years (in quite elderly people). Who cares? Then, the 0.91 hazard ratio means that we have 428 deaths per 100,000 person years for the high adherence group and 468 deaths for the low adherence group (428/468 is 0.91). A difference of 40 deaths; who cares even less?
Eat veg 6 times a day, fruit 3-6 times a day, cereals 3-6 times a day (presumably spend a lot of time near a bathroom!) drink alcohol, but not too much, drown everything in olive oil and the CVD deaths difference might be 40 in 100,000 person years. That would never even get a column inch on p37.
BUT, put magic numbers in the magi mix and make up an entirely new number, 12.5%. Then, multiply a study of 24,000 people, as if it applies to the tens of millions of people in the UK. Then you get a really big number and THEN you get the headlines.
This will be the new way of reporting studies, folks – mark my words. Within a few months, researchers and those who write their press releases will be wondering – why did we ever do that “20% greater risk if you eat sausages” thing? People don’t even understand percentages. They understand 20,000 people will be saved. Hoorah! From now on, everything will be scaled up, as if whole countries stand to benefit. As Dr Malcolm Kendrick would say: “We shall all be saved! Hoorah again!”