We’re doing a classic article this week – looking at the PREDIMED study. PREDIMED stands for Prevención con Dieta Mediterránea (Prevention with Mediterranean Diet). PREDIMED is the study that led to claims such as “olive oil and/or nuts and/or ‘The Mediterranean Diet’ can prevent heart disease.” To date there have been 238 publications from this one study. The main publication from the study (Ref 1) has been cited by other articles 2,596 at the time of writing this note. That is immense.
As we will see, PREDIMED was a strange trial. Ostensibly it was two interventions, compared against one control. Had the interventions changed one thing alone – olive oil in one arm and nuts in the other arm – and compared these against a control that changed nothing, perhaps we could eulogise about olive oil and nuts. However, this was not what happened.
Upon closer examination, it transpires that there was no control. To be more specific – the group that experienced the most change was in fact the so-called control.
The authors’ conclusion was “Among persons at high cardiovascular risk, a Mediterranean diet supplemented with extra-virgin olive oil or nuts reduced the incidence of major cardiovascular events.” My conclusion would be: “Among Spanish people, aged 55-80, mostly women, at high cardiovascular risk, a low-fat diet increased the incidence of major cardiovascular events (only strokes significantly) by 3 events per 1,000 person years.”
Let’s take a good look at this fascinating study…
PREDIMED was described as a randomised controlled trial (RCT), but participants were also followed-up after the trial ended. It was thus an RCT that became an observational study. Recruitment started in October 2003; the last participant was recruited in June 2009 and the trial formally ended in December 2010 (we’ll come back to this).
The primary aim of PREDIMED was to compare the effects of two Mediterranean-type Diets (MeDiets for short), with a low-fat control diet, on a composite endpoint of myocardial infarction (heart attack), stroke, and death from cardiovascular causes [composite means those three things added together]. Secondary end points were stroke, myocardial infarction and death from cardiovascular causes [separately], and death from any cause. The primary and secondary aims are important to note because the primary aim adds three things together and the secondary aim was to examine these three things separately (and to look at deaths from all causes and they also noted diabetes, dementia and some cancers for interest). The significance of these aims will become clear.
Research should start with a hypothesis to be tested. The hypothesis in PREDIMED was: “that two traditional MeDiets, one enriched with virgin olive oil (VOO) and another enriched with nuts, both high in total fat and unsaturated fat, would be superior to the usually recommended low-fat diet for the primary prevention of CVD in a high-risk population” (Ref 2).
The study involved 7,447 Spanish men (55-80 years old) and women (60-80 years old) who were free of cardiovascular disease (CVD) at baseline, but considered to be at high risk of developing CVD. The inclusion criteria were that the participant should either have type 2 diabetes or three, or more, CVD risk factors. The CVD risk factors (I don’t agree that all of these are risk factors) were considered to be: current smoking (>1 cig/day during the last month); hypertension (systolic BP ≥140 mmHg or diastolic BP ≥90 mmHg or being on high blood pressure medication); LDL cholesterol ≥160 mg/dl or being on lipid-lowering medication; HDL cholesterol ≤40 mg/dl in men or ≤50 mg/dl in women; body mass index ≥25 kg/m2; and family history of premature coronary heart disease (CHD).
Participants were randomly assigned to three groups: i) a so-called MeDiet (short for Mediterranean-type diet) with extra virgin olive oil (EVOO); ii) a MeDiet with mixed nuts; or iii) a control group on a low-fat diet (effectively the standard government dietary guidelines). The EVOO participants were given 1 litre of EVOO per week and the nut group was given 30 grams of nuts per day, free-of-charge, to ensure compliance. The following companies supported/sponsored the study by donating the food for the intervention groups: extra-virgin olive oil (Hojiblanca and Patrimonio Comunal Olivarero, both in Spain); walnuts (the California Walnut Commission); almonds (Borges, in Spain); and hazelnuts (La Morella Nuts, in Spain).
The objective of the study was to measure “primary end points” defined as a major cardiovascular event – a heart attack, stroke or death. There were 288 such incidents – 96 in the olive oil group, 83 in the nut group and 109 in the low-fat group. The average (median) follow-up period was 4.8 years. The incidents were presented as events per 1,000 person years. There were 8.1, 8.0 and 11.2 incidents per 1,000 person years for olive oil, nuts and low-fat respectively (these were unadjusted numbers). The groups were all very similar for age, gender, smoking, BMI etc at the start of the study (as would be expected with 7,447 people randomly assigned to 3 groups) and so the adjusted figures were not very different. The adjusted figures were reported as – if there were 1 incident in the control group, there would be 0.7 incidents in the olive oil group and 0.72 incidents in the nut group. That’s where the 30% comes from.
This was reported as: “In this trial, an energy-unrestricted Mediterranean diet supplemented with either extra-virgin olive oil or nuts resulted in an absolute risk reduction of approximately 3 major cardiovascular events per 1000 person-years, for a relative risk reduction of approximately 30%, among high-risk persons who were initially free of cardiovascular disease” (Ref 1).
As ever, the headlines are big, the real numbers are small. Barely 1 person in 100, in the highest risk groups for cardiovascular disease, at the worst possible age to get cardiovascular disease, actually had an incident in any year of the study. For the nut and oil groups it was slightly fewer than 1 person in 100, for the low-fat group it was slightly more than 1 person in 100. The media reported the 30% figure at the time and it is still claimed today that nuts and/or olive oil and/or a Mediterranean diet can reduce heart disease by 30%. This is really misleading.
Table 3 added some detail to the 8.1, 8.0 and 11.2 per 1,000 person years figures (Ref 1). Remember that the primary endpoint was adding heart attacks, strokes and cardiovascular deaths together and the secondary end point involved looking at them separately? The secondary end point results told us that only the number of strokes were significantly different in either MeDiet group when compared with the control. Heart attacks and deaths were not significantly different. Hence my conclusion in the introduction that qualified strokes as being the significant difference.
The authors tried to explain this away by saying that the study “wasn’t sufficiently powered” (i.e. didn’t have enough people and/or wasn’t long enough) to look at the secondary outcomes. They added that the primary endpoint (not separating them) was the most important measure because it was the primary endpoint and so they should be looked at together. I really didn’t buy this.
We need to look at the diet next, before we look at the flaws of the study.
I must start by saying that there are at least two Mediterranean diets. There is what people actually eat in the Mediterranean and there is what the PREDIMED study defined as a Mediterranean diet. Other studies/journal articles have also defined their versions of the Mediterranean diet. The basic principle of what people actually eat in the Med is best summarised by the phrase “if it moves, eat it!” Staples consumed in Mediterranean countries include: lamb; beef; pork; rabbit; venison; chicken; turkey; fish; and seafood. The French will eat snails, frogs legs, horse – anything that moves. The Mediterranean diet also features a lot of (full-fat) dairy – cheese, cream and yoghurt more than milk (coffee tends to be black and strong) – and a lot of vegetables, some fruit and, finally, white things (pasta, potatoes, rice and bread). I have been playing ‘spot the wholegrain’ for 30 years in the Mediterranean region and I fail at almost every visit. Some brownish bread has crept in (probably to appease British tourists), but paella and risotto dishes never use brown rice and you’ll never be offered whole wheat pasta in any traditional dish.
The PREDIMED researchers defined their Mediterranean diet as adherence to a 14 point questionnaire (watch carefully for the more than (>) and less than (<) symbols and per day vs. per week):
1. Use olive oil as main culinary fat
2. Olive oil >4 tablespoons/d (per day)
3. Vegetables ≥ 2 servings/d
4. Fruits ≥ 3 servings/d
5. Red or processed meats < 1/d
6. Butter, cream, margarine < 1/d
7. Soda drinks < 1/d
8. Wine glasses ≥ 7/ wk (per week)
9. Legumes ≥3 /wk
10. Fish or seafood ≥ 3/wk
11. Commercial bakery ≤ 2/wk
12. Nuts ≥ 3/wk
13. Poultry more than red meats
14. Use of sofrito sauce* ≥ 2 /wk
* Sofrito is a sauce made with tomato/onion/garlic/herbs & olive oil.
The goldmine from the main PREDIMED paper (Ref 1) is the appendix, which can be seen here. Two parts of the appendix confirmed that participants assigned to each of the three groups were eating largely the same diet at the start of the study (again – as would be expected if 7,447 people are randomly assigned to 3 groups). Figure S3 in the appendix presented the adherence to the 14 point ‘Mediterranean diet’ questionnaire at the start of the study. This was approximately 8.5-9 out of 14 on average (mean) for the participants at the start of the study. Table S6 in the appendix detailed the intake each day of 13 items: virgin olive oil; refined olive oil; nuts; vegetables; fruits; wholegrain cereals; refined cereals; legumes; fish/seafood; meat/meat products; pastries/cakes; dairy products and alcohol. The intake of each group was virtually identical e.g. the olive oil, nut and control groups were all having 1.9 servings of dairy products per day – a serving was 200g! All groups were consuming 0.8-0.9 servings of meat/meat products per day. A serving was 150g, so this added up to almost a kilo per week.
Annoyingly, the 13 items in Table S6 did not match the 14 point questionnaire and so items cannot be examined as rigorously as I would like. What we can say, however, is that the groups had the same diet at baseline and the groups were adhering to the researchers’ idea of a Mediterranean diet reasonably well at the outset (8.5-9 out of 14).
The interventions were then as follows. The summary table for the diet advice can be seen here.
The extra virgin olive oil group (EVOO) was supposed to adhere to the 14 point MeDiet as closely as possible AND to consume at least 4 tablespoons (50 grams) of extra virgin olive oil per day. The nut group was supposed to adhere to the 14 point MeDiet as closely as possible AND to consume 30 grams of mixed nuts (15 g walnuts, 7.5 g hazelnuts and 7.5 g almonds) daily. The two groups allocated MeDiets received intensive education to follow the MeDiet and they were given olive oil and nuts for free. The control group did not receive education about a MeDiet. This group was given advice to follow a low-fat diet. This is important and we’ll come back to this in the flaws.
Three years after the study started, a questionnaire was developed for the low-fat group. This was a nine point check list:
1. Consume no more than 2 tablespoons of olive oil per day.
2. Remove all visible fat from all meat and other dishes before eating.
3. Consume no more than 1 serving of fat-rich meats per week (a burger/ham/cold meat serving was defined as 100g and a bacon/salami serving was defined as 30g).
4. Consume no more than 1 serving of butter/marg/mayonnaise/ice cream per week (spread serving size was 12g and the ice cream serving size was 100g).
5. Consume only low-fat dairy products (or no dairy).
6. Use Sofrito sauce no more than two times per week.
7. Consume fatty fish less than once a week.
8. Consume commercial bakery products (cakes, cookies, biscuits etc) less than once per week (serving size 80g for cake/40g for biscuits).
9. Consume nuts, crisps, chips, commercial snacks less than once a week.
The diet – what changed?
As a result of the advice given to the two MeDiet groups and the advice given to the low-fat group, many changes occurred over the period of the study:
i) The adherence to the MeDiet 14 point score increased from approximately 8.5-9 out of 14 at the start of the study to approximately 10.5 out of 14 for the two MeDiet groups combined (Figure S3 appendix). (The MeDiet score for the control group increased to over 9 from just after year 3 until the end of the study – we’ll come back to this). That’s not a huge change for the MeDiet groups. Given that the EVOO group would add a point by consuming their free olive oil and the nut group would add a point by consuming their free nuts, these groups didn’t make much more change than this. This could be an argument for the researchers (and the olive oil and nut companies) to rave about the virtues of olive oil and nuts. However, don’t jump to this conclusion too soon…
ii) Within the detail of the 14 point questionnaire, there were significant differences between the two MeDiet groups and the control group in 12 of the 14 items at 3 years (Table S5 appendix). This was reported in the main paper too (Ref 1). (Point ii may appear to be at odds with point i – all will be explained.)
iii) The participants assigned to the EVOO group AND those assigned to the nut group both significantly increased their consumption of extra-virgin olive oil (to 50 and 32 g per day, respectively) AND nuts (to 0.9 and 6 servings per week, respectively) (Table S6 appendix). So both oil and nut groups increased intake of both oil and nuts.
iv) The other significant changes in food intake were for fish and legumes. The two MeDiet groups significantly increased weekly servings of fish (by 0.3 servings) and legumes (by 0.4 servings) in comparison with those in the low-fat group (Table S6 appendix). Because the low-fat group was told not to eat fish, the fish difference was as much driven by the control group eating less, as it was by the MeDiet groups eating more.
v) The summary table for the diet advice encouraged both groups to have at least two servings of vegetables and three servings of fruit a day and yet the low-fat group nine point check list did not reinforce this. The two MeDiet groups achieved average compliance of 69% for the veg servings and 64% for the fruit servings. The control group achieved only 58% and 55% for veg and fruit compliance respectively (Table S5 appendix).
vi) The main effect of the low-fat advice was that all groups started with dietary fat intake of approximately 39% of daily calories (refreshingly high to start with.) This increased to around 41% for the 2 MeDiet groups (oil and nuts) and fell to 37% for the low-fat diet group (Table S7 appendix).
We are now ready for the major flaws in this classic study:
The major flaws
1) This was a randomised, but not controlled, trial.
The fundamental principle of a randomised controlled trial (RCT) is that one thing alone is changed in the intervention group and nothing is changed in a control group, so that the impact of one change can be measured. We know that being involved in a trial is a change in itself, so this must also be allowed for. e.g. as soon as someone is enrolled into a trial they feel special/involved/worthwhile and they want to comply. This has been tested as an intervention in itself and we know that people achieve better results being monitored than they do being left alone.
To test even the made-up Mediterranean diet, PREDIMED should have taken the 7,447 people – who started off eating the same diet – and told half of them to agree to continue to eat in this way for the next 6 years and told the other half of them to try to adhere to the full 14 points as far as possible. Both should have been followed up identically, so that the additional factor of ‘feeling special’ did not favour one group alone.
The trial could have tested two things: Group 1 – try to score 14 out of 14 on the MeDiet; Group 2 – eat exactly as you do now – the only difference being added nuts OR olive oil. Both of these would need to be measured against a control group told to eat exactly as you do now, with no changes at all.
PREDIMED did not do this. It tried to test a MeDiet AND olive oil and a MeDiet AND nuts with a changed diet for the control group – a low-fat diet. PREDIMED thus had three interventions. This brings us nicely to flaw two…
2) The biggest intervention was the control.
When I explained the PREDIMED design to my husband, Andy, his immediate reaction was – so the low-fat diet was the intervention! The two MeDiet groups were asked to do more of what they had largely been doing (and add a bit of oil/a/few nuts), while the ‘control’ group was turned upside down.
We know that the average person (across all groups) was scoring 8.5-9 out of 14 MeDiet points at the start of the trial (Figure S3 appendix). For the participants in the EVOO and nut groups, this score increased to approximately 10.5 out of 14 during the trial. This – not even 2 point difference – can be explained by the free olive oil/nuts. The control group MeDiet score also rose to between 9 and 9.5 soon after year 3 (Figure S3). This seems to contradict Table S5, which tells us that there were significant differences between the MeDiet groups and the control in 12/14 measures. i.e. Figure S3 shows little change, Table S5 shows much change – what’s going on? It comes down to the impact that the low-fat diet advice had on adherence to the MeDiet…
Four parts of the low-fat diet group advice – the advice to limit olive oil, limit sofrito, limit fish and limit nuts – moved participants away from the MeDiet that they had largely been consuming. Two bits of advice were new and unique to the low-fat group – trim fat off everything and only eat low-fat dairy products. Three bits of low-fat diet advice would support adherence to the MeDiet – limit meat generally, limit butter & spreads and limit commercial bakery products. The dietary advice thus required far more change for the control group than the so-called intervention groups and the advice was both supporting and rejecting the MeDiet in different parts.
Another fact confirms that change in the control diet was more impactful than change in the intervention groups. Not only did the study change multiple factors from the outset, the study failed to keep follow-ups constant between all groups to avoid the ‘I feel special/I want to comply’ confounder. Participants in the two MeDiet groups received individual and group dietary-training sessions and they completed the 14 point questionnaire at baseline and at quarterly sessions thereafter. The control group received dietary training at the baseline and completed the MeDiet questionnaire (so they knew the goal for the other groups.) They were then followed up each year, not quarter, for three years. The main paper then reported: “the realization that the more infrequent visit schedule and less intense support for the control group might be limitations of the trial prompted us to amend the protocol in October 2006.” No kidding! Thereafter, the control group participants also received individual advice and were invited to group sessions with the same regularity and intensity as the MeDiet groups. The 9 point check list was added at this point, to match the 14 point questionnaire being completed by the MeDiet groups at every follow-up.
Researchers compared the differences between the MeDiet groups and the control group before and after October 2006. Before 2006, there was no significant difference. After 2006, there was a significant difference (the adjusted hazard ratio was 0.49 [95% CI, 0.26 to 0.92] for those who like the stats). The study thus started to achieve a significant difference when the low-fat diet group started to be monitored, as they should have been from the outset, and presumably when adherence to the low-fat diet improved.
Dropout rates were higher in the control group (11.3%) than in the Mediterranean diet groups (4.9%). This may have been because the control group didn’t feel loved; it may have been because (Spanish) people don’t like low-fat diets.
3) The trial was stopped early.
The trial was intended to last six years. This was considered to be the time that it would take to reach a significant difference for the primary end point with 7,447 people. It appeared from the reporting of the trial that every year, after two years, statistical tests were run (O’Brien-Fleming stopping boundaries) to see if the trial could be stopped early. That tells me that they were looking to stop the trial as soon as possible. The main paper reported that: “The stopping boundary for the benefit of the Mediterranean diets with respect to the primary end point was crossed at the fourth interim evaluation; on July 22, 2011, the data and safety monitoring board recommended stopping the trial on the basis of end points documented through December 1, 2010.”
I consider stopping early a major flaw. The fact that a statistical test said that the trial could stop early does not mean that it should have been stopped early. i) The secondary endpoints, by the researchers own admission, had not reached statistical significance, so the trial should have been continued until they did. ii) The protocol had been changed substantially in October 2006 and so the trial should have been continued for longer to compensate for this. iii) the lines were starting to converge. If you look at Figure 1A (Ref 1), pictured below, the gap between the control diet and the MeDiets appears to be narrowing at approximately five years (the inset figure is an exaggerated version of the flat lines in the main graph, otherwise no one would be able to see any difference.) The trial should not be stopped at this point, as it is important to see if this trend continues.
The researchers listed what they considered the limitations of their study to be in the Discussion part of the main paper (Ref 1). This is common practice. They listed them as: i) the control group protocol changing; ii) losses to follow-up – especially in the low-fat group; and iii) the results may not be generalisable (applicable to other populations) as they studied people living in a Mediterranean country at high risk of cardiovascular disease. I agree with these, but there were far more serious flaws not reported.
PREDIMED failed the basic rule of RCTs by changing more than one thing. The biggest changes were asked of the so-called control group, making the low-fat diet the primary intervention. Very small differences – 3 in 1,000 person years – in the primary outcome (heart attacks, strokes and CVD deaths added together) were observed between the MeDiet and the low-fat diet group. The only statistically significant finding was for strokes. Even without the major flaws, the absolute differences were not worth worrying about.
The takeaway message for researchers is – if you set out to do a large, long, expensive RCT, then make sure it is an RCT. Even if the control had genuinely been a control, we would not have known if olive oil, nuts, fish, legumes, fruit and vegetables, or a 1.5-2 point movement on a 14 point scale had made a difference. Would a MeDiet plus coconut oil have achieved the same result? A MeDiet plus sunflower seeds? If the ‘control’ had not been told to minimise healthy fish intake and avoid nutrient rich full-fat dairy, and if the ‘control’ had been measured on fruit and vegetable intake, would anything have happened at all?
The takeaway message for members of the public is – we can’t say that any of this applies to you unless you are a Spanish male aged 55-80, or preferably a Spanish female aged 60-80, with no CVD, but CVD risk factors. If this is you, you should probably avoid going on a low-fat diet, because those who do tend to have a couple more incidents (significantly strokes) per 1,000 people each year. The message that you really should not take away, however much the sponsors may like this, is the idea that olive oil or nuts are somehow magical. If you think that approximately 3-4 walnuts, 4-5 hazelnuts and 4-5 almonds a day will reduce your risk of CVD by 30%, you really are nuts!
Ref 1 Estruch R et al. “Primary Prevention of Cardiovascular Disease with a Mediterranean Diet.” New England Journal of Medicine. (2013).
Ref 2: Martínez-González et al “Cohort Profile: Design and methods of the PREDIMED study” The International Journal of Epidemiology. (2010).