On May 8th 2015, The Independent, and a few other newspapers, reported “High-protein diets increase risk of weight gain, study finds”.
The abstract can be found here. Sadly the full article is not on open view, but I’ve got a copy to dissect it for you below.
The PREDIMED study
In this study, the researchers used the PREDIMED (PREvencion con DIeta MEDiterranea) data to look at the association between protein intake and weight and then protein intake and deaths.
I’ve written about this study before. PREDIMED is actually a randomised controlled trial, rather than a population study, as the 7,447 participants were divided into three groups at the start of the study. Two groups were put on a diet called “The Mediterranean Diet”, which was described as follows: “The traditional Mediterranean diet is characterized by a high intake of olive oil, fruit, nuts, vegetables, and cereals; a moderate intake of fish and poultry; a low intake of dairy products, red meat, processed meats, and sweets; and wine in moderation, consumed with meals.” This is not what people actually eat in the Mediterranean. They eat a diet high in: meat, especially red and cured; fish, especially oily; dairy products, especially cheese; eggs; vegetables; and fruits in season. They also eat potatoes and white grains (bread, rice, pasta) and they drink red wine – not sure about the moderation!
The control group was put on a low-fat diet (that was an adverse intervention, not a control, therefore). The other two groups were put on this Fictitious Mediterranean Diet (FMD from now on) and they were also told to avoid soda drinks, bakery goods, spreads, red and processed meat (apart from the red meat, this is excellent advice). The low-fat diet group was told to have at least three servings a day of bread, pasta, potatoes, rice etc – those nicely fattening products that raise triglyceride levels. The FMD groups were told to have oily fish. The low-fat group was told to avoid it. One of the intervention groups was encouraged to add 50g or more of olive oil daily to their FMD and the other FMD group was given 30g of mixed nuts per person per day.
Of the 7,447 people in the study, 57% were women. The women were aged 60 to 80 and the men were aged 55 to 80. The participants had no previous cardiovascular disease at enrollment, but they were deemed “at-risk”, as only people with type 2 diabetes or at least three other “major risk factors” (smoking, obesity, family history of heart disease etc) were included in the study.
This particular study using PREDIMED data
7,216 of the original 7,447 subjects were included in this protein study. 231 people were left out for having extremely high or low energy intake and/or incomplete dietary data. The 7,216 people were placed into five groups in order of average energy from protein intake – low to high. There were 1,443 people in each group (1,444 in the middle group). The lowest protein intake group averaged 83 grams per day and the highest 96.6 grams, so we’re talking a couple of eggs difference between highest and lowest protein intake here, or 50 grams of tuna.
There were some striking differences in the five groups. They were similar for age, BMI and waist circumference, but the highest protein group were healthier for smoking and drinking: 72% of the highest protein group had never smoked vs 47% of the lowest protein group; 1.1% of daily calories came from alcohol for the highest protein group (HPG) vs 4.5% for the lowest protein group (LPG).
However, some other substantial differences were working against the HPG. Prevalence of diabetes was 60% in the HPG vs 37% in the LPG. Family history of cardiovascular disease was 25.1% vs 19.6%. 40% of the HPG were taking oral anti-diabetic drugs vs 24% of the LPG and three times as many of the HPG were taking insulin as the LPG. The HPG also did less activity than the LPG.
The energy intake of the five groups was one of the most striking differences: an average of 1,972 calories per day in the HPG and 2,453 calories per day in the LPG. The substantial difference in calorie intake impacts the percentage of energy from protein intake (the denominator effect for the mathmos). The article thus claims that the lowest calorie intake group was at greater risk of weight gain!
Model 1 in the paper adjusted for age, smoking, exercise, alcohol and similar lifestyles factors (and for the PREDIMED intervention!) Models 2 and 3 further adjusted for health conditions and medications, but they also played around with other macro nutrients (fats and carbs), as we’ll see.
After the baseline differences, the next interesting thing was that the study defined weight change as those who lost or gained ≥ 10% of body weight. That’s a lot in study terms. A weight change of 5% is more typically used as a measure. Everyone else was put in the “maintained weight” category. This high bar meant that, during a median follow-up of 4.8 years, 186 cases of weight loss were recorded and 149 of weight gain. So, the headlines about weight gain are immediately based on 149 people, not 7,447.
Then, of these 149 people who gained weight, 93 were in what was considered a normal protein intake range, 37 had protein intake lower than this and 19 had protein intake higher than this. Now we’re down to the headlines being based on 19 people. (This part of the paper looked at just three groups – low/normal/high – not the original five).
For the raw data facts, 2.1% of the lower protein group gained 10% or more of their starting body weight; 2.1% of the normal protein group gained 10% or more of their starting body weight and 1.9% of the higher protein group gained 10% or more of their starting body weight. The higher protein group thus had a lower incidence of weight gain (fractionally) than the other two groups.
When it came to losing 10% or more body weight, this was achieved by 2.6% of the lower protein group, 2.4% of the normal protein group and 3.5% of the higher protein group.
So the higher protein group had a lower incidence of weight gain and a higher incidence of weight loss. None of these results was statistically significant, but they make a mockery of the media headlines.
The one significant result
When protein intake alone was looked at, using the standard measure of grams of protein per kilogram of body weight (with data unadjusted or fully adjusted), not one significant result could be found for weight gain, or weight loss, or waist circumference increase, or waist circumference decrease. Nothing. Zippo. Zilch.
The researchers should have stopped there. Or reported that protein intake has no impact on weight or waist circumference either way when 7,216 people are followed-up for an average 4.8 years.
But no. They decided to adjust for fat intake, which allowed the claim that the higher protein intake must be at the expense of carbohydrate and they adjusted for carbohydrate intake, which allowed the claim that the higher protein intake must be at the expense of fat. They looked at all protein options possible (total protein intake, animal protein intake, vegetable protein intake, the ratio of animal to vegetable protein intake), for weight loss, weight gain, waist circumference increase, and waist circumference decrease. That’s a total of 128 different options that they looked at. Seven showed a significant difference and only one of these was reported in the abstract:
1) The one statistically significant finding reported in the abstract was that, when protein replaced carbohydrate the hazard ratio was 1.9 with the confidence interval being 1.05-3.46 – 1.05 being so close to the non-significant 1.0. (The confidence intervals are wide because of the very small numbers we’ve ended up with – 19 people).
2) The six statistically significant findings in the body of the article, but not the abstract or headlines, were all related to low intake of vegetable protein. They were that:
i) People with the lowest intake of vegetable protein were less likely to lose weight (in both the carb and fat ‘replacement’ models). The opposite didn’t help – people with higher vegetable protein were not more likely to lose weight.
ii) People with the lowest intake of vegetable protein were less likely to have a reduction in waist circumference. They were also less likely to have an increase in waist circumference. Again – this was the case for both the carb and fat ‘replacement’ models – making four statistically significant, but rather irrelevant and unexplained, results.
I haven’t repeated all of the above for deaths. The hazard ratios for deaths are presented in Table 2 of the paper. Let’s take the raw data again (Note below): There were 69 deaths (among 1,443 people) from any cause during the follow-up in the highest protein group (HPG). There were 95 deaths (among 1,443 people) from any cause during the follow-up in the lowest protein group (LPG).
If we play the association, relative risk game that is usually played, that could be presented as 40% higher deaths in the lowest protein group. The fact is that the absolute death rate in the HPG was 4.8% and it was 6.6% in the LPG. Cardiovascular death rates were 1.0% in the HPG and 1.7% in the LPG. Cancer death rates were 1.9% in the HPG and 2.6% in the LPG.
The researchers noted U-shaped patterns in the data (the middle group tended to be the low point and either side tended to be higher). Instead of comparing low protein intake with high, they compared everything to the middle group and then only reported how bad the high protein intake was. The low protein intake was arguably worse.
(Note – the adjusted models don’t make intuitive sense: smoking & alcohol work in favour of the HPG, but diabetes and meds – especially insulin – work against the HPG. Model 1 adjusts for the lifestyle factors, so I would expect the results for Model 1 to swing against the HPG – and they do. However, Models 2 and 3 further adjust for medical conditions and medications, so I would expect these to swing back in favour of the HPG and they don’t).
The actual study conclusion was “Higher total protein intake, expressed as percentage of energy, was significantly associated with a greater risk of weight gain when protein replaced carbohydrates.”
The headline should have been: “There is not even an association between protein intake, weight gain, weight loss, and/or change in waist size.” The supplementary headline should have been “When we play with numbers to try to get a result, we still can’t find anything in 121/128 cases and the other 7 are nothing to write home about.”
Or: “People with low protein intake are 40% more likely to die.” ;-)
Protein is in all foods except pure fats (oils/lard) and pure carbohydrate (sucrose). Protein is in lettuce, apples, bread, steak – every other food. There is no pure protein food on the planet. Skinless chicken breasts and white fish are the closest we get to being able to eat protein alone. Hence eating more protein in a natural (real food) diet means eating more carbohydrate (beans, pulses, fruits, vegetables, potatoes, oats) or more fat (meat, fish, eggs) or both (dairy products, nuts, seeds). Eating more protein unnaturally would mean removing the skin from chicken – or worse – protein shakes etc.
I do think that an unnaturally high intake of protein is potentially harmful. Protein shakes etc will place unnecessary strain on the kidneys, deplete vitamin A and may have long-term harm. However, unnaturally high protein diets have a metabolic advantage (thermic effect of nutrients) and would be expected to have a positive impact on weight while having a negative impact on health. Death within 4.8 years I would find highly unlikely, unless the protein intake was extreme, which was not the case in this study (83-96.6 grams/day).
That’s a general point – it’s not what this study is about. I’m afraid that this study reinforces a trend that has been taking place for some time – researchers playing with numbers and over-emphasizing any slightly significant finding, while ignoring the truer picture that there was no finding, to grab headlines. The bar for nutritional studies needs to be substantially raised and adhered to.