Low carb diets could shorten life (really?!)

The BBC headline was ”Low-carb diets could shorten life, study suggests” (Ref 1). In the US, CNN went with “Low and high carb diets increase risk of early death, study finds” (Ref 2). There were many similar, irresponsible, headlines worldwide that emanated from a study published in August 2018 in The Lancet Public Health journal (Ref 3). The Sydney Morning Herald warned “People on low carb diets die younger, says science” (Ref 4).

Let’s look at the ‘science’…

We need to make a critical point up front: every headline using the words “low carb” was wrong. The first sentence of the paper was “Low carbohydrate diets…” This was also wrong. The full paper used the words “low carbohydrate” 40 times. That was also wrong – 40 times. Low carb diets have not been studied by this paper. Full stop. The average carbohydrate intake of the lowest fifth of people studied was 37%. That’s a high carb diet to anyone who eats a low carb diet. As we will see below, the researchers managed to find just 315 people out of over 15,000 who consumed less than 30% of their diet in the form of carbohydrate. The average carb intake of these 315 people was still over 26%. Not even these people were anywhere near low carb eating. Hence, if you do eat a low carbohydrate diet, don’t worry – this paper has nothing to do with you.

You’re welcome to continue reading to see what else was wrong with this paper.

The study

The study was in three parts, but all were based on population studies and so the usual limitations of these apply. The first part was a study of 15,428 adults aged 45-64 years, in four US communities, who completed a dietary questionnaire at enrolment into the Atherosclerosis Risk in Communities (ARIC) study (between 1987 and 1989) (Ref 5). The primary outcome of interest was all-cause mortality. The second part of the study involved combining the data from the ARIC study with data from seven multinational population studies in a meta-analysis. The final part was an assessment of whether the substitution of animal or plant sources of fat and protein for carbohydrate affected mortality.

The findings

The findings from Part 1, the ARIC study, were that there were 6,283 deaths during the 25 year follow-up. The conclusion from this part of the study was that 50-55% of energy from carbohydrate was associated with the lowest risk of mortality. The average intake of carbohydrate in the ARIC study was 49%. The paper presented a U-shaped curve to indicate that carbohydrate intake below 30% and above 65% was associated with the highest risk of mortality.

In Part 2, when the meta-analysis pooled together the results from ARIC with seven other population studies, the U-shaped association was observed again. Both lower carbohydrate intake (<40%) and higher carbohydrate consumption (>70%) were associated with higher mortality than ‘moderate’ carbohydrate intake.

Part 3 reported that the results varied depending on the source of macronutrients. It was claimed that mortality increased when carbohydrates were exchanged for animal-derived fat or protein and mortality decreased when the substitutions were plant-based.

The headline

The headline that generated so much media attention was a purely statistical calculation in the paper, which resulted in the claim “we estimated that a 50-year-old participant with intake of less than 30% of energy from carbohydrate would have a projected life expectancy of 29·1 years, compared with 33·1 years for a participant who consumed 50–55% of energy from carbohydrate… Similarly, we estimated that a 50-year-old participant with high carbohydrate intake (>65% of energy from carbohydrate) would have a projected life expectancy of 32·0 years, compared with 33·1 years for a participant who consumed 50–55% of energy from carbohydrate.”

I’ll show below how this claim was generated – when we address the biggest issue with the paper.

10 things wrong with this article

There are a number of issues with all epidemiological studies. The two most important generic limitations are:

1) Association does not mean causation; and

2) Relative risk is presented when absolute risk is invariably tiny.

All dietary epidemiological studies also suffer from three further limitations:

3) The Food Frequency Questionnaire.

4) Incomplete adjustment of the data.

5) The healthy person confounder.

This particular study then additionally suffers from the following limitations:

6) Failure to adjust for a serious confounder (alcohol).

7) The small comparator group issue.

8) The selection of the reference group.

9) The meta-analysis.

10) The claims related to food ‘exchanges.’

I’m going to go through each of these issues, with particular reference to this Lancet Public Health paper, to show why the findings of this paper and the headlines can’t be trusted:

1) Association does not mean causation.

Population studies enable us to observe that people who, for example, eat broccoli die older than people who don’t. They cannot conclude that eating broccoli causes you to die older. Equally possible is that people who tend to eat broccoli tend to be generally healthy and therefore tend to die older.

If association is observed, the association should be subjected to The Bradford Hill Criteria to test association (Ref 6). The first test is “strength of the association” – if this isn’t double, or greater, there’s little point looking at any of the other criteria – you aren’t going to be able to claim causation. This brings us to…

2) Relative risk is presented when absolute risk is invariably tiny.

This study didn’t present its claims in the usual way “37% greater chance of dying” – which is a measure of relative risk. It is as if the world has become immune to such headlines. This paper chose a new way of trying to shock the world: “You’ll die 4 years earlier unless you eat the ‘perfect’ intake of carbohydrate.”

The main paper didn’t present the relative risk numbers. The appendix did, however. The following data are extracted from Supplementary Table 1 from the appendix:

The table below shows: the carb ranges subjectively selected by the researchers (we’ll come back to this); the number of people that ended up in each range; and the deaths that occurred in that carb range during the 25 year follow-up. I have then included the Hazard (risk) Ratio (HR) for the most adjusted model (Model 2). Model 2 adjusted for age, race, gender, which test center the person attended, total energy consumption, diabetes, cigarette smoking, physical activity, income level and education. Please note that Model 2 – the most adjusted model – didn’t adjust for BMI. (It didn’t adjust for something else either, as point 6 will cover).


Table 1

Carb range <30% 30-40% 40-50% 50-55% 55-65% >65%
People 315 2,242 6,097 3,026 3,034 714
Deaths 163 986 2,533 1,162 1,150 289
Adjusted HR 1.37 (1.16-1.63) 1.21 (1.11-1.32) 1.11 (1.03-1.19) Ref (1.0) 1.01 (0.93-1.10) 1.16 (1.02-1.33)

The Hazard Ratios give us the relative risks. It is claimed that, relative to the group chosen as the reference group (50-55% carbohydrate intake), a carb intake <30% gives a relative risk of 1.37 (37% greater) and a carb intake of >65% gives a relative risk of 1.16 (16% greater). Neither of these “strengths of association” come anywhere close to double and so the possibility of carbohydrate intake being causal in these relative risks can be discounted. We could stop here. However, there are many more issues to note…

The numbers are always reported as relative risk; the absolute numbers are usually not worth getting excited about. In the ARIC study, there were 6,283 deaths among 15,428 adults who were studied for 25 years having been recruited between the ages of 45-64 years. The average age was 54, so an average person would have been 80 at the end of the follow-up period, so survivors did pretty well. The overall death rate in this study was 40%. The annual death rate was 1.63%.

A 37% difference on an annual death rate of 1.63% is the difference between a death rate of 1.38% and 1.88% (Ref 7). That’s the absolute difference and that’s in middle aged people followed for 25 years.

3) The Food Frequency Questionnaire.

Each population study relies for dietary information on a self-reported recall of what one ate some time ago. The paper reported that the ARIC study used the Harvard Food Frequency Questionnaire (FFQ), which was developed by Walter Willett – one of the paper authors – and colleagues. This questionnaire involved asking people how often, on average, over the previous year, they consumed ‘standard portion sizes’ of each of 66 items. Credit to Nina Teicholz for getting hold of the precise questionnaire used in the ARIC study (Ref 8). If you only look at one reference, please let it be this one – there’s nothing like seeing a FFQ first hand. As you can see, there were nine possible responses, ranging from “never” to “six or more times a day.”

Can you remember what you ate last year? How standard were your portions? Did you have 5-6 ‘pats’ of butter a week or did it tip over to 1 a day? What’s a pat anyway? Did your diet then stay the same for 20-25 years?

This questionnaire was administered when the participants were first recruited (Baseline – between 1987 and 1989). It was repeated between 1993 and 1995 at Visit 3. The paper reported that, up to Visit 3, carbohydrate intake from Visit 1 (baseline) was used for the analysis – obviously. After Visit 3, “the cumulative average of carbohydrate intake was calculated on the basis of the mean [average] of baseline and Visit 3 FFQ responses.” It was assumed therefore, that the ARIC participants did not change their diet for approximately 20 years, even if they did change their diet between 1987-89 and 1993-95. That’s a big assumption.

There was a more serious assumption. The paper reported: “We did not update carbohydrate intake exposures of participants that developed heart disease, diabetes, and stroke before Visit 3, to reduce potential confounding from changes in diet that could arise from the diagnosis of these diseases.” This means, if Fred is in the lower carb group at baseline and he develops diabetes and goes to see a dietician who tells him to eat ‘healthy’ whole grains, he will be more likely to die (from diabetes), but this will be a death attributed to the lower carb group. Conversely, however, if Sally is in the moderate carb group at baseline and she develops heart disease and decides to cut her carb intake, she will be more likely to die (from heart disease), but this will be a death attributed to the moderate carb group. Ostensibly, this appears to be a reasonable assumption. In this paper it isn’t, for two reasons:

i) Conventional dietary advice is to consume approximately 55% of one’s diet in the form of carbohydrate. If someone in the ARIC study developed cardiovascular disease or diabetes between the two questionnaires (i.e. between 1987-89 and 1993-95), they would be in the healthcare system. As a result, they would be advised to consume c. 55% of their diet in the form of carbohydrate. This period pre-dated the literature, which has grown in the past decade, showing the benefit of very low carbohydrate diets for obesity, diabetes and chronic conditions (Ref 9) and hence no individual would have been advised to cut their carbohydrate intake following a health diagnosis. People who developed a life threatening chronic condition would, therefore, have wrongly stayed assigned to a lower carbohydrate group rather than being reassigned to a higher carbohydrate group. The converse would almost certainly not have happened.

ii) We’ll come on to the small comparator group issue soon, but it has an impact with this assumption too. You can see in Table 1 above that the <30% carb group is by far the smallest – just 315 people. Because it is so small, it is far more sensitive to small changes. It would take only 68 people to be reallocated from the <30% carb group (if they were diagnosed with a condition and advised to increase their carb intake and then assigned to their new carb group) for this group to have the same (unadjusted) death rate as the ‘optimal’ carb group (down from 51.7% to 38.4%). If, conversely, 68 people from the ‘optimal’ carb group changed their carb intake and were reallocated from that group, the death rate in this group would barely drop a percentage point (from 38.4% to 37%).

The assumption not to revise the carb group for those diagnosed with a serious condition was highly unfavourable to the lowest carb intake group.

We know that there is something seriously wrong with the FFQ data in this paper because of the average calorie intake in the characteristics table. The characteristics table splits the 15,428 people into five equal groups (quintiles) from lower carb intake to higher carb intake. The calorie intake ranges from 1,558 calories per day in the lower carbohydrate quintile to 1,660 calories in the middle carbohydrate intake group. No group apparently consumed more than an average of 1,660 calories a day in this American study in the past 25 years. Really?!

4) Incomplete adjustment of the data.

Table 1 above contains extracted data from the appendix to the paper. This reported that the data were adjusted for age, race, gender, which test center the person attended, total energy consumption, diabetes, cigarette smoking, physical activity, income level and education. BMI was not included in factors adjusted for and thus this appears to have been an adjustment omission. That is an additional flaw of this paper.

Dietary epidemiology studies are rarely adjusted for the whole diet. This paper did not adjust for any other aspect of the diet. Carbohydrate intake was the sole focus of the paper – the type of carbohydrate, or intake of vegetables oils, or processed meat was not adjusted for in the ARIC study.

Carbohydrate can be found in the following diverse items in the FFQ used in this study: dairy; fruit; vegetables (and these vary from broccoli to yams); candy; pie; cakes; cookies; bread (whole wheat or not); cereal; rice; pasta (whole grains or not); chips; nuts and sugar. Carbohydrate from green vegetables, nuts and dairy products is very different to carbohydrate from chocolate, chips and cookies.

None of this was adjusted for. The whole diet was not taken into account (Ref 10).

5) The healthy person confounder.

I wondered what kind of person would be consuming a lower carbohydrate diet in the late 1980s/early 1990s (when the questionnaire was done). The characteristics table in the paper tells us exactly what kind of person was in the lowest carbohydrate group. They were far more likely to be: male; diabetic; current smokers; with a higher BMI; and far less likely to be in the highest exercise category. The ARIC study adjusted for most of these characteristic, but not BMI. Even if they had adjusted for all characteristics, as I often say, you can’t adjust for a whole type of person. An unhealthy person is not differentiated by carbohydrate intake alone once some unhealthy characteristics have been adjusted for.

I joked in a recent review of a classic whole grain epidemiological paper that I expect people who consume whole grains regularly (that’s <5% of Americans) to: not smoke; not drink; be affluent; do yoga; be slim; shop at Whole Foods; eat at restaurants, not takeaways; have children called Olivia and Tarquin and so on. The whole grain consumption is a marker of good health, not the maker of good health (Ref 11).

Visualise for a moment the unhealthy person confounder in this study – the overweight, smoking, diabetic, male couch-potato, and then we can turn to the first error unique to this paper, as opposed to all epidemiological papers…

6) Failure to adjust for a serious confounder (alcohol).

Many thanks to George Henderson (who tweets as @puddleg – well worth a follow), for spotting that there was no mention of alcohol in the main paper at all. This means that alcohol was not accounted for or adjusted for. This is also despite the fact that the Food Frequency Questionnaire used in the ARIC study did include questions about alcohol (beer, wine and liquor to be precise). Maybe alcohol accounted for the missing calories in the total energy intake numbers? The whole alcohol issue is a major error.

There is also a confounder of this error in that, we know from the characteristics table in the main paper that those in the lower carb intake group were more likely to be smokers. There is a positive association between smoking and drinking: smokers are more likely to be drinkers (of coffee and alcoholic drinks). This unexplained omission was highly unfavourable to the lower carb intake group.

7) The small comparator group issue.

We now come on to the single biggest issue in my view – indeed it may be fair to call it a manipulation. It’s what I call the “small comparator group issue.” I have explained this here. It is summarised here again for completeness:

The characteristics table in the main paper split the 15,428 people into equal groups (of 3,085-3,086) from the lowest to the highest carb intake. This is the objective way to review data, because there is no argument that you drew the line in a particular place to bias the finding. The appendix revealed what had been done to produce the U-shaped finding that grabbed the headlines. The numbers were extracted in Table 1 above – repeated here for convenience:


Table 1

Carb range <30% 30-40% 40-50% 50-55% 55-65% >65%
People 315 2,242 6,097 3,026 3,034 714
Deaths 163 986 2,533 1,162 1,150 289
Adjusted HR 1.37 (1.16-1.63) 1.21 (1.11-1.32) 1.11 (1.03-1.19) Ref (1.0) 1.01 (0.93-1.10) 1.16 (1.02-1.33)

The groups have been subjectively chosen – not even the carb ranges are even. Most covered a 10% range (e.g. 40-50%), but the range chosen for the ‘optimal’ group (50-55%) was just 5% wide. This placed as many as 6,097 people in one group and as few as 315 in another. The subjective group divisions introduced what I call “the small comparator group issue.”

If 20 children go skiing – 2 of them with autism – and 2 children die in an avalanche – 1 with autism and 1 without – the death rate for the non-autistic children is 1 in 18 (5.5%) and the death rate for the autistic children is 1 in 2 (50%). Can you see how bad (or good?) you can make things look with a small comparator group?

To then get the media headlines about life expectancy, the researchers applied a statistical technique (called Kaplan-Meier estimates) (Ref 12) to try to estimate when people would die and conversely life expectancy. This is purely a statistical exercise – we don’t know when people will die. We just know how many have died so far.

This exercise resulted in the claim “we estimated that a 50-year-old participant with intake of less than 30% of energy from carbohydrate would have a projected life expectancy of 29·1 years, compared with 33·1 years for a participant who consumed 50–55% of energy from carbohydrate… Similarly, we estimated that a 50-year-old participant with high carbohydrate intake (>65% of energy from carbohydrate) would have a projected life expectancy of 32·0 years, compared with 33·1 years for a participant who consumed 50–55% of energy from carbohydrate.”

Do you see how both of these claims have used the small comparator group extremes to make the reference group look better?

Back to the children skiing… If we were to use the data we have so far (50% of autistic children died and 5.5% of non-autistic children died) and to extrapolate this out to predict survival, life expectancy for the autistic children looks catastrophic. This is exactly what has happened with the small groups – <30% carb and >65% carb – in this study.

8) The selection of the reference group.

Food Frequency Questionnaires have so many limitations; there are arguments that they are worthless. They have the fewest limitations when they study the extremes. People are far more likely to accurately recall if they never have something and if they have something six or more times a day. Secondly, dietary studies are at their ‘least worst’ when they compare the lowest intake with the highest intake group for a particular food item, for example eggs, and not when they try to aggregate something found in many different foods (in this case carbohydrate).

This study featured both of these limitations. First the researchers subjectively chose unevenly distributed groups (Table 1) and then they used the narrowest group chosen (50-55%) as the reference group. This chosen reference group was in the middle of the groups – far from the extremes. This is the least robust part of a FFQ – where someone is trying to recall and estimate if they consumed a table spoon of salad dressing four times a week, or was it five? Then, this was confounded further by the study looking at many food items and not just one. What exactly was that small, selectively chosen, reference group representing? High green vegetables, whole grains and nuts or moderate candy, cakes and cookies?

9) The meta-analysis

Part 2 of the paper took the ARIC (US) study above and claimed to have combined it with seven other population studies in a meta-analysis. Three of the seven other studies were from Europe: Lagiou et al – Swedish women; Nilsson et al – Swedish men and women; and Trichopoulou et al – Greek men and women. Two were from the US. These were the Harvard favourites – the female Nurses Health Study and the male Health Professionals Follow-up Study. Both of these studies were authored by Fung et al. The final two studies were the Nippon study from Japan and the multinational PURE study (Ref 13).

The purpose of this part of the paper seems to have been to corroborate the U-shaped curve finding claimed in Part 1. The researchers managed to do this, but not without a couple of magic tricks…

i) On p3, under “Statistical analysis”, the paper reported “… papers were eligible for inclusion if they … adjusted for at least three of the following factors: age, sex, obesity, smoking status, diabetes, hypertension, hypercholesterolaemia, history of cardio-vascular disease, and family history of cardiovascular disease.” That means that any of the included studies need not have adjusted for many vital factors that would determine all-cause mortality, while having nothing whatsoever to do with carbohydrate consumption.

ii) On p3, under “Statistical analysis” again, the researchers described their meta-analysis as an update of a previously published meta-analysis from 2013 (Ref 14). The 2013 meta-analysis, by Noto et al, included Lagiou et al, Nilsson et al, Trichopoulou et al, and the two Fung et al studies (NHS and HPFS). The latter two were combined into one finding, so there were four sets of numbers in total. The ‘updated’ meta-analysis simply added the ARIC study to the Noto et al meta-analysis. Other than this, the exact same Hazard Ratios from the Noto et al paper were used. The narrative even reported “This relationship remained significant if the ARIC study was excluded from the analysis (1·31, 1·07–1·58).” Well it would do, because the meta-analysis without ARIC was an exact repeat of the 2013 Noto et al meta-analysis. This was hardly an update. The next point may help to explain why the researchers did this…

iii) The researchers decided to split their meta-analysis into two parts: Part A contained the three studies from Europe along with the Fung et al combined result and the ARIC study from the US; Part B contained just the Japanese and PURE study. The rationale for this was given as: “Because there was significantly lower consumption of carbohydrate in European and North American regions compared with Asian countries, low-income countries, and multinational cohorts, studies fell into two categories in the meta-analysis: North American and European studies (mean carbohydrate intake approximately 50%) that compared low carbohydrate diets with primarily moderate carbohydrate consumption as the reference, and Asian and multinational studies (mean carbohydrate intake approximately 61%) that compared high carbohydrate consumption with moderate carbohydrate consumption as the reference.”

I interpret the rational as – if we pool them all together we don’t get a significant result, but if we pool Europe and America (which are lower in carb anyway, but lower than average carb is relatively worse) and if we pool Japan with the multi-national study (these two being higher in carb, but higher than average carb is relatively worse), then we get two significant results.

The PURE study covers Asia, Africa, North and South America, the Middle East and Europe. If the researchers insist on isolating Europe and North America, then the PURE data for Europe and North America should be put in that meta-analysis. I’m not sure if the rationale for ‘what’s left’ is higher carbohydrate regions, or poor regions, or Asian regions.

The PURE data tell us that the high carb regions (averaging above 60%) should be China, south Asia and Africa, which I think you’ll find are irredeemably confounded by these regions being poor and the diet being high carbohydrate as a consequence of low affluence. If the researchers want to ignore Africa, South America and the Middle East altogether and isolate Asia as a region, then the Japanese study could be combined with the Asian regions from the PURE study, since PURE helpfully separates the Asian regions (their Fig 2A). For interest, the PURE study found that there was no statistical significance for the highest intake of carbohydrate vs. the lowest in Asian regions (1·09, 0·94–1·26).

My review of the meta-analysis part of the paper suggests that it is a ‘fudge’ to provide support to the ‘sweet spot’ for carbohydrate claim made in Part 1 of the paper (the ARIC study).


10) The claims related to food ‘exchanges.’

The final part of the paper reported that mortality results varied depending on the source of macronutrients. It was claimed that “mortality increased when carbohydrates were exchanged for animal-derived fat or protein and mortality decreased when the substitutions were plant-based.”

This is so disingenuous, it’s difficult to know where to start. When the aforementioned George Henderson and I worked together on the recent UK saturated fat rebuttal (Ref 15), George really hammered home to me how often the ‘exchanging/swapping/replacing’ foods argument is made in epidemiological studies and yet this doesn’t happen. Swapping one food out and another in is the domain of randomised controlled trials (RCTs). Epidemiological studies collect information from Food Frequency Questionnaires. An individual may consume less carbohydrate and more animal products but a) they didn’t swap one for the other and b) individuals are not studied. Epidemiological studies aggregate all the people and make general statements about people in the same group.

This paper aggregated people with lower carbohydrate and higher fat and protein from animal foods. It also aggregated people with lower carbohydrate and higher fat and protein from plant sources (Ref 16). The data for this part of the paper come from the ARIC study again.

The paper helpfully shares some of the observations, which should have been called dietary confounders. (Direct extracts from the paper are in italics and quotation marks; my comment follows):

– “The plant-based low carbohydrate dietary score was associated with higher average intake of vegetables but lower fruit intake.” So, did low fruit (sugar) intake confer any observed benefit?

– “By contrast, the animal-based low carbohydrate dietary score was associated with lower average intake of both fruit and vegetables.” So, did the lack of vegetables confer any observed detriment?

– “Overall, total protein intake was higher in the animal-based diet.” So, did higher protein intake confer any observed detriment?

– The study determined the five foods that differed most significantly between the highest and lowest groups of animal-based and plant-based low carbohydrate dietary score: “The animal-based low carbohydrate diet had more servings per day than did higher carbohydrate diets of beef, pork, and lamb as the main dish; beef, pork, and lamb as a side dish; chicken with the skin on; chicken with the skin off; and cheese.” The more accurate description from the FFQ is “beef, pork or lamb as a sandwich or mixed dish, e.g. stew, casserole, lasagna, etc.” So, now we’re talking bacon sandwiches, microwave lasagna and lamb curries. For chicken intake in the US, I suspect we’re talking KFC (an American institution since 1930).

The plant-based low carbohydrate diet had more servings per day of nuts, peanut butter, dark or grain breads, chocolate, and white bread than did higher carbohydrate diets.” In contrast, this list doesn’t look like a ready meal or takeaway menu. This part of the paper confirmed the “healthy person confounder”, or, in this case – the unhealthy male, smoking, drinking, overweight, diabetic, takeaway lover!

Close

Even if this study had analysed groups fairly in the quintiles in the characteristics table…

Even if the Food Frequency Questionnaire (FFQ) had been robust and accurately reflective of what people actually ate during the whole 25 year study…

Even if people had been allocated properly to reflect their actual carb consumption following a health diagnosis…

Even if an average calorie intake of 1,560-1,660 could be explained…

Even if the study had adjusted for the whole diet…

Even if ‘carbohydrates’ didn’t mean tens of different things (from kale to cake)…

Even if the paper had managed to overcome the whole ‘healthy person’ confounder…

Even if alcohol had been taken into account and adjusted for…

Even if the researchers hadn’t manipulated the data to benefit from the small group comparator advantage…

Even if the life expectancy had been calculated fairly, without this significant small comparator group manipulation…

Even if the reference group had been set at the most robust spectrum of the quintiles (the extremes, not the middle)…

Even if the subject under examination (an entire macronutrient) were suitable for averaging across already limited FFQs…

Even if the strength of association had been double…

Even if examination of the Bradford Hill criteria had established that causation might be likely…

…the purpose of epidemiological studies is to establish relationships that should then be tested in randomised controlled trials.

Even if all of those ‘even ifs’ had stacked up, the researchers would then merely have had something to test in a randomised controlled trial…

As Professor Noakes once tweeted, if epidemiology has validity, RCTs will back it up. Over to you Professor Willett to test your hypothesis!

References

Ref 1: https://www.bbc.co.uk/news/health-45195474

Ref 2: https://edition.cnn.com/2018/08/17/health/low-carb-high-carbohydrate-diet-risk-of-death-intl/index.html

Ref 3: Seidelmann SB, Claggett B, Cheng S, et al. Dietary carbohydrate intake and mortality: a prospective cohort study and meta-analysis. The Lancet Public Health 2018. https://www.thelancet.com/journals/lanpub/article/PIIS2468-2667(18)30135-X/fulltext

Ref 4: https://www.smh.com.au/lifestyle/health-and-wellness/people-on-low-carb-diets-die-younger-says-science-20180819-p4zye3.html

Ref 5: https://jandonline.org/article/S2212-2672(17)31238-8/pdf

Ref 6: https://www.zoeharcombe.com/2016/09/the-bradford-hill-criteria/

Ref 7: I keep a spreadsheet to calculate this in the simplest way possible. The goal in this particular example is to find the two numbers that maintain the average of 1.63%, while one is 37% bigger than the other.

Ref 8: https://regepi.bwh.harvard.edu/health/FFQ/files/80out.pdf

Ref 9: Westman EC, Yancy WS, Mavropoulos JC, Marquart M, McDuffie JR. The effect of a low-carbohydrate, ketogenic diet versus a low-glycemic index diet on glycemic control in type 2 diabetes mellitus. Nutrition & metabolism. 2008.

Feinman RD, Pogozelski WK, Astrup A, et al. Dietary Carbohydrate restriction as the first approach in diabetes management. Critical review and evidence base. Nutrition. 2014.

Hallberg et al. “Effectiveness and Safety of a Novel Care Model for the Management of Type 2 Diabetes at 1 Year: An Open-Label, Non-Randomized, Controlled Study.” Diabetes Therapy. 2018. (https://link.springer.com/article/10.1007%2Fs13300-018-0373-9)

Ref 10: The adjustment generally looked odd. The table below contains the data from the appendix. I added in a line (where it says ZH) for an unadjusted HR. This simply takes the 50-55% group as the reference point of 1.0 and then works out what the death rate in each group would be relative to 1.0/ This gives an HR of 1.35 (with no adjustment) for the lower carb group.

Given that the lower carb consumers had most of the characteristics stacked against them (gender, diabetes, smoking, BMI and lower exercise), I would expect the bottom line of that table – the adjusted risk ratio (HR) – to have reduced the ZH line (my calculation with no adjustment) substantially. You can see that it has barely moved for the lower carb group. That doesn’t make sense

Carb range <30% 30-40% 40-50% 50-55% 55-65% >65%
People 315 2,242 6,097 3,026 3,034 714
Deaths 163 986 2,533 1,162 1,150 289
Death rate 51.7% 44.0% 41.5% 38.4% 37.9% 40.5%
ZH – no adjust[1] 1.35 1.15 1.08 1.00 0.99 1.05
Adjusted HR 1.37 (1.16-1.63) 1.21 (1.11-1.32) 1.11 (1.03-1.19) Ref 1.01 (0.93-1.10) 1.16 (1.02-1.33)

Ref 11: https://www.zoeharcombe.com/2018/08/whole-grains-cvd-cancer-mortality/

Ref 12: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3059453/

Ref 13: Dehghan M, Mente A, Zhang X, et al. Associations of fats and carbohydrate intake with cardiovascular disease and mortality in 18 countries from five continents (PURE): a prospective cohort study. The Lancet 2017 doi: 10.1016/S0140-6736(17)32252-3

Ref 14: Noto H, Goto A, Tsujimoto T, Noda M. Low-carbohydrate diets and all-cause mortality: a systematic review and meta-analysis of observational studies. PLoS One 2013.

Ref 15: https://www.zoeharcombe.com/2018/07/saturated-fat-consultation-sacn-my-response/

Ref 16: The paper described “We created animal-based and plant-based scores by dividing participants into deciles for either animal-derived or plant-derived fat and protein, and carbohydrate intake, expressed as a percentage of energy as previously described. For carbohydrate, participants in the lowest decile received 10 points, whereas participants in the highest decile received 1 point. The order was reversed for animal-derived or plant-derived fat and protein, so that the highest score represented low carbohydrate and high animal-derived or plant-derived fat and protein intake.”

39 thoughts on “Low carb diets could shorten life (really?!)

  • July 17, 2019 at 4:51 am
    Permalink

    Awesome response to a terrible paper. Statistical shenanigans and not a real low carb diet in site.

  • May 13, 2019 at 11:19 am
    Permalink

    I’m all for keto diets to loose weight. But cut the cruelty(animal foods) out. Cholesterol, saturated fat and heme iron is not necessary and may have negative health effects anyways.

  • November 27, 2018 at 8:05 pm
    Permalink

    I have being following your bloggs over two years now, and it has always been interesting. refreshing, educative and eye open for some of us that are just coming up nutrition path. Thank you Dr. Harcombe, but how do we get evidence based articles to read and reference if this can come from “The Lancet Public Health Journal” that meant to be among scientific reference articles?

    • November 28, 2018 at 8:34 am
      Permalink

      Hi Libby
      Thank you for your lovely words:-)
      Sadly we can’t stop the non-sense – we can just keep exposing it!
      Best wishes – Zoe

  • September 2, 2018 at 3:29 am
    Permalink

    Congrats on the great rebbutal. If I am not mistaken, it would take only 43 people, not 68. (163-40=120… 120/315=0.38). Am I wrong?
    “It would take only 68 people to be reallocated from the <30% carb group (if they were diagnosed with a condition and advised to increase their carb intake and then assigned to their new carb group) for this group to have the same (unadjusted) death rate as the ‘optimal’ carb group (down from 51.7% to 38.4%)."

    • September 2, 2018 at 8:56 am
      Permalink

      Thank you!
      Sadly I think you are wrong – you need to take the number from the numerator and the denominator to properly remove them from the group. So it’s 163 – 68 and 315 – 68.
      Best wishes – Zoe

  • August 31, 2018 at 7:44 pm
    Permalink

    I’be been watching for years how the ketogenic thing, after tons of evidence of its beneficial effects, is being diluted and I wondered why is this happening – Modified Atkins, etc.. Just yesterday our opinion article is published “Ketogenic Ratio Determines Metabolic Effects of Macronutrients and Prevents Interpretive Bias” -about gradual disappearing of the century-long criterium.” I don’t want to look promoting it but in case you find it helpful, it’s here: https://doi.org/10.3389/fnut.2018.00075

    • October 28, 2018 at 10:56 pm
      Permalink

      Thank you. I’ll promote it – you had me at Woodyatt.
      Those who do not remember history are condemned to the reinvention of wheels.

  • August 29, 2018 at 9:55 pm
    Permalink

    Hi Zoe,
    I was in Seoul, South korea when the news came out. It was all over the networks — CNN, BBC, and the like. I rushed and obtained a copy of the LANCET study and read it line by line. At a glance it looked convincing especially if you just glance at their “U-shaped” graph. I was worried that when I return back home to the states, I will be barraged with questions by patients and colleagues alike. Thank God for your article, it helped me dissect the paper. More power to you!
    –Patrick Ticman, M.D. ( Las Vegas)

  • August 29, 2018 at 9:06 pm
    Permalink

    Your point about small group comparisons is right on and points up the real problem. As the groups get smaller there is more variation. And less then 30 % is a big interval. Suppose the deaths started going down between 25 and 10 % — you know benefit of ketosis. Well then outliers will bias the outcome. In fact. suppose you plotted the data for deaths in every 2 % of carbohydrate consumption. Maybe the “U” shape is due to the smoothing out of the data. Maybe it is actually a “W”-shaped curve, or maybe a “WUWU”-shaped curve, in other words, maybe the likely error in taking two points from a food-frequency questionnaire every six years showed that the data has such high fluctuation that no information could be obtained. Group statistics always hides information (or the fact that there is too much variability to get any).

      • August 30, 2018 at 9:05 pm
        Permalink

        Oh for the days when the likes of Gerald Reaven used to plot individual results on graphs rather than use statistical shenanigans. Often you would see a bunch of subjects with similar responses differing in magnitude, then a few who zinged off in different directions.

    • September 12, 2020 at 9:09 pm
      Permalink

      Yes, it’s alway vital to see a scattergram diagram to see the hijinks researchers are pulling off.

  • August 29, 2018 at 2:00 pm
    Permalink

    And now there’s another one… They’re really piling it on aren’t they?

    https://www.eurekalert.org/pub_releases/2018-08/esoc-lcd082318.php
    “Munich, Germany – Aug. 28, 2018: Low carbohydrate diets are unsafe and should be avoided, according to a large study presented today at ESC Congress 2018.

    This study prospectively examined the relationship between low carbohydrate diets, all-cause death, and deaths from coronary heart disease, cerebrovascular disease (including stroke), and cancer in a nationally representative sample of 24,825 participants of the US National Health and Nutrition Examination Survey (NHANES) during 1999 to 2010. Compared to participants with the highest carbohydrate consumption, those with the lowest intake had a 32% higher risk of all-cause death over an average 6.4-year follow-up. In addition, risks of death from coronary heart disease, cerebrovascular disease, and cancer were increased by 51%, 50%, and 35%, respectively.

    The results were confirmed in a meta-analysis of seven prospective cohort studies with 447,506 participants and an average follow-up 15.6 years, which found 15%, 13%, and 8% increased risks in total, cardiovascular, and cancer mortality with low (compared to high) carbohydrate diets (see figure for total mortality).”

    —————–
    I did a bit of digging and found one of the NHANES food questionnaires. Here’s a typical question. I’m not sure what useful data you can get from the answers to this question:

    During the past month, how often did you eat any kind of cheese? Include cheese as a snack, G/Q/U cheese on burgers, sandwiches, and cheese in foods such as lasagna, quesadillas, or casseroles. {Please do not count cheese on pizza.}
    INTERVIEWER INSTRUCTIONS:
    INCLUDE: MACARONI AND CHEESE, ENCHILADAS.
    DO NOT INCLUDE: CREAM CHEESE OR CHEESES MADE FROM NON-DAIRY FOODS, SUCH AS SOY
    OR RICE, OR CHEESE ON PIZZA.

    Now it might just be me, but there’s a huge difference between cheese on its own as a snack and as an ingredient of a dish that’s mostly carbs (lasagna, enchiladas, etc.)

    I couldn’t tell you how many times in the last month I’d eaten any food, well coffee is an every day thing. I eat meat most days, but sometimes fish or eggs (or cheese) and I couldn’t tell you how many days I ate which, nor could I come close to estimating quantity per meal.

    • August 29, 2018 at 3:11 pm
      Permalink

      Hi Hugh
      This isn’t even a new study – it’s the exact same meta-analysis that was in Noto et al (2012) which was also the one to which Willett and co added one study (and then took it away bizarrely) – see point 9 on the post https://www.zoeharcombe.com/2018/08/low-carb-diets-could-shorten-life-really/

      It’s the same Lagiou et al, Nilsson et al, Trichopoulou et al, and the two Fung et al studies (NHS and HPFS). How many times do they want to re-publish the same meta-analyses FFS!?

      Best wishes- Zoe

      • September 4, 2018 at 4:36 pm
        Permalink

        Good to know…

        The anti-low carb people must be spending huge amounts of money bashing it. I guess the market for processed and plant-based foods is tanking, along with prescriptions for Januvia and insulin…

        • October 9, 2018 at 3:59 pm
          Permalink

          “The anti-low carb people must be spending huge amounts of money bashing it”

          I think this is the most likely explanation. How do we find their funding and donation sources?

      • September 12, 2020 at 9:46 pm
        Permalink

        How often will they republish the same old studies? Why do new research when you can republish the same ones, where you know what you will get?

        Suppose you do a new study and it gives results you don’t like? Then you are on the horns of a dilemma.
        You either have to publish a result you (and your sponsor don’t like) or bury your results. Either is bad for business.

  • August 27, 2018 at 2:50 pm
    Permalink

    Thank you again for an impressive demolition job!

    I’ve learned so much from your posts about how to read news reports of scientific studies in general, not just nutrition. It seems that sloppy science is everywhere these days. It seems that research is designed to prove whatever hypothesis the researchers and their backers want, not to test the hypothesis against reality.

  • August 26, 2018 at 7:09 pm
    Permalink

    Gosh, thank you so much for this breakdown. This stuff is borderline criminal, in that those headlines will certainly alter some people’s diets for the worse. I’ve already heard from naysayers in my own family. The food questionnaire issue, in particular, results in shockingly poor data. Appreciate the breakdown at the end, for future reference!

  • August 26, 2018 at 5:38 pm
    Permalink

    Brava Zoe, Brava! I was arguing this yesterday with a friend who was going to go higher carb based on this study.

  • August 26, 2018 at 2:06 pm
    Permalink

    Very interesting that for your number 7 point above one must go fishing around the appendix, a typical strategy in these Harvard studies (many of their other ones do not even supply that). Shenanigans all around.

  • August 25, 2018 at 1:36 pm
    Permalink

    Hi Dr Harcombe,
    Thank you for your demolition of poor science hyped up by the press (to sell more newspapers, perhaps).
    Appreciated.
    Regards
    robert lipp

    • August 26, 2018 at 6:38 pm
      Permalink

      Also to market more carbs and veganism.

      Not sure that this wan’t the best demolition job yet – I seem to have read a few!

      There must by now be hundreds of thousands of well controlled diabetics, and if you take into account the huge “Banting” groups in South Africa and Nigeria there are literally millions of people losing weight and vastly improving their health.

      What do all these people have in common? NO-ONE STUDIES THEM! Back in the day, I knew a number of diabetics who were excluded from studies for being “too well controlled” – in most cases the exclusion criterion was an HbA1c of 8%, in some cases 6.5%.

      DIRECT has been in the mainstream media a bit probably because it uses a starvation diet of commercial products. Where was VIRTA written up in the Guardian or covered on the BBC?

      Admittedly Aseem Malhotra gets a fair bit of air time and I’ve even seen David Unwin on occasion, but vegans get far more. IMO this is pseudoscience carefully designed to support current memes. That red meat is tricky stuff isn’t it, we ate it for millennia and then suddenly in the last half century it turned round and attacked us. Can’t possibly be the bun and the fries and the Big Gulp or the “heart healthy” vegetable oils.

      • August 27, 2018 at 2:46 pm
        Permalink

        It’s pretty sad when an A1c of 8% is considered too well-controlled. Given that the normal non-diabetic range is between 4% and 5.6%…

        And most of those “well-controlled” diabetics are on Metformin, insulin, and probably a statin and something for their blood pressure.

        But eating real low carb is considered “too difficult” for these poor diabetics to manage, even though the work involved in balancing carb intake and insulin is probably more time-consuming than not eating the carbs in the first place.

        • August 30, 2018 at 9:03 pm
          Permalink

          Yes you can’t market more drugs to people who already have an A1c below 6 so what’s the point of studying them?

          Back in the day when the ADA reckoned “medical nutrition therapy” could only reduce A1c by up to 1.8%, diabetics in their very own forum were routinely achieving 5 – 8% and even up to 13% reduction. None of these people were following the ADA diet. Hello ADA, your diet is wrong.

          Then there are the old diabetics, who are now dying of old age rather than complications, because they were put on low carb diets back when it was standard advice. Some of them had no problem keeping it up for decades. How soon we forget.

          • September 4, 2018 at 4:42 pm
            Permalink

            A friend of mine is downsizing. I went over to help him clear out stuff. I found a bunch of paperwork from his late wife’s initial diagnosis as a T1 diabetic from the Joslin clinic (before it was the Joslin) 1961 or so. I brought it home to study (with permission). She died about 4 years ago, after many years of failing health — kidney failure and heart issues. She’d been a nutritionist specialising in diabetic diets, but was ADA all the way. I suspect her adherence to the ADA’s recommended diet contributed significantly to her ill health.

            I’ve only had time to look at a little of the stuff I brought home, but it seems that the carb requirement hasn’t changed in 50 years…

          • September 4, 2018 at 4:46 pm
            Permalink

            Wow! That’s really criminal :-(

  • August 24, 2018 at 7:21 pm
    Permalink

    If only i could aggregate your findings into a few killer observations / facts to counter other people’s gullibility / lack of attention span. Still great work as always.

    • August 25, 2018 at 7:52 am
      Permalink

      Hi bfbf334
      That’s a good point! One liner retorts to the people who thought this was science would be very useful! Maybe some of the “Even if…” at the ends would do it? Pick a few of your faves and lob them back…
      Best wishes – Zoe

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.