CEBM – Centre for Evidentially Biased Medicine

A friend pointed this post out to me, which I would not have seen otherwise, as this would not be a site I would likely visit. The authors of this blog are David Nunan and Kamal Mahtani (“bloggers” henceforth). My response, on the part of the blog that relates to this paper, is as follows.

1) The bloggers’ bias

The bloggers’ bias is immediately evident with the prejudice between Cochrane and non-Cochrane studies. Evidence should be judged on its own merits, not with pre-conceived opinions about what is better.

2) Conflicts of interest?

My consistency of interest and complete independence from any external funding has been addressed here. The bloggers have neglected to mention their own conflicts of interest. The CEBM is part of the Nuffield Department at Oxford University. Probably the single greatest campaigner against cholesterol (and thus supporter of the diet-heart hypothesis) in the UK is Professor Sir Rory Collins who heads up the Clinical Trial Service Unit (CTSU) at Oxford University. Collins was involved in the setting up of the CEBM. David Sackett “co-founded the CEBM and established close contact with the Clinical Trial Service Unit, directed by Richard Peto and Rory Collins...” Two years ago, the CTSU finally admitted that its pharmaceutical funding amounted to £268 million – no doubt this figure has risen since then. The bloggers thus enjoy salaries and pensions from the UK University richly rewarded by Big Pharma and known for its relentless attack on cholesterol and thus defence of the diet-heart hypothesis.

3) The Rose study

The Rose study [10] was an unusual study with two interventions and one control. It can therefore be presented in a number of ways. Looking at all-cause mortality, as the bloggers have done, Harcombe et al (2015) [1] treated the studies as separate interventions. Hooper et al (2011) [2] combined the interventions and reported RR as 3.85, 95% CI 0.51 to 29.20 (Analysis 1.1 and Analysis 1.2). Hooper et al (2015) [3] assigned the control event to the olive oil intervention and reported RR as 1.50, 95% CI 0.17 to 13.05 for the olive oil intervention and RR as 5.31, CI 0.32 to 89.44 for the corn oil intervention (Figures 3 and 5). Had the control event been assigned to the corn oil intervention, the RR’s would have been 2.32, 95% CI 0.30 to 17.92 for the corn oil intervention and 3.63, CI 0.20 to 65.44 for the olive oil intervention.

The key point is that none of this makes any difference. The weighting in meta-analysis assigned to Rose in Hooper et al 2011 was 0.1%. The weighting assigned to Rose in Hooper et al 2015 was 0.1% for the corn oil intervention and 0.1% for the olive oil intervention. The weighting assigned to Rose in Harcombe et al (2015) was 0.46% for the corn oil intervention and 0.41% for the olive oil intervention (the relatively higher weighting reflecting fewer studies available at the time the dietary guidelines were introduced.) Even with fewer interventions, Rose accounted for less than 1% of overall weighting in meta-analysis.

Good researchers would have spotted this. Unbiased researchers would have reported it.

4)  Events vs mortality

The methodology in Harcombe et al (2015) clearly stated that inclusion criteria were “data on all-cause mortality, CHD mortality and cholesterol measurements were available.” There are two reasons why we did not consider studying events. First, as the bloggers will know, deaths are hard end points, the factual accuracy of which is not open to debate; in contrast, events are soft end points, which are open to subjectivity. Second, and far more important in this particular examination, it is wholly inappropriate to study events when they have already happened! Five of the six studies only addressed secondary prevention and the sixth included both primary and secondary prevention participants. The death rate during these studies was 30%; almost one in three men didn’t make it to study end. Faced with these odds, an incident of angina, for example, is inconsequential. What matters is prevention of a fatal heart attack and all-cause mortality is important to ensure that a reduction in one type of mortality does not increase another.

Notwithstanding this, the following points need to be made about the bloggers’ Figure 2:

i) The data contain errors/bias (which could have been avoided without the bias of uncritically favouring Cochrane):

–      The Research Committee low fat diet [4] was excluded by the bloggers, with no reason given.

–      The Sydney Diet Heart Study [5], was also excluded by the bloggers without mention. This publication was available to the UK committee and it started in 1966, more than 10 years before the US committee recommendations, and thus could have been reviewed by the US committee were it interested in evidence. Event data are not available for the Sydney Diet Heart Study, which favours proponents of the diet-heart hypothesis, as the mortality results were significantly worse in the intervention group and therefore events would be expected to be similar.

–      The Oslo Diet Heart Study reference used by the bloggers was the 1966/1968 paper [6]. This was the data available after five years. The same 1966/1968 reference was given by Hooper et al (2015) in their Figures 3 (all-cause mortality), 5 (CVD mortality) and 6 (CVD events) [6]. However, the CVD mortality data (Fig. 5) came from the 11 year report of the Oslo Diet Heart Study [7] where more valuable, 11 year data, were available. 11 year data were also available for all-cause mortality in the 1970 publication (which was the publication used by Harcombe et al 2015). It is not known where the 5 year all-cause mortality data used by Hooper et al (2015) (Fig. 3) came from, although the full report states that published and unpublished (i.e. non-peer reviewed) data were used for the Oslo Diet Heart Study.

Leren et al moderated their enthusiasm for the diet intervention greatly over time. The ratio of control to diet deaths (if the Hooper et al data of 65 and 48 respectively are accepted) after 5 years was 1.35; the ratio of control to diet deaths after 11 years was 1.07. CVD event data only being used after 5 years served again to disadvantage the control.

–      The bloggers also accepted without question the LA Veterans (not Veterins) data for CVD events. I’m vaguely curious to know if the bloggers are able to source these numbers.

–      With respect to this study, the bloggers should have reported confounders for differences. As Harcombe et al (2015) noted in the discussion: “The LA Veterans study [8] recorded the lowest risk ratio for CHD deaths for the intervention group: 0.816 (Fig. 3). However there were important differences in the groups at study entry. The intervention group had 12 octogenarians, compared with 21 in the control group. 11% of the experiment group were heavy smokers (more than one pack a day) compared with 17% of the control group.” Attrition bias was also an issue with this study, with veterans free to leave and enter the home throughout the eight year duration, reducing participants from 422 and 424 in the control and diet groups to 169 and 100 respectively.

ii) The conclusion is wrong.

The purpose of Harcombe et al (2015) was to examine the best evidence available (RCTs) to dietary committees at the time of setting two dietary fat guidelines: 30% total fat and 10% saturated fat. It is interesting that the bloggers made no case against total fat (and nor does Hooper et al 2011 or 2015). However, the bloggers claim “the data available at the time of the 1977 US and 1983 UK guidelines appear to support a low saturated fat diet for the reduction of major cardiovascular events (their emphasis) and this is not true.

This bold statement is based on four studies [6, 8-10], one with two interventions (bloggers’ Fig. 2). Page 14 of Hooper et al (2015) reported on the dietary interventions first by documenting the RCTs that “clearly aimed to reduce saturated fat in their interventions arms, either directly or indirectly”; and second by listing the RCTs that “assessed SFA intake during the study period and showed that SFA intake in the intervention arm was statistically significantly lower than that in the control arm.” Of the four studies in the bloggers’ Fig. 2, while all stated an aim to reduced saturated fat, only one achieved it [8]. The one with the significant confounders for age and smoking and the one for which the bloggers are invited to source event data.

The bloggers cannot even state that SFA was reduced, let alone that it had any impact on CVD events.

5) The conclusions of Harcombe et al (2015)

Nothing in the CEBM blog changes the conclusions from Harcombe et al (2015), reinforced in Harcombe (et al) [11-14].

  • Dietary recommendations were introduced in the US (1977) and in the UK (1983) to (1) reduce overall fat consumption to 30% of total energy intake and (2) reduce saturated fat consumption to 10% of total energy intake.
  • No Randomised Controlled Trial (RCT) had tested government dietary fat recommendations before their introduction.
  • There was no significant difference in all-cause mortality or CHD mortality as a result of dietary intervention.
  • Mean serum cholesterol levels decreased in both intervention and control groups. Reductions were significantly higher in the intervention groups; this did not result in significant differences in CHD or all-cause mortality.
  • Recommendations were made for 276m people following 6 secondary studies of 2,467 men. Results lacked generalisability. (The bloggers make sweeping conclusions about saturated fat guidelines based on even fewer, 1,731, sick men.)
  • No study recommended change. Only one study made a positive claim for its intervention after five years [6] and subsequently this was moderated [7]. Rose et al [10] warned of possible harm by administering corn oil. The Research Committee [4] concluded “A low-fat diet has no place in the treatment of myocardial infarction” (p504). The MRC Soya-bean oil [9] intervention found no evidence that myocardial infarction relapse would be materially affected by unsaturated fat in the diet. The LA Veterans Study [8] reported that total longevity was not affected and expressed concern about unknown toxicity of their intervention. Woodhill et al [5] noted that survival was significantly better in the control than the diet group.
  • RCT evidence did not support the introduction of dietary fat guidelines.

6) The current evidence for extant dietary fat guidelines

The bloggers noted that “The 2015 Cochrane review identified eight outcomes that they considered to be clinically important.” I have addressed here [14] how Hooper et al (2015) were unique among all other robust meta-analyses in finding anything statistically significant against saturated fat (not least being the only reviewers to include four small studies about conditions other than CVD with largely unpublished data).

The eight important outcomes in Hooper et al (2015) were:

–        All-cause mortality – no significant difference;
–        Cardiovascular mortality – no significant difference;
–        Myocardial infarctions (MIs) – no significant difference;
–        Non-fatal MIs – no significant difference;
–        Stroke – no significant difference;
–        CHD mortality – no significant difference;
–        CHD events – no significant difference;
–        Combined cardiovascular events – RR 0.83 (0.72 to 0.96).

The conclusion of the report emphasised the one small significant finding. Had I written the abstract it would have read something like:

Having reviewed all RCT data available, dietary fat interventions have demonstrated: no significant difference for all-cause mortality; no significant difference for cardiovascular mortality; no significant difference for myocardial infarctions; no significant difference for non-fatal MIs; no significant difference for stroke; no significant difference for CHD mortality; no significant difference for CHD events and a small statistically significant difference for combined cardiovascular events, which ceases to be significant when subjected to sensitivity analysis for actual saturated fat reduction [credit to Dr. Trudi Deakin for spotting this one]. Had any significant results been found, they would not have been generalisable to whole populations, since not one study of healthy men and women was included.”

The bloggers obvious enthusiasm to try to find fault with those who challenge the diet-heart hypothesis (me, Nina Teicholz, Dr. Aseem Malhotra, Dr. Malcolm Kendrick, Dr. Uffe Ravnskov) does their claim to be evidence based no favour. They merely come across as biased in favour of the belief for which their University/salary is known.

Zoë Harcombe

References:

1. Harcombe Z, Baker JS, Cooper SM, et al. Evidence from randomised controlled trials did not support the introduction of dietary fat guidelines in 1977 and 1983: a systematic review and meta-analysis. Open Heart 2015;2(1) doi: 10.1136/openhrt-2014-000196.
2. Hooper L, Summerbell CD, Thompson R, et al. Reduced or modified dietary fat for preventing cardiovascular disease. Cochrane database of systematic reviews (Online) 2011(7):CD002137 doi: 10.1002/14651858.CD002137.pub2.
3. Hooper L, Martin N, Abdelhamid A, Davey Smith G. Reduction in saturated fat intake for cardiovascular disease. Cochrane Database of Systematic Reviews 2015(6) doi: 10.1002/14651858.CD011737.
4. Research Committee. Low-fat diet in myocardial infarction: A controlled trial. The Lancet 1965;2(7411):501-4
5. Woodhill JM, Palmer AJ, Leelarthaepin B, McGilchrist C, Blacket RB. Low fat, low cholesterol diet in secondary prevention of coronary heart disease. Advances in experimental medicine and biology 1978;109:317-30
6. Leren P. The effect of plasma-cholesterol-lowering diet in male survivors of myocardial infarction. A controlled clinical trial. Bull. N. Y. Acad. Med. 1968;44(8):1012-20
7. Leren P. The Oslo Diet-Heart Study. Circulation 1970;42: 935-42 doi: 10.1161/01.CIR.42.5.935.
8. Dayton S, Pearce ML, Hashomoto S, Dixon WJ, Tomiyasu U. A Controlled Clinical Trial of a Diet High in Unsaturated Fat in Preventing Complications of Atherosclerosis. Circulation 1969;40(1S2):II-1-II-63 doi: 10.1161/01.cir.40.1s2.ii-1.
9. Medical Research Council. Controlled trial of soya-bean oil in myocardial infarction: Report of a research committee to the Medical Research Council. The Lancet 1968;292(7570):693-700 doi: http://dx.doi.org/10.1016/S0140-6736(68)90746-0.
10. Rose GA, Thomson WB, Williams RT. Corn Oil in Treatment of Ischaemic Heart Disease. BMJ 1965;1(5449):1531-33 doi: 10.1136/bmj.1.5449.1531.
11. Harcombe Z, Baker J, Davies B. Evidence from prospective cohort studies does not support current dietary fat guidelines: A systematic review and meta-analysis. Br. J. Sports Med. 2016 doi: doi:10.1136/bjsports-2016-096550.
12. Harcombe Z, Baker JS, Davies B. Evidence from prospective cohort studies did not support the introduction of dietary fat guidelines in 1977 and 1983: a systematic review. Br. J. Sports Med. 2016 doi: 10.1136/bjsports-2016-096409.
13. Harcombe Z, Baker JS, DiNicolantonio JJ, Grace F, Davies B. Evidence from randomised controlled trials does not support current dietary fat guidelines: a systematic review and meta-analysis. Open Heart 2016;3(2) doi: 10.1136/openhrt-2016-000409.
14. Harcombe Z. Dietary fat guidelines have no evidence base: where next for public health nutritional advice? Br. J. Sports Med. 2016 doi: 10.1136/bjsports-2016-096734.

5 thoughts on “CEBM – Centre for Evidentially Biased Medicine

  • avatar
    November 8, 2016 at 12:11 am
    Permalink

    A despicable personal attack with only just argument: you didn’t disclose a conflict of interest (a non-existent one, in my opinion). That is not science-based-medicine, it is money-based-vendetta from Big Pharma.

    Reply
      • avatar
        November 8, 2016 at 11:21 pm
        Permalink

        Hi Zoë,

        my comment in the site of those bloggers has been censored.

        This is a screenshot I took of the comment “awaiting moderation”.

        I suppose they didn’t like it.

        Best regards,
        Vicente

        Reply
  • avatar
    November 7, 2016 at 10:36 pm
    Permalink

    This is excellent Zoe, it is a pleasure to read something by someone who knows the material as well as you do.
    It’s an interesting sideline in Hooper 2015 that all the reduction in minor events was seen in the trials where a) cholesterol was lowered by the intervention, and b) where replacement of saturated fat with polyunsaturated fat (not carbohydrate) was the form of SFA reduction. Yet this didn’t lower mortality.

    There’s a possible explanation for this – linoleic acid is supposed to lower LDL by increasing production of LDL receptors by the liver. The proposed advantage here is that there is then less time in circulation for the oxidisable material on the LDL particles, chief among which are the cholesteryl esters (CE) previously transferred from HDL by action of CETP, to become oxidised.
    The CE most prone to oxidation contains linoleic acid. Increases in dietary LA increase LA in CE proportionately.
    http://aje.oxfordjournals.org/content/145/12/1114.full.pdf
    So the effect of this intervention is to set up a race between the rate of CE clearance and the increase in CE oxidizability. People whose liver responds sensitively to LA may benefit (their livers may not, but that’s another story). People who have a sluggish LDL-R response to LA may be exposed to increased harm. Overall, as the RCTs demonstrate, the race is a draw. What benefit there is in terms of events can perhaps be explained by the essentiality of PUFAs and the higher vitamin E content of vegetable oil diets, as well as by the change in carbohydrate quality prescribed by many of these dietary interventions when the control diets’ baked desserts and pizza as sources of saturated fat were replaced.

    Reply
    • avatar
      November 8, 2016 at 8:11 am
      Permalink

      Hi George
      Many thanks for your support and this very interesting explanation. I have another one – plant sterols (http://thescipub.com/abstract/10.3844/ojbsci.2014.167.169)

      Cholesterol fell in both control and intervention. I would expect this post heart attack, as cholesterol would be high in the aftermath of a heart attack trying to repair cell damage. Plant sterols would explain the greater lowering of cholesterol in the interventions with vegetable oils but no benefit in mortality. Indeed our article on Plant Sterols showed lower cholesterol and higher heart disease – as you know – many of the interventions added oils while removing many processed foods. I would have expected the removed processed food to make a difference alone – maybe it did and was offset by the plant sterols!

      What we do know is 1) there is no evidence for current dietary fat guidelines (and therefore the accompanying carb default) and 2) we have not fully examined the potential harm of those guidelines!

      Best wishes – Zoe

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *