Cherry Picking Science

Chapter summary

  • Science evolves quickly in some ways and slowly in others. Scientists dislike claiming anything is “proven”.
  • The author says most researchers cherry pick the literature that supports their case. He includes himself.
  • A good experiment controls all variables and changes only one or two. The author says many experiments of the past 20 years have not done this, which is understandable in epidemiology where people self-report.
  • Not all science is published. Null and inconclusive results seldom appear, so the published record leans one way.

An example: high-fructose corn syrup (HFCS)

  • White (2009), J Nutr: HFCS is maligned. Misconceptions abound, and HFCS is not uniquely responsible for obesity.
  • Morgan (2013), Pediatric Obesity: Reviewed six eligible studies (four systematic reviews, two meta-analyses) on HFCS drinks and childhood obesity. The evidence was inconclusive.
  • Ferder (2010), Current Hypertension Reports: Experimental and clinical evidence links HFCS to obesity and metabolic syndrome, and some changes appear even without weight gain.
  • Three peer-reviewed papers, three messages: cleared, inconclusive, harmful.

Where views differ

  • Cherry picking is a vice: Selective citing, hidden funding and unpublished null results mislead the public.
  • Cherry picking is human: The author says everyone does it. Researchers form a hypothesis, review the literature and report results. It is not always wilful.
  • Reviewers’ defence: Systematic reviews and meta-analyses exist to reduce cherry picking, but they depend on what was published and how studies were chosen.

How to read the evidence

  • Prefer large, long, well-controlled studies and systematic reviews to single papers.
  • Ask who paid for the research and who benefits from the conclusion.
  • Check whether the study controlled for weight, calories and lifestyle.
  • Look for the studies that disagree. If the story still holds, trust it more.
  • Remember a field full of “interests, prejudices, axes to grind, and products to sell”.

Take-away points

  • If scientists send mixed messages, laypeople will struggle. Do not be paralysed.
  • Test the advice on yourself: change one thing, measure, and decide.

References

  • White, J. (2009) Misconceptions about high-fructose corn syrup: is it uniquely responsible for obesity, reactive dicarbonyl compounds, and advanced glycation endproducts? J Nutr.
  • Morgan, R. (2013) Does consumption of high-fructose corn syrup beverages cause obesity in children? Pediatr Obes 8(4):249-254. doi:10.1111/j.2047-6310.2013.00173.x
  • Ferder, L. et al. (2010) The role of high-fructose corn syrup in metabolic syndrome and hypertension. Curr Hypertens Rep 12(2):105-112. doi:10.1007/s11906-010-0097-3
  • Taubes, G. (2010) Calories, fat or carbohydrates? garytaubes.com.
Scroll to Top