Putting Critical Thinking to the Test

Posted on by

As a quick follow-up to yesterday’s post on Critical Thinking, I wanted to highlight a recent study that has been getting a lot of attention. The study in question is titled “Are Organic Foods Safer or Healthier Than Conventional Alternatives?: A Systematic Review“. If you were to read the press release or the headlines of most major newspapers, websites, or television programs, you would conclude the study was that organic food offers no nutritional value over conventional food.

However, there are lots of other things going on with this study. The authors conclude 5 main findings;

  1. No robust evidence to support that organic food is more nutritious than conventional foods
  2. There were statistically higher levels of total phenols in organic produce, omega-3 fatty acids in organic milk and chicken, and vaccenic acid in organic chicken than in conventional products
  3. Conventional produce has a 30% higher risk for pesticide contamination than organic produce
  4. No difference in the risk for contamination of produce or animal products with pathogenic bacteria
  5. Conventional chicken and pork have a higher risk for contamination with bacteria resistant to 3 or more antibiotics than were organic alternatives

Before going into the significance of these findings, there are a couple of limitations of the study that should be considered. First, it is clear that there aren’t really a lot of good, controlled studies that have assessed the differences between an organic and conventional diets. The authors note that of the 5908 potentially relevant articles, only 17 evaluated health outcomes among human populations consuming organic and conventional foods, and 223 compared organic and conventional fruits, vegetables, grains, meats, poultry, milk, or eggs directly, with only 3 studies reporting on both human and food outcomes. The authors acknowledge the relatively small sample size, and I think that serves to highlight that there is room for more work to be done in this area. Another key fact of the small sample size, and the lack of specifically designed and controlled studies is that there is a lot of inherent variability both within and between studies. The authors note,

Three potential sources of heterogeneity [variation] are study methods (for example, measurement and sampling methods, study design, or organic standard used), physical factors (for example, season, weather, soil type, ripeness, cultivar, or storage practices), and variation within organic practices.

The fact that nutritional content can vary immensely, due to such differences as the genetic makeup of different varieties, the ripeness of the produce when it was picked, even the weather, makes the comparison between strictly organic and conventional practices pretty difficult. Controlling and accounting for those other factors within the individual studies was not frequently done, making accounting for them in the meta-analysis with sub-group analysis nearly impossible. Another of the major sources of variation is the use of the generic “organic” term, without specific criteria or discussion of the practices, the term organic can apply to a wide variety of practices. In fact the authors cite another review which found that variation within organic and conventional farming systems is likely as large as differences between the two systems. This scale of this variation alone is enough to make the comparisons between generic “organic” and generic “conventional” practices moot.

Delicious (although not superiorly nutritious) organic radishes from our garden.

But despite the limitations of the study there are some promising findings. I personally have never bought organic food for the perceived nutritional superiority. Rather the reduced pesticide burden (see the EWG’s Shopper’s Guide) in my foodstuffs and the environment, has been a deciding factor when I do buy organic. I also believe that with organically grown produce, a plants’ natural pest defense mechanisms (secondary metabolites) are more active and concentrated as compared to the conventional produce. These secondary metabolites are frequently touted as bestowing the health and anti-oxidant properties of many fruits and vegetables, and include alkaloids, glycosides, glucosinolates, terpenoids, and phenols. So given those reasons for purchasing organic, there are some very promising findings from the study that one can happily grab onto.

I think what bothered me the most about this article, was the way that it was presented in the media. The media grabbed on to the press release and the one line that stated “The evidence does not suggest marked health benefits from consuming organic versus conventional foods” while almost completely ignoring the decreased concentration of pesticide residue and increased concentration of total phenols in organic produce, and the fact that there was no difference in risk for contamination with pathogenic bacteria between either farming practice. Rather the focus of most coverage was on the lack of nutritional benefits of organic food. A headline like “Study sees no nutritional edge in organic food” is sure to bring in curious readers and grab attention, but given the overall tone of the article, it isn’t entirely appropriate. Unfortunately, most people in the public won’t read the full article, rather they will depend on what is presented to them by the media, and when the findings and presentation are so skewed, I find that troubling. Fortunately it wasn’t long before other news outlets and pundits started to consider the whole journal article and give it a thorough review and analysis (here is a nice summary from Grist).

I think a really good takeaway message from this article and the reaction it is garnering, is that if you want to get attention and citations, title your article with a question.

Related articles

1 thought on “Putting Critical Thinking to the Test”

  1. Pingback: Frankenfoodie Friday | On a Quasi-Related Note

Leave a comment