Posts Tagged ‘Bad Science’

Abstract dangers

Scientific journals aren’t for everyone. Journal articles use technical writing and can be rather dry. They can be long, they can be dull, they can show nothing new and exciting, or the research they describe can be so poorly thought out you wonder how a reviewer ever allowed the paper to go to the presses. Many good article are behind pay walls, so even if you want to read them, sometimes you can’t.


Fortunately an abstract of most papers can be found for free.  An abstract is a brief summation drawn up by the authors to get their point across.  Maybe it’s just me, but I sometimes think that abstracts can be a bit like movie trailers- they introduce the major players and they give you a general plot of the movie (and they try and hook you in by showing you all the good scenes).


Like a movie trailer, abstracts can be deceiving.  Take the trailer for The Matrix Reloaded– how excited were you when you first saw that trailer? How much did you wish the movie had never been made after you saw the actual feature?


Unfortunately, while in the cinematic world people are unlikely to act like they’ve seen the whole movie when all they’ve done is watch a trailer, in the world of scientific literature it often seems that people assume that reading the abstract is as good as reading the paper.


It is not.


The list of examples is endless, but this morning I stumbled across an example of this that finally pushed me to write about abstract abstraction.


It all started when I saw a tweet proclaiming “A high saturated fat mixed meal induces inflammation & insulin resistance & elevated glucose cf [compared to] other types of fats”.  Considering my interest in fats and my particular fondness for saturated fats you may not be surprised to hear that I decided to dig a little deeper.


The paper is from an open access journal. The full text is available here.


To be fair, the title of the paper is not quite as sensational as the tweet that led to it- The effect of two iso-caloric meals containing equal amounts of fats with a different fat composition on the inflammatory and metabolic markers in apparently healthy volunteers– but the “conclusions” offered in the abstract (the line that anyone who is just skimming the article will jump to) is rather dubious:


Metabolic and modest inflammatory changes occur within a few hours after the ingestion of a high SFA meal in apparently healthy adults.


I don’t have the time or the inclination to totally dismantle this paper (I really wonder how they did their statistics to show there was a significant difference), but I do want to point out how unwise it can be to draw conclusions from this abstract.


Let’s compare the methods sections. In the abstract, the authors say that healthy participants “were given two iso-caloric meals with similar amounts but different composition of fats: a meal high in monounsaturated fats (MUFA), and a meal high in saturated fat (SFA).”


The methods section in the paper reveals more detail:


The chosen meals represented two very popular meals habitually preferred by the general population: 1. Chicken sausages with fried potatoes, ketchup and mayonnaise (defined as SFA); 2. Pasta with olive oil, ketchup and nuts (defined as MUFA).




Two entirely different meals, and we’re supposed to believe that any differences in blood markers (of which I am skeptical) are due to the change in the type of fat- fat types that aren’t particularly well represented in at least one of the meals.  Chicken is not high in saturated fat.  Chicken fat is predominantly unsaturated, a combination of MUFA and polyunsaturated fats (PUFA), with less than a third of chicken fat being saturated.  What were the potatoes fried in? These days most things are fried in PUFA rich vegetable oils not SFA rich animal fats or coconut oil.  And mayonnaise? Mayonnaise contains very little saturated fat (because it’s usually made with PUFA-rich vegetable oils).  At least the second diet utilizes olive oil, which is rich in MUFA.


The authors state that they used the Israeli Food Database to calculate the breakdown of SFA:MUFA:PUFA in each diet and that the “SFA” and “MUFA” diets contained 24:33:17g and 8:51:14g respectively. Without knowing more about the ingredients (what fats and oils were used in the SFA diet and what nuts were used in the MUFA diet) it’s hard to know if the breakdown is accurate.  The meals are so different in every regard, it’s silly to quibble over the exact proportion of each fatty acid type.


The point of this post isn’t (or wasn’t) to pick this paper apart.  The purpose was to show that we should be cautious when drawing conclusions from abstracts.


The authors chose to say that any changes (that may or may not be real) occurred after “the ingestion of a high SFA meal”, but they could equally have said “after the consumption of mayonnaise (or potatoes)”… Likewise, they could have claimed that pasta (or nuts) “protects against metabolic changes induced by ketchup”. Of course, all of these claims would be ridiculous- though perhaps less ridiculous than suggesting any changes were due to ingestion of a high SFA meal (something they didn’t even test)!


Read Full Post »