August 17, 2015
There are plenty of cases of scientific fraud that were covered by popular media as being the truth. Is there a huge hole in scientific communication that allows anyone to write anything and communicate it to millions of people as science? Is open access a part of this hole? Do we need more trust or more scepticism?
John Bohannon is a journalist known to open access advocates mostly for his “sting” that hit low-quality publishers. But, more recently, together with Gunter Frank MD and two journalists, Diana Löbl and Peter Onneken, he conducted a real study on humans, intentionally designed to create some false positives. The study “proved” that chocolate eating may cause weight loss, which was of course good news for a chocolate-addicted and weight-obsessed society. It might sound funny, but in fact Bohannon et. al’s study was not the first one to claim so. This was also a conclusion of the paper by Beatrice A. Golomb et. al, Association Between More Frequent Chocolate Consumption and Lower Body Mass Index, which was published in 2012 in JAMA Internal Medicine, a high impact journal. According to Google Scholar Golomb’s paper was cited 52 times. So although, the design of the Bohannon et. al study was intentionally bad, the conclusion should not be seen as something strange.
Bohannon’s group published the result of the bogus study in a low-quality open access journal (indexed in DOAJ!), that apparently has no editorial policy. But, they did not stop there. They send a bogus press release to the media, and as a result news covering their fake results has been spread by dozens of established, popular magazines and newspapers.
It might be seen as an alarming case, since completely fake results were published in a journal claiming to be scientific, and then were treated as science by journalists who passed it on to the general public, which was unaware that everything was part of a plot created by other journalists. Did Bohannon et. al prove that there is a huge hole in scientific communication that allows anyone to write anything and communicate it to millions of people as science? Is open access a part of this hole? I think that these are not the most appropriate questions. More important ones ask what science should really be and how people should look at it.
At the very beginning let’s ask whether there was anything new in the “chocolate scandal”? I think not, except for the fact that for the first time journalists, who are the part of the scientific communication themselves, intentionally used this system to fool society. Because the fact that scientists fake their work from time to time, and that these faked results are widely disseminated by popular media, was already well known.
Only 10 months before Bohannon announced that whole chocolate story was fake, a very similar case occurred, but not as a result of provocation. I mean here the retraction of the paper “proving” that green coffee pills help overweight subjects to lose body fat. The study was published in an open access journal, (indexed in DOAJ also) and before it was retracted, it was covered by doctor Mehmet Oz’s popular health talk show. Dr Oz informed his audience that: “This miracle pill can burn fat fast for anyone who wants to lose weight. This is very exciting and it’s breaking news.” Green coffee pills soon are a popular product, available in many countries, and for sure Dr. Oz helped their marketing. But this breaking news was fake. A retraction note written by the authors of the green coffee pills article says that: “The sponsors of the study cannot assure the validity of the data so we, Joe Vinson and Bryan Burnham, are retracting the paper.”
The story of the bogus pills was covered from beginning to end by Abby Phillip in the Washington Post in October 2014. Her articles clearly showed that if one has enough money, he or she can advertise his or her products in popular media with scientific aura.
But this is not only the case for the dietetic industry or for low-quality journals (which in both cases were open access). Simultaneously to the finale of Bohannon’s chocolate story, Science, which is an old, extremely renown and expensive toll access journal, retracted a paper which showed that a brief meeting with gay canvassers can change people’s opinions about same-sex marriage for a long time. Same sex marriages were hot topic in the US at the beginning of the year, so the paper, before its retraction, was covered by the Wall Street Journal and plenty of other popular media. Another impressive thing about the study was that it’s first author was a PhD student in political science. And in the social sciences, to publish in such an impact-full journal so early on is an excellent start to a career. But the study was retracted, because the author failed to show the raw data that his investigation was based on.
In all of these cases millions of people were informed about issues that might be important at least to some of them. In every case, the authority of science, which is seen as a well organized enterprise that produces truth was used to make information appear more trustworthy. As a result, people spent more money on green coffee beans and probably some gay rights advocacy organizations changed their strategies after being informed about the Science article. It is also possible that some of them are not aware of the retractions, and still think that their actions are based on academic knowledge, that was verified by the academic community of peers.
The list of famously false science that has been reported by mainstream media as being true is much longer. It also happened in the case of the alleged HIV vaccine, the case that ended with a jail sentence for the scientist who spiked rabbit blood to fake HIV vaccine results. I was also able to find a story from 2012 in the Iowa State Daily, which informs readers about the research for a HIV vaccine, but fails to provide an update that year after publication of the article where it came out that the alleged successes of the described project was based on a fraud. Thus, someone could still believe that a team from Iowa State University is close to developing an efficient HIV vaccine.
Let me close this short list with a case of a scientist, who falsely claimed (in the super-renown Science also) that he had successfully cloned a human embryo. Subsequently, it came out that he was falsifying almost all his results for several years, but in the meantime he made headlines all over the world.
How can we be careful enough?
Bohannon concluded in his article about the ‘chocolate issue’ that:
Hopefully our little experiment will make reporters and readers alike more skeptical. If a study doesn’t even list how many people took part in it, or makes a bold diet claim that’s “statistically significant” but doesn’t say how big the effect size is, you should wonder why. But for the most part, we don’t. Which is a pity, because journalists are becoming the de facto peer review system. And when we fail, the world is awash in junk science.
I think that asking about the number of participants is generally a good idea, but I am afraid that it will mostly help us in avoiding John Bohannon’s further “stings”. Because when one is falsifying his or her research, he or she can do it in various ways. A green coffee pills article by Joe Vinson and Bryan Burnham, a real-life fraud from the field of dietetic studies, would pass Bohannon’s test: both the number of subjects and the size of effect was provided in the paper and in the media articles covering the story. The researched group was very small (16), but this was not the reason why the study was eventually criticized by the Federal Trade Commission and retracted. The point is that ”the study’s lead investigator repeatedly altered the weights and other key measurements of the subjects, changed the length of the trial, and misstated which subjects were taking the placebo or GCA during the trial.”
What the real problem is?
In my opinion, we cannot avoid fraud and we also cannot recognize each case at a glance. Bohannon, in his article, “I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here’s How.” touched several interesting issues. The title itself is interesting and problematic, since without a proper background we can only say that there were several studies which indicate that dark chocolate may indeed have a positive influence on human metabolism and indirectly on weight, but of course we can’t say what the truth is. So, the problem is not that the team of journalists was able to fool people into thinking something totally ridiculous. The problem is that it is really easy to make headlines nowadays. And there is “too much noise” about some of very recent, sexy scientific papers and almost no public academic discussions of a high quality.
In fact, academia is an organized form of scepticism. It emerged because some people were curious about how the world looks, and were not satisfied with the dogmas offered by religion. Empiricism and scepticism, fact based discussions are at the heart of science. Science should not be treated as a different form of religion – something that we can uncritically believe in. Science is about constant criticism and it is quite normal that there are a lot of competing theories, point of views and ideas. Unfortunately, some of them are wrong, and some of them fraudulent. To be more sure that something is true we need time for discussion. But the media are in a constant hurry and they prefer to provide simple facts that raise awareness.
We need more scepticism and more openness in science. Because when science is open it is also more transparent. And if more people take part in the process of creating, discussing and disseminating academic knowledge, people will be less likely to believe in everything that they read.