John Oliver Explains Why You Should Think Twice About 'Scientific Studies'

Last month, TV stations and internet outlets got hold of a piece of news positing that most dogs do not like being hugged by their owners. It was reported as a "study," meaning, at least in the minds of consumers, that it was fact. Science, after all, is infallible, right? Because so many of us have dogs, and because so many of us like to hug said dogs, the news was relevant and shocking and everything else that makes for a great, eye-catching headline. The only problem was that there was nothing scientific about it; it was just an animal expert giving his opinion after seeing photos of a bunch of dogs being hugged. This very website even had to run a correction clarifying this point.

This is only one example of the countless "scientific studies" the media reports as fact. On Sunday's episode of Last Week Tonight, John Oliver dove into all of the different ways these studies deceive consumers, and how dangerous it can be to ignore the nuances of how a study is conducted. Here are some examples—the results may shock you.

Chocolate Is Good for Pregnant Women

In 2016, a double-blind study was published in the American Journal of Obstetrics and Gynecology testing the effect different kinds of chocolate have on "placental function" and "risk of preeclampsia." There was no control group of women who didn't eat chocolate. Ultimately, the study found there was "no significant difference…in the rate of pre-eclampsia."

But a press release touted the "benefits of chocolate during pregnancy," which was all media outlets needed to produce a story. Yes, they could report, eating chocolate while pregnant is good. It was on TV. It was on the internet. We watched, and we clicked and, presumably, pregnant women ate, to  no particular health benefit.

Champagne Prevents Dementia

Again, this was a widely reported story, as seems to always be the case when anything resembling a "drinking = good for you" conclusion can be extrapolated. The only problem with this one is that it was only performed on rats.

Women Are More Open to Romance When They Are Full Instead of Hungry

Do we really need a study to tell us that someone is going to be more open to something if they aren't hungry? Of course not, but even if we did, this wouldn't be the study to do it as it only sampled 20 women. As Oliver says, "You cannot presume that 20 women can speak for all women. This is science, not the United States Senate."

Nevertheless, the study garnered a segment on Fox News, as well as coverage across the internet. The Today show's headline read, "Science proves it: Women who are hungry don’t care about romance."

Smelling Farts Prevents Cancer

In 2014, Time published a story about how "smelling farts might prevent cancer." So did several other media outlets. But in reality, as Oliver points out, the study merely said that "certain sulfide compounds are useful pharmacological tools to study mitochondrial dysfunction." The story in Time was later edited, with a correction noting that the initial article "incorrectly summarized the findings and implications of this study."

The scientists who conducted the study told Last Week Tonight that they still get phone calls and emails from radio and TV shows wanting them to talk about farts.

Driving While Dehydrated Is Just as Dangerous as Driving Drunk

This is ridiculous, obviously, but it was reported on Fox News and elsewhere. Not only was this study based on data from only 11 men, it was funded by the European Hydration Institution as well as, yes, Coca-Cola.

So why does this happen? Scientists are under constant pressure to publish findings and for those findings to be new and exciting. They need funding and they need tenure—this is what gets them. They will use something called p-hacking, which Oliver summarizes as, "collecting lots of variables and then playing with your data until you find something that counts as statistically significant but is probably meaningless." In short, science can be used to prove just about anything and, as the endless deluge of questionable "studies" shows, it has been.

Regardless of how dubious a study may be, news media gladly accept the baton and report the findings as groundbreaking. Outlets are under just as much pressure to generate new and exciting headlines. Oliver's conclusion is that the media need to be diligent about letting their consumers know the details of the studies they report as fact—like if a study is only based on data from a handful of people, or if the tests were conducted on rats or whatever other inconvenient nuances might make people think twice before loading up on red wine, champagne and chocolate as part of a quest for eternal youth.

Unfortunately, this is unlikely to happen. Consumers will need to be aware of what lies beneath the headlines, and know to take study findings with a grain of salt—which, as it turns out, both causes and cures cancer.