It’s been a bad month for science hoaxes. First, almost everyone reported on that dodgy study about how there’s large quantities of fecal matter in beards (Media Watch on the contagion here). And now we find another widely reported study, on how chocolate can make you thin, was a deliberate hoax aimed at showing how the media doesn’t know shit about science.

Writing on iO9, John Bohannon, a specialist science journalist, reveals how he teamed up with a documentary team to chart how easily the media can be fooled.

It’s not that the study was an outright fake. It was actually done, with 16 participants. But with so few subjects, it’s relatively easy to achieve a statistically significant result, and with a helpful statistician on hand, Bohannon was able to come up with a newsworthy number. This was then submitted to the International Archives of Medicine, one of hundreds of journals popping up that look legitimate but don’t use a formal peer-review system, and that charge researchers to run their research. The $850 publication fee paid, the group then put out a snappy press release, and hey presto. Bohannon writes:

“We landed big fish before we even knew they were biting. Bild rushed their story out — “Those who eat chocolate stay slim!” — without contacting me at all. Soon we were in the Daily Star, the Irish Examiner, Cosmopolitan’s German website, the Times of India, both the German andIndian site of the Huffington Post, and even television news in Texas and an Australian morning talk show [that would be Channel Nine’s Today show].

“When reporters contacted me at all, they asked perfunctory questions. ‘Why do you think chocolate accelerates weight loss? Do you have any advice for our readers?’ Almost no one asked how many subjects we tested, and no one reported that number. Not a single reporter seems to have contacted an outside researcher. None is quoted.”

It seems to your correspondent that such dodgy reporting, or even outright hoaxes, are becoming more common. Speaking anecdotally, Michael Vagg, a clinical senior lecturer at Deakin University School of Medicine, says that’s certainly plausible, given an intensification of research expectations in academia that’s come about at the same time as the loss of many specialist reporters in the mainstream press.

“Academics at universities are under a lot of pressure to be productive, and that’s seen the growth of a number of vanity journals that publish without a lot of robust peer review. A lot of the really bad research, which I see in some media releases, is published in journals which I know have a low standard,” he told Crikey. “Some of them aren’t even on the PubMed database.”

Journalists wanting to know if a miracle new cure is legit wouldn’t be steered wrong by looking through the science database, a giant storehouse of “even vaguely reputable medical journals” which is free to access, Vagg notes. “For any reputable procedure, you’d expect to see a few hundred entries in PubMed on the pros and cons,” he said.

Not-for-profit bodies like the Australian Media Science Centre can also help steer journalists in the right direction, he adds. Although, to take the cynical view, that would require outlets to factor in a loss of credibility more than they do another click or view.