Via Samizdata, we find something very, very disturbing: a significant number of published medical studies are fraudulent:
As he described in a webinar last week, Ian Roberts, professor of epidemiology at the London School of Hygiene & Tropical Medicine, began to have doubts about the honest reporting of trials after a colleague asked if he knew that his systematic review showing the mannitol halved death from head injury was based on trials that had never happened. He didn’t, but he set about investigating the trials and confirmed that they hadn’t ever happened. They all had a lead author who purported to come from an institution that didn’t exist and who killed himself a few years later. The trials were all published in prestigious neurosurgery journals and had multiple co-authors. None of the co-authors had contributed patients to the trials, and some didn’t know that they were co-authors until after the trials were published. When Roberts contacted one of the journals the editor responded that “I wouldn’t trust the data.” Why, Roberts wondered, did he publish the trial? None of the trials have been retracted.
...
Mol, like Roberts, has conducted systematic reviews only to realise that most of the trials included either were zombie trials that were fatally flawed or were untrustworthy. What, he asked, is the scale of the problem? Although retractions are increasing, only about 0.04% of biomedical studies have been retracted, suggesting the problem is small. But the anaesthetist John Carlisle analysed 526 trials submitted to Anaesthesia and found that 73 (14%) had false data, and 43 (8%) he categorised as zombie. When he was able to examine individual patient data in 153 studies, 67 (44%) had untrustworthy data and 40 (26%) were zombie trials.
Bolded text is my emphasis. The problem seems to be institutional: researchers must "publish or perish" and it's the most eye-catching studies that get published because publishers are trying to increase subscription revenue in order to survive. Neither researcher nor publisher are incentivized to not make up data, because there is very little reputational risk involved here - only 0.04% of studies get retracted.
If this analysis is consistent across all medical fields, then it's a coin toss as to whether a random medical study has made-up data.
And notice that this only considers the motivation of individual advancement or profit; there's another whole political motivation that can apply in the cases of, say, the safety and effectiveness of Hydroxychloroquinine. Or Climate Change. In both of these cases, the more you know about the actual science and the more you examine the data, the less trustworthy "consensus" science appears.
But none of this will stop morons who know none of this from sneering that they "trust the science".
5 comments:
This is really no surprise to anyone that actually works with 'peer reviewed' science... sigh
In 2005, John Ioannidis published one of the most downloaded papers in history, "Why most published research findings are false." I became aware of that in '13 and have been pushing it ever since. Now we add in that not only is the math done wrong and the wrong conclusions drawn, the studies simply were never done. It was all made up.
He's quoted in the BMJ piece as well.
If it cannot be replicated, it is wrong. Period.
Um, I figured this out in upper division statistics for poli sci about 40 years ago, when we had a study partner who was rather "ethically challenged".
Data is only as trustworthy as the provider, and if you don't verify it, it's all crap.
And FTR:
"If it's science, it's not consensus. If it's consensus, it's not science." - Michael Crichton (Harvard medical school grad, author, screenwriter, director, producer, who simultaneously had the top-selling book, top-grossing movie, and top-rated TV show, all at the same time), CalTech Michelin Lecture, Aliens Cause Global Warming
When I was young, "Science" was a thing for me, and what scientists said could be trusted.
Most of a human lifetime later I see "science" as a human activity, as subject to delusion, self-delusion, herd behavior, politics, "follow the money" considerations, end stage TDS, etc., as anything else. I've seen the assured pronouncements of science change and change and change, over time.
Almost no one telling us to trust the science is able to actually evaluate the science. It's about who you trust, and why.
Post a Comment