Wednesday, October 18, 2017

There's a reason that so much science is crap

Peter calls out attention to tens of thousands of scientific papers that are possibly (likely? who knows?) invalid.  This is not a surprise to anyone who is paying attention.  It comes from two very simple principles:

1. Success in a scientific career is determined by how frequently the scientist publishes papers in scientific journals.

2. Scientific journals are interested in novelty - new results that have not been published before.  It's pretty rare to see a paper published confirming the results of a prior paper.

In short, there's absolutely no need for results confirming other papers - these will never get published anyway.  Since there's no need for confirmation, a scientist can focus all his efforts on novelty.  Because novel results sometimes must skirt the edge of what is seemly, we can expect spillover into the unseemly.

So how much spillover do we get?  We don't know.  And quite frankly, neither does anyone who is engaged in "Science".  We've known for a long time that something is very wrong in the state of Science™, with the rate of major advances slowing noticeably.  It could very well be that the reason is that there is so much bogus science being done, that distracts young scientists from other more significant areas of study.  It may be that with no need to get reproducibility, it's just easier to put out novel garbage than it is to do more significant work.  And when you think about which of those two patch will be better for a young scientist's career, it's no surprise that we swim is a sea of scientific crap.

None of this has to be deliberate, or fraudulent.  It's just the way that the scientific game is played.

Science that is repeatable is called "Engineering" anyway.  Strange how you don't seem to hear about lots of retractions from engineering journals.  Science is like the stereotypical aging starlet still trying to get drinks in bars off of her old good looks and fame.


SiGraybeard said...

My personal all time favorite quote comes from a guy that was working to replicate experiments for possible cancer drugs (IIRC).

"I explained that we re-did their experiment 50 times and never got their result. He said they'd done it six times and got this result once, but put it in the paper because it made the best story. It's very disillusioning."

Best story? Best freaking story!!!??????!!!! Arrrrggggghhhhhhhh Science isn't supposed to be stories!!! One out of six? Doesn't that mean you screwed up that one time but that's the one you published?!?


drjim said...

+ 1 bazillion on SiG!

Ted said...

Also you can’t fake Enigineering and make the results fit the premis. Things tend to fall over or go bang when you do.
Where as in the “Science” of Global Warming it’s all about what might happen sometime in the Future ..... probably.

Windy Wilson said...

Global Warming today is all about fitting the facts to the narrative, a la “the Usual Suspects”. It means that today science becomes a storyteller’s contest, where delivery and vivid metaphor are everything. It is the gripe I had about Keynesian economics when I was in college in the seventies. Graphs of relationships without numbers or scale. Everything depended on telling a good story, not replicable experiments.