Tom Scott shouldn’t feel too bad about his findings as false finding are the stock and staple of the entire domain of science. In fact, the post “Why all research findings are false” by the “Devil’s Neuroscientist explores this in depth:
- Science never proves anything. You may read in the popular media about how scientists ‘discovered’ this or that, how they’ve shown certain things, or how certain things we believe turn out to be untrue. But this is just common parlance for describing what scientists actually do: they formulate hypotheses, try to test them by experiments, interpret their observations, and use them to come up with better hypotheses. Actually, and quite relevant to the discussion about preregistration, this process frequently doesn’t start with the formulation of hypotheses but with making chance observations. So a more succinct description of a scientist’s work is this: we observe the world and try to explain it.”
- Science is always wrong – This analogy highlights why the fear of incorrect conclusions and false positives that has germinated in recent scientific discourse is irrational and misguided. I may have many crises but reproducibility isn’t one of them. Science is always wrong. It is doomed to always chase a deeper truth without any hope of ever reaching it. This may sound bleak but it truly isn’t. Being wrong is inherent to the process. This is what makes science exciting.”
His analysis provides a very thorough illustration of not just the tenet that “Correlation is not Causality”, but the general limitations of using statistical analysis (“Just like science at large, statistics never prove anything, except perhaps for the rare situations where something is either impossible or certain – which typically renders statistical tests redundant.”).
Sadly, in our current times when “science” has never been so vital to our welfare, a surging number of “science deniers” are turning their backs on this discipline. Typically, spurred by charlatans who’s mischief is refuted or exposed by science, one of the most prominent rhetorical trick these huckster use is to point to science’s own admission of its fallibility. A specific failing identified in science does not make all science a failure (throwing the baby out with the bath water). The logical fallacy they are plying is “false equivalence”. Science is embrace of incompleteness and inaccuracy might surface many specific failings, but the overall effect is to make science stronger not weaker.