“You can’t tell whether I’m lying, delusional, ignorant, or simply incompetent. Sometimes even I don’t know!”
I ran across a 2016 article by Callie Joubert that summarizes skeptical ideas I’ve read about for years, but most people and physicians don’t know about. Bottom line: scientific research and medical studies aren’t nearly as reliable as you think.
Read the whole thing, but here are some excerpts:
We tend to think of science as a dispassionate (impartial, neutral) search for truth and certainty. But is it possible that we are facing a situation in which there is a massive production of wrong information or distortion of information? Is it possible that certain scientific disciplines are facing a crisis of credibility? Mounting evidence suggests this is indeed the case, which raises two questions: How serious is the problem? And what could explain this?
***
The title of an editorial in the prestigious medical journal The Lancet, dated April 6, 2002, asks the question, “Just How Tainted Has Medicine Become?”4 The article states, “Heavily, and damagingly so, is the answer.” Among other things, in 2001, researchers completed experiments with biotechnology products in which they had a direct financial interest and doctors did not tell their patients that others had died using these products when safer alternatives were available. In the same journal, dated April 11, 2015, Dr. Richard Horton stated the gravity of the problem as follows: “The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue . . . science has taken a turn towards darkness.”
In 2004, under the heading of “Depressing Research,” the editor of The Lancet had this to say about antidepressants for children: “The story of research into selective serotonin reuptake inhibitor (SSRI) use in childhood depression is one of confusion, manipulation, and institutional failure. . . . In a global medical culture where evidence-based practice is seen as the gold standard for care, these failings [i.e., of the USA Food and Drug Administration to act on information provided to them about the harmful effects of these drugs on children] are a disaster.”6 After being editor of the New England Journal of Medicine for 20 years, Dr. Marcia Angell stated that “physicians can no longer rely on the medical literature for valid and reliable information.”7 She referred to a study of 74 clinical trials of antidepressants that indicates that 37 of 38 positive studies were published. In contrast, 33 of the 36 negative studies were either not published or published in a form that conveyed a positive outcome. She also mentions the fact that drug companies are financing “most clinical research on the prescription drugs, and there is mounting evidence that they often skew the research they sponsor to make their drugs look better and safer.”
In 2011, researchers at Bayer decided to test 67 recent drug discoveries on preclinical cancer biology research. In more than 75 percent of cases, the published data did not match their attempts to replicate them.8 In 2012, a study published in Nature announced that only 11 percent of the sampled preclinical cancer studies coming out of the academic pipeline were replicable.9
In the prestigious Science journal, in 2015, the Open Science Collaboration10 presented a study of 100 psychological research studies that 270 contributing authors tried to replicate. An astonishing 65 percent failed to show any statistical significance on replication, and many of the remainder showed greatly reduced effect sizes. In plain terms, evidence for original findings is weak.
***
A discovery in physics, the hardest of all hard sciences, is usually thought of as the most reliable in the world of science. However, two of the most vaunted physics results of the past few years—“cosmic inflation and gravitational waves at the BICEP2 experiment in Antarctica, and the supposed discovery of superluminal neutrinos at the Swiss-Italian border—have now been retracted, with far less fanfare than when they were first published.”
***
Parker here again….
The science skeptic best known to physicians is John P.A. Ioannidis:
Empirical evidence from diverse fields suggests that when efforts are made to repeat or reproduce published research, the repeatability and reproducibility is dismal.
Another quote form Ioannidis:
There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.
Ioannidis again:
Most physicians and other healthcare professionals are unaware of the pervasiveness of poor quality clinical evidence that contributes considerably to overuse, underuse, avoidable adverse events, missed opportunities for right care and wasted healthcare resources. The Medical Misinformation Mess comprises four key problems. First, much published medical research is not reliable or is of uncertain reliability, offers no benefit to patients, or is not useful to decision makers. Second, most healthcare professionals are not aware of this problem. Third, they also lack the skills necessary to evaluate the reliability and usefulness of medical evidence. Finally, patients and families frequently lack relevant, accurate medical evidence and skilled guidance at the time of medical decision‐making.
If you like videos, here’s Ioannidis on YouTube.
Staying skeptical,
Steve Parker, M.D.
h/t Vox Day
Advanced Mediterranean Diet 