We hope that science progresses like this:
- Someone does a study and proposes a theory
- Other scientists read the study and think about it
- Other labs try to repeat the experiment and get the same results
- Other researchers come up with alternative theories and test those
- A consensus in the community is reached
- The theory becomes practice
For many reasons, though, medical research often skips steps 2-5. When you’re looking for a cure, when your patient is asking questions, yesterday is not soon enough for an answer. As soon as a promising study suggests a treatment, doctors and their patients want to try it.
And when science is silent, patients and parents take matters into their own hands. The October issue of Scientific American ran a fascinating and sad article Desperate for an Autism Cure about parents of autistic children experimenting with their own therapies.
But there’s no need to turn to unsubstantiated claims for bad advice. Medical journals publish studies every day suggesting treatments, cures, therapies, and drugs that may or may not be effective. The short loop of study leading directly to treatment, or at least to medical advice, makes the problem of bad science much worse in medical research.
Suppose that the study is wrong? Suppose the researchers were biased by the drug company that funds their work? Suppose their findings were published because the journal wanted something interesting, something new, something heretical on their front page?
Dr. John Ioannidis does suppose these things, and he proves them, too. There’s a great article about him and his team in the November 2010 Atlantic. Consider the excerpt below:
He and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies—conclusions that doctors keep in mind when they prescribe antibiotics or blood-pressure medication, or when they advise us to consume more fiber or less meat, or when they recommend surgery for heart disease or back pain—is misleading, exaggerated, and often flat-out wrong. He charges that as much as 90 percent of the published medical information that doctors rely on is flawed.
Despite the depressing topic, this article is well-written, engaging, and definitely worth your while. It examines the different ways that bias can enter medical research and explains why the so-called safety nets designed to catch errors, such as peer-review, scientific honesty, and researcher integrity, are failing.
While reading this article I was struck in particular by one quote from Athina Tatsioni, one member of Ioannidis’ team:
“Usually what happens is that the doctor will ask for a suite of biochemical tests—liver fat, pancreas function, and so on,” she tells me. “The tests could turn up something, but they’re probably irrelevant. Just having a good talk with the patient and getting a close history is much more likely to tell me what’s wrong.”
This put me in mind of another article I’d recently read about a Stanford physician seeking to re-energize the practice of giving a physical exam. The New York Times Health section profiled Dr. Abraham Verghese, his philosophy, and his teaching methods.
Apparently, “he likes to joke that a person could show up at the hospital with a finger missing, and doctors would insist on an M.R.I., a CT scan and an orthopedic consult to confirm it.” Ha ha, quite clever!
Something tells me that Ioannidis and Verghese would get along. It seems as though Verghese’s approach is a good way to mitigate some of the problems that Ioannidis’ work has uncovered. Good bedside manner, however, can only be helped by some good science, which it seems is in short supply.
Ioannidis reminds us that science is supposed to be filled with errors and spurious theories and confusing results. That’s the way it works. Truth is a messy undertaking. Both doctors and patients need to be cautious and allow the machinery of science time to work. And recognize that sometimes, we just don’t know the answer yet.