What happened to Evidence-based medicine?

Rationally Speaking | Official Podcast of New York City Skeptics - Current Episodes - RS 174 - John Ioannidis on "What happened to Evidence-based medicine?"

"scientists who have absolutely no conflict — at least financial conflict, we all have our own conflicts where our theories, whatever we have proposed for whatever has made us famous. Consciously or subconsciously, we will try to defend it, so it's very difficult for a scientist, really, to just kill his or her main achievements with new evidence.

Even other entities that may seem to be totally unconflicted and seemingly just wishing the best for research and for outcomes and patients… for example, physicians — we give an oath to try to help human beings, but nevertheless, we live in an environment where we need to make a living. Making a living for a physician nowadays means that they need to get more patients that they see, and more tests that they order, and more drugs that they prescribe, and more hospital care that they offer.

This may not necessarily be towards improving the outcomes, the real, important, patient-relevant outcomes, in these people. Sometimes, specialists, they have to decide: if I get some evidence that shows that what I do and what I make a living from is not useful, that it's not effective and maybe it's even harmful, this will mean I will practically need to lose my job, and maybe retrain in my 50s to do something completely different. Would they do that?...

I fear that, just like alternative medicine and other kinds of pseudoscience have managed to sort of mimic the trappings of good science, by doing RCTs and stuff like that, I worry they'll also learn how to mimic the trappings of revising and changing their mind. But they’ve already written the bottom line (of the argument).I'm ethnically Jewish, so I'm sort of familiar with the way that Talmudic debate seems to encourage open-mindedness, and questioning assumptions and so on — but still, you're never supposed to actually end up concluding God doesn't exist...

Currently, what happens is that we have lots of people who try to be principal investigators. They run their small team and they work behind closed doors. They try to protect their "privacy" of their science and their competitiveness in a way by not sharing information with others... What we get eventually is a fragmented universe of tons of mostly small, underpowered, biased studies. Then you have a systematic reviewer or a meta-analyst who comes forth and says, "Now I'm trying to piece these together, trying to understand what that means. If I get 50 or 100 or 500 small, underpowered, biased studies together, let me see how that looks." Obviously, that doesn't look very nice most of the time. So instead of that paradigm, what I have argued is that we should think more about cumulative agendas of team science, where we are trying to attack interesting questions as a large scientific community... There's many fields that have already done that and have even gone a step further.

If you look at high energy physics and particle physics, this is exactly what's happening. You have 30,000 scientists working at CERN and practically designing their experiments in common and having a common research agenda, and then they can come up with a discovery like the Higgs boson. If we had not done that and we had followed the current paradigm, then what we would have done would be what is happening in current biomedical research, which means you have 30,000 principal investigators. Each one of them has to send in a grant application, get reviewed, get funded. They have to promise that they will find Higgs boson. Within four years actually because if they don't find it within four years how are they going to renew their grant? If you do that, what you end up getting is 30,000 Higgs boson “discoveries,” and none of them will be the real Higgs boson."


Since doctors have financial conflicts of interest, we can't trust anything they say and we should ignore medical advice