From Cochrane To Chatbots: Why Evidence Matters Now More Than Ever In The AI Era
This article was originally published in Forbes on 02 September 2025. In oncology, discovery is moving at warp speed. Hundreds of new studies are published each week, sometimes more than a dozen in a single day. For patients and providers, this fire hose of information could equate to life-saving breakthroughs—a new biomarker, a novel dosing schedule or a survival-prolonging treatment. But it also equates to risk. Misinterpret a flawed trial, overtrust anecdotal experience or trust the wrong person, and patients are left at the mercy of ineffective or even harmful treatments. And so, as this deluge of data overwhelms us with such velocity, the imperative is ensuring that decisions are well-informed by precise, whole and transparent evidence. That all starts with one man: Archibald Leman Cochrane. Archibald Cochrane: The Original Evidence Disruptor Archibald Cochrane was a British physician and epidemiologist who served as a prisoner-of-war doctor in World War II. With effectively no medicines at his disposal, he watched patients suffer because care was based more on habit than proof. After the war, he became an outspoken advocate for a revolutionary principle: Medicine must be based on evidence that has been tested and proven, not on practice or expert opinion. In 1972, he published Effectiveness and Efficiency: Random Reflections on Health Services, a text that shook the medical establishment through criticism of its reliance on anecdotes. He argued that randomized controlled trials (RCTs) were required to determine whether treatments were effective and called for doctors and patients to be presented with objective summaries of all pertinent evidence. His vision inspired the Cochrane Collaboration, founded in 1993, which remains the global leader in producing systematic reviews. Pre-Cochrane, the concept of a controlled clinical trial did not exist. He was the one who established the use of controlled clinical trials, which have now become the gold standard for evaluating new treatments. Building On Solid Ground Cochrane’s work gave us the pyramid of evidence, often used to illustrate the hierarchy of reliability. The higher you climb in the pyramid, the stronger the foundation for life-and-death decisions. Why Systematic Reviews Save Lives In oncology, where treatment options evolve daily, systematic reviews are essential. A single study, whether positive or negative, rarely tells the full story. Systematic reviews, however, consider the entire body of evidence and account for consistency, quality and nuance. Nearly every modern guideline (from the World Health Organization to the FDA) requires systematic reviews as the foundation of clinical recommendations. These methods are necessary because lives depend on them. When Chatbots Pretend To Be Experts We’ve now entered the AI era, where large language models and chatbots can generate fluent answers to intricate medical questions in seconds. To overwhelmed oncologists confronting a deluge of literature, this might look like salvation. But these tools don’t follow the hierarchy of evidence. They don’t methodically scan all the studies, grade the quality of trials or reveal how they arrived at their conclusions. They generate responses based on text patterns that are sometimes accurate, sometimes incomplete and sometimes entirely fabricated. I’ve heard an example just last week from one of my medical colleagues: An AI tool produced a reference under his name for an article he had never written. It was a perfect illustration of how these systems behave like an overeager student desperate to provide an answer, even if that means inventing one. A chatbot’s “best guess” could mean proposing a treatment that introduces unnecessary toxicity to patients or worse, missing a well-documented new trial that would improve survival. The irony is that most of us would never hand over the task of booking a flight to a chatbot without carefully double-checking everything—the departure point, the destination city, the time of day. Yet somehow, when it comes to medical treatment, many are willing to accept incomplete or outdated chatbot outputs at face value. That disconnect should give us pause. Fluency Isn’t Truth One of the greatest risks of AI in medicine is that it sounds authoritative. Chatbots excel at fluency, but fluency is not the same as truth. At this time, AI simply isn’t ready for the weight we’re placing on it. Too many people are so captivated by what it can do that they forget the most basic principle of science and medicine. You must double-check the information. That fascination is dangerous. It encourages blind trust in answers that may be incomplete, misleading or outright wrong. If we allow fluency to masquerade as reliability, medicine risks sliding backward to the pre-Cochrane era, when anecdotes and authority carried more weight than solid data. Smarter Together I’m not saying that AI has no place in evidence-based medicine. Far from it. Let machines handle the tedious but essential work of scanning thousands of papers, formatting endless references and keeping reviews continuously updated as new trials are published. These are the repetitive housekeeping tasks that often slow researchers down, yet must be done with precision. I often compare it to household chores. AI should be doing the vacuuming so humans can spend their time on meaningful work. That’s exactly how my team approaches it. We’ve built a model that uses 36 different AI systems, integrated under the supervision of PhD-level experts, to make systematic reviews dramatically faster without compromising accuracy. Don’t outsource judgment to machines. Give human experts more bandwidth to do what only they can do: Interpret the evidence, weigh the nuances and make the right decisions for patients. Evidence Still Rules Cochrane warned half a century ago that a great deal of medicine lacked solid evidence. His admonition is even more urgent today. With AI, the danger is overconfidence in fluent machines that sound convincing but aren’t built on rigorous evidence. AI must serve evidence, not replace it. Healthcare leaders, policymakers and clinicians need to insist on transparency, rigor and comprehensiveness as absolute necessities. Because in oncology, and medicine in general, lives are at stake. And regardless of how much the tech advances, one thing will always be true: Evidence still rules. This article is
From Cochrane to Chatbots: Have We Forgotten the Evidence?
If medicine had a godfather of evidence, his name would be Archie. Archibald Leman Cochrane, that is. Cochrane wasn’t just a physician—he was a rebel against tradition. In the 1970s, he looked around at the medical world and said, essentially: “Too much of what we do is based on habit and authority, not actual proof.” He proposed a radical idea: let’s rank evidence by reliability. At the bottom sat case reports and expert opinions—interesting, but hardly solid. Above that came observational studies, then randomized controlled trials (RCTs). And sitting proudly at the top of the pyramid? Systematic literature reviews (SLRs): structured evaluations that capture all the studies, critique their quality, and synthesize their findings. That hierarchy became the foundation of what we now call Evidence-Based Medicine (EBM). Why Systematic Reviews Matter Systematic reviews aren’t academic busywork. They’re the reason guidelines from the World Health Organization, NICE, and the American Society of Clinical Oncology (ASCO) are trustworthy. They’re why the FDA demands comprehensive, systematic evidence before approving a therapy. The principle is simple: no single trial tells the full story, but when you put the whole picture together—carefully, transparently, reproducibly—you can make decisions that change lives. But Have We Forgotten Cochrane? Fast forward to today. AI chatbots can answer clinical questions in seconds. They sound authoritative, but they don’t follow Cochrane’s hierarchy. They don’t systematically review literature. They don’t grade evidence for quality. And they definitely don’t show their work. In oncology, where new studies are published daily, this is not a minor issue. A chatbot that casually cites one study—or worse, invents a citation—could mislead a physician into recommending something harmful, or missing a life-saving option. It’s like we built the evidence pyramid over decades, and now, dazzled by shiny AI, we’re forgetting why we built it in the first place. The Way Forward The future of evidence isn’t abandoning systematic reviews. It’s making them living: constantly updated, rigorous, and transparent. AI does have a role to play here—not as a chatbot dispensing unverified answers, but as a tool that accelerates and augments systematic reviews. That’s exactly what we’re building at Oncoscope: living SLRs, human-vetted and PhD-curated, augmented by AI. Reliable, current, and ready to support the most important decisions in oncology. Because in cancer care, evidence isn’t an academic debate. It’s a matter of life, harm, or hope. So the question is: are we going to let chatbots distract us from Cochrane’s lesson—or use AI to fulfill it? Anna Forsythe is the Founder and President of Oncoscope-AI, the first platform to bring together real-time oncology treatment data, clinical guidelines, research publications, and regulatory approvals — all in one place, just like Expedia for cancer care. Available free to oncology professionals worldwide, Oncoscope-AI is redefining how cancer care information is accessed and applied. A clinically trained Doctor of Pharmacy (PharmD), Anna also holds a Master’s in Health Economics and Policy from the University of Birmingham (UK) and an MBA from Columbia University. She previously co-founded Purple Squirrel Economics (acquired by Cytel in 2020) and led Global Value and Access at Eisai Pharmaceuticals, following earlier roles at Novartis and Bayer in clinical research and health economics.