Why Chatbots Aren’t Enough In Oncology
This article was originally published by Anna Forsythe in Forbes on 13 November 2025. In the fast-moving world of oncology, clinical decision making has never been more complex—or more urgent. Thousands of new cancer studies are published every month, each with findings that could alter treatment pathways or reshape guidelines. For oncologists, research teams, hospitals and payers, the challenge isn’t simply finding information—it’s finding the right information, quickly and confidently. The market is full of AI-powered tools promising help. Many rely on large language models (LLMs) and chatbot-style interfaces that offer answers in conversational form. The appeal is obvious: type in a query, get an instant response. But in oncology—where the stakes are measured in survival rates—ease of use is not enough. Why Decisions Are So Complicated Consider a patient with late-stage lung cancer whose tumor harbors a rare genetic mutation. This is the reality of modern oncology, which offers targeted therapies for specific genetic mutations. The physician must weigh the disease stage, prior therapies, co-morbidities and preferences. They must verify whether a targeted therapy exists, check FDA approvals, review guideline recommendations and explore whether a clinical trial could provide access to the latest investigational drug. This involves combing through journal articles, conference abstracts and regulatory documents—each a piece of the puzzle. There is no “one-size-fits-all” solution in an era where targeted therapies produce individualized pathways. A chatbot might return a single response based on an editorial or opinion piece it “remembers,” presenting it as definitive. The nuance—say, that another trial showed limited efficacy in heavily pre-treated patients, or that guidelines recommend a different approach after immunotherapy—can easily be lost. The Gold Standard: Systematic, Comprehensive, Expert-Vetted Medicine relies on the hierarchy of evidence. At its peak sit systematic reviews and meta-analyses—studies that evaluate and synthesize all available research. Regulatory agencies like the FDA, as well as organizations such as the American Society of Clinical Oncology (ASCO) and the National Comprehensive Cancer Network (NCCN), have long required systematic reviews as the foundation for guidelines and approvals. An effective oncology decision support tool must therefore also be systematic, with transparent, reproducible searches of all relevant research. It must be comprehensive, drawing from peer-reviewed journals, guidelines, conference abstracts and regulatory filings. It must be robust in distinguishing between high-quality randomized trials and weaker evidence. Just as importantly, it must update continuously (ideally daily) to reflect the latest research. Medical decisions based on outdated knowledge risk outdated care. Trained oncologists and other specialists can ensure the conclusions are accurate. Where Chatbots Fall Short I’ve found that even the most advanced LLMs cannot meet those criteria. Their weaknesses are structural. Built for speed and limited in transparency, chatbots rarely disclose their sources. They may omit references entirely, and without systematic searching, key studies are often missed. Their datasets often exclude recent guideline updates or pivotal conference results. Moreover, as black boxes reliant on opaque algorithms, chatbots provide no evidence grading. An editorial can appear with the same weight as a phase three trial. They may even fabricate references—so-called and largely reported on “hallucinations.” In my experiments, queries have sometimes led to outdated and false information. In one instance, a chatbot cited a non-existent study to me. Transparency of the dataset is critical, especially in a field where thousands of new studies are published each month. Using AI on an iPhone to call a taxi is convenient, but in oncology, where each decision can alter survival, these shortcomings aren’t just inconvenient; for a patient with a rare mutation, it can mean the difference between hope and harm. Beyond Oncology: A Universal Lesson The risk of relying on incomplete or unverified evidence isn’t unique to cancer care. In finance, successful portfolio managers don’t bet other people’s money on one analyst’s hunch; they use meta-analyses of market data. In aviation, flight safety depends on synthesizing thousands of reports and assessments. No pilot would fly based on a chatbot’s opinion about turbulence. In public health, vaccine rollouts depend on systematic reviews of global trial data, not a handful of preliminary studies. Across industries, convenience cannot replace rigor. The ideal system in oncology—and other data-driven fields—is an expert-driven partner that can provide trustworthy insights. The Human and AI Solution Despite certain limitations in its use within chatbots, the beauty of AI is how it can scan millions of documents in seconds, helping detect patterns and surface relevant studies. With the mountain of data produced every day, that feature is undeniably important. But human experts are needed to bring judgment, clinical context and critical thinking to the mix. I think the winning model is a living systematic literature review (SLR)—continuously updated by AI, structured through reproducible methodology and validated by experts. (Disclosure: I lead an AI-assisted oncology evidence platform this type of approach.) LLMs power today’s chatbots—but they can also hallucinate or misread complex evidence. The approach I champion still uses LLMs, but with continuous expert oversight. Every data point should be verified by trained analysts and clinicians, eliminating hallucinations and ensuring full transparency. That said, I find this hybrid model effective but demanding. It requires capital, expertise and time to build for each cancer type. And even then, people still prefer someone or something they can talk to. The future may lie in combining both approaches—a conversational chatbot connected to a rigorously curated, expert-verified database. But by working to overcome these hurdles, pharmaceutical companies, payers and healthcare networks stand to benefit as much as clinicians. Beyond oncology, systematic, AI-augmented evidence synthesis has the potential to streamline internal decision making, support value-based care initiatives, strengthen negotiations and reduce duplication across research teams. The Bottom Line AI is here to stay, and its potential in healthcare is enormous. But in oncology—and in every field where lives or livelihoods are at stake—it must be deployed with discipline. Chatbots may offer instant, conversational answers, but approachability is not the same as reliability. Anna Forsythe Anna Forsythe, pharmacist & health economist, is the Founder & CEO of Oncoscope-AI
Systematic Literature Review Versus Chatbots: Why In Oncology, It’s Not a Choice
In the age of artificial intelligence, speed is often mistaken for rigor. Nowhere is this more dangerous than in oncology, where treatment decisions can mean the difference between life and death. Some technology companies tout “systematic literature reviews” (SLRs) generated in minutes by chatbots that claim to scan thousands of papers across the internet. The appeal is obvious: quick, accessible, and seemingly comprehensive. But in reality, these outputs are neither systematic nor reliable. For oncologists, payers, and researchers, understanding the distinction between a true SLR and a chatbot’s surface-level search is not just academic—it’s essential. The Gold Standard: What a True SLR Involves Systematic literature review is the gold standard for evidence synthesis in medicine. It is the foundation of evidence-based practice because it minimizes bias, ensures completeness, and enables decisions to rest on the strongest available science. A rigorous SLR begins with a protocol: a predefined roadmap that frames the research question and methods. It requires carefully constructed search strategies, typically using combinations of keywords and controlled vocabulary, to capture every relevant publication across peer-reviewed databases. The process doesn’t stop there. Grey literature—such as abstracts from scientific conferences—must also be included, since cutting-edge oncology data often appears in congress presentations long before it reaches a journal. From there, studies undergo multi-step screening against strict inclusion and exclusion criteria: patient population, interventions, comparators, outcomes, and study design (the classic PICO framework). Each selected paper is then critically appraised for quality and relevance. Only after this painstaking filtering does the work of synthesis and interpretation begin. This is not a clerical exercise. It requires advanced training, sound judgment, and clinical insight to evaluate conflicting results, contextualize findings, and translate them into actionable conclusions. Why Chatbots Fall Short Chatbots, even those powered by large language models (LLMs), cannot replicate this process. At best, they skim unstructured text. At worst, they hallucinate citations or omit critical studies. They lack protocols, inclusion criteria, appraisal of study quality, or a transparent audit trail. What results may look convincing on the surface—but lacks the depth and reliability required in oncology. When a chatbot says it can “review 1,000 studies in seconds,” what it’s really doing is producing a text summary based on whatever sources it happens to ingest. There is no guarantee that the sources are peer-reviewed, complete, current, or even real. That is not an SLR. Why It Matters in Oncology Oncology is not forgiving of shortcuts. Selecting the right therapy for a patient is an exercise in precision: choosing between regimens, sequencing targeted therapies, balancing efficacy and toxicity, and staying current on breakthroughs that can extend survival or improve quality of life. In this context, incomplete, outdated, or fabricated evidence isn’t a minor flaw—it’s a threat to patient safety. The rigor of a systematic literature review is not a “nice to have”; it’s the foundation for making responsible decisions in cancer care. The Path Forward AI absolutely has a role to play in evidence synthesis. When paired with human expertise and transparent methodology, it can accelerate searches, streamline screening, and reduce administrative burden. But AI must serve the process—not replace it. In oncology, the choice isn’t between a chatbot and a systematic literature review. It’s between cutting corners and saving lives. The stakes are too high for anything less than living, rigorous, and human-guided evidence. Anna Forsythe Anna Forsythe is the Founder and President of Oncoscope-AI, the first platform to bring together real-time oncology treatment data, clinical guidelines, research publications, and regulatory approvals — all in one place, just like Expedia for cancer care. Available free to oncology professionals worldwide, Oncoscope-AI is redefining how cancer care information is accessed and applied.
From Cochrane To Chatbots: Why Evidence Matters Now More Than Ever In The AI Era
This article was originally published in Forbes on 02 September 2025. In oncology, discovery is moving at warp speed. Hundreds of new studies are published each week, sometimes more than a dozen in a single day. For patients and providers, this fire hose of information could equate to life-saving breakthroughs—a new biomarker, a novel dosing schedule or a survival-prolonging treatment. But it also equates to risk. Misinterpret a flawed trial, overtrust anecdotal experience or trust the wrong person, and patients are left at the mercy of ineffective or even harmful treatments. And so, as this deluge of data overwhelms us with such velocity, the imperative is ensuring that decisions are well-informed by precise, whole and transparent evidence. That all starts with one man: Archibald Leman Cochrane. Archibald Cochrane: The Original Evidence Disruptor Archibald Cochrane was a British physician and epidemiologist who served as a prisoner-of-war doctor in World War II. With effectively no medicines at his disposal, he watched patients suffer because care was based more on habit than proof. After the war, he became an outspoken advocate for a revolutionary principle: Medicine must be based on evidence that has been tested and proven, not on practice or expert opinion. In 1972, he published Effectiveness and Efficiency: Random Reflections on Health Services, a text that shook the medical establishment through criticism of its reliance on anecdotes. He argued that randomized controlled trials (RCTs) were required to determine whether treatments were effective and called for doctors and patients to be presented with objective summaries of all pertinent evidence. His vision inspired the Cochrane Collaboration, founded in 1993, which remains the global leader in producing systematic reviews. Pre-Cochrane, the concept of a controlled clinical trial did not exist. He was the one who established the use of controlled clinical trials, which have now become the gold standard for evaluating new treatments. Building On Solid Ground Cochrane’s work gave us the pyramid of evidence, often used to illustrate the hierarchy of reliability. The higher you climb in the pyramid, the stronger the foundation for life-and-death decisions. Why Systematic Reviews Save Lives In oncology, where treatment options evolve daily, systematic reviews are essential. A single study, whether positive or negative, rarely tells the full story. Systematic reviews, however, consider the entire body of evidence and account for consistency, quality and nuance. Nearly every modern guideline (from the World Health Organization to the FDA) requires systematic reviews as the foundation of clinical recommendations. These methods are necessary because lives depend on them. When Chatbots Pretend To Be Experts We’ve now entered the AI era, where large language models and chatbots can generate fluent answers to intricate medical questions in seconds. To overwhelmed oncologists confronting a deluge of literature, this might look like salvation. But these tools don’t follow the hierarchy of evidence. They don’t methodically scan all the studies, grade the quality of trials or reveal how they arrived at their conclusions. They generate responses based on text patterns that are sometimes accurate, sometimes incomplete and sometimes entirely fabricated. I’ve heard an example just last week from one of my medical colleagues: An AI tool produced a reference under his name for an article he had never written. It was a perfect illustration of how these systems behave like an overeager student desperate to provide an answer, even if that means inventing one. A chatbot’s “best guess” could mean proposing a treatment that introduces unnecessary toxicity to patients or worse, missing a well-documented new trial that would improve survival. The irony is that most of us would never hand over the task of booking a flight to a chatbot without carefully double-checking everything—the departure point, the destination city, the time of day. Yet somehow, when it comes to medical treatment, many are willing to accept incomplete or outdated chatbot outputs at face value. That disconnect should give us pause. Fluency Isn’t Truth One of the greatest risks of AI in medicine is that it sounds authoritative. Chatbots excel at fluency, but fluency is not the same as truth. At this time, AI simply isn’t ready for the weight we’re placing on it. Too many people are so captivated by what it can do that they forget the most basic principle of science and medicine. You must double-check the information. That fascination is dangerous. It encourages blind trust in answers that may be incomplete, misleading or outright wrong. If we allow fluency to masquerade as reliability, medicine risks sliding backward to the pre-Cochrane era, when anecdotes and authority carried more weight than solid data. Smarter Together I’m not saying that AI has no place in evidence-based medicine. Far from it. Let machines handle the tedious but essential work of scanning thousands of papers, formatting endless references and keeping reviews continuously updated as new trials are published. These are the repetitive housekeeping tasks that often slow researchers down, yet must be done with precision. I often compare it to household chores. AI should be doing the vacuuming so humans can spend their time on meaningful work. That’s exactly how my team approaches it. We’ve built a model that uses 36 different AI systems, integrated under the supervision of PhD-level experts, to make systematic reviews dramatically faster without compromising accuracy. Don’t outsource judgment to machines. Give human experts more bandwidth to do what only they can do: Interpret the evidence, weigh the nuances and make the right decisions for patients. Evidence Still Rules Cochrane warned half a century ago that a great deal of medicine lacked solid evidence. His admonition is even more urgent today. With AI, the danger is overconfidence in fluent machines that sound convincing but aren’t built on rigorous evidence. AI must serve evidence, not replace it. Healthcare leaders, policymakers and clinicians need to insist on transparency, rigor and comprehensiveness as absolute necessities. Because in oncology, and medicine in general, lives are at stake. And regardless of how much the tech advances, one thing will always be true: Evidence still rules. This article is