How Could AI Change Medicine?
Introduction: The Question That Really Matters
Units 1 and 2 were largely descriptive. Here is what AI can do. Here is how it works. These are important foundations, but understanding AI's capabilities is only the first step; the next challenge is deciding how to use it responsibly.
AI tools introduce new choices for patient care, workflow design, and governance.
This unit examines those choices by looking closer at both the opportunities AI creates and the risks it brings. The aim is neither to celebrate AI uncritically nor to assume the worst. The future of AI in Healthcare is not predetermined. It will be shaped by the decisions people and organisations make.
Responsible choices begin with a clear understanding of what is at stake.
We'll cover four areas:
- The genuine benefits advanced AI could bring to medicine
- The clinical risks: bias, deskilling, and misdiagnosis at scale
- The accountability gap: what happens when AI is involved and something goes wrong
- The systemic risks: power, data, and what medicine might lose
This unit tries to neither celebrate AI uncritically nor dismiss it. If you find yourself more persuaded by one side, that's worth sitting with. You will form your own view, this unit just tries to help make it an informed one.
The Genuine Benefits: What AI Could Make Possible
Some of what AI could do for medicine is genuinely remarkable; and matters most to the people for whom current healthcare is failing.
Catching diseases earlier
Many of the conditions that cause the most suffering are far more treatable when caught early. Ovarian cancer, detected at stage one, can be cured in up to 90% of patients. By stage four, long term survival falls to ~20% (NLoM). Parkinson's disease tells an equally urgent story: by the time the tremor appears and diagnosis typically follows, up to 80% of the brain's dopamine-producing neurons have already been destroyed (NIDS). Chronic kidney disease progresses silently for years; caught early, its progression can be substantially slowed. Caught late, the outcome is dialysis or transplant (NLoM).
AI systems that could detect the earliest signals of these diseases from imaging, blood biomarkers or subtle patterns in a patient's electronic health record could shift millions of diagnoses from late to early. That is not a marginal improvement in outcomes. For the individuals involved, that could be life-changing.
Randomised and prospective trials in Sweden, Germany and the US are already showing early promise in this direction. The challenge is proving benefit, then deploying equitably.
Extending specialist care beyond specialist centres
A patient in a rural area of sub-Saharan Africa, a remote region of Australia, or a small town far from a major hospital in any country, currently has fundamentally different access to specialist care than someone in a major city. That gap is not a given; it is a consequence of how healthcare has historically been organised around human experts who cannot be in more than one place at once.
AI changes that equation.
A 2024 Nature Medicine trial involving 1,196 pregnant women across six hospitals in Nigeria (a country with one of the highest reported rates of peripartum cardiomyopathy) found that AI-based ECG screening accurately detected this serious heart condition that may have otherwise gone undetected, in settings where specialist cardiology is scarce.
This is one of the most compelling arguments for AI in healthcare. Not that it will improve care in the best-resourced systems, but that it has the potential to radically improve care in the least-resourced ones, where the need is greatest.
But that outcome is not automatic. The risk is that advanced AI, like many powerful technologies before it, is developed primarily for and by the well-resourced, improving care for those already well served while leaving existing inequities untouched. Realising the more equitable outcome requires deliberate decisions from developers, funders, regulators, and health systems.
Reducing the burden on clinicians
Burnout among healthcare professionals affects healthcare systems worldwide. Physicians in the US, Europe, Asia, and beyond are leaving the profession in significant numbers. This is driven not by a loss of commitment to patients, but by the weight of administrative demands, the pace of modern healthcare, and the emotional toll of working in under-resourced systems.
As mentioned in Unit 1, AI that genuinely reduces administrative burden gives clinicians more time & head space to do what they trained to do: care for people. This is not a trivial benefit. A clinician who is less overwhelmed is likely to make better decisions and give better care.
Accelerating the science of medicine itself
Drug discovery, vaccine development, understanding the genetic basis of complex diseases. These are areas where AI progress could have implications that extend well beyond healthcare as currently practised.
AlphaFold, developed by Google DeepMind, largely solved a problem that had challenged biology for fifty years: predicting how proteins fold into their three-dimensional shapes. This matters because most drugs work by binding to proteins, and understanding protein structure is foundational to drug design. AlphaFold's predictions are now freely available to researchers worldwide helping to accelerate research across thousands of projects and programmes. The work was awarded the Nobel Prize in Chemistry in 2024.
Of the benefits described here - earlier diagnosis, extending access, reducing clinician burnout, accelerating science - which feels most significant to you? Is that because of your professional role, your personal experience of healthcare, or something else?
The Clinical Risks: Bias, Deskilling, and Harm at Scale
While advanced AI may hold immense promise for medicine, its real and documented risks require clear evaluation to ensure safe and responsible use.
Algorithmic bias and health inequalities
As we learnt in Unit 2, AI systems built with machine learning learn from historical data. Healthcare data like most historical data reflects the inequities of the systems that produced it. If certain populations have historically received worse care, or different treatment, those patterns are baked into the datasets AI learns from. Bias can affect each stage of the tool development, as summarised nicely below from a study in the British Journal of Radiology.
The risks of algorithmic bias in healthcare are not theoretical, they have already caused documented harm at scale. A widely cited 2019 study published in Science provides lessons from the past. It examined an algorithm used by US health insurers to allocate additional care management resources. The algorithm used healthcare costs as a proxy for health need. Because Black patients had historically spent less on healthcare not because they were healthier, but because they had less access. The algorithm systematically underestimated their health needs. Roughly the same level of illness in a Black patient was scored as lower need than in a White patient, resulting in fewer resources being allocated. This was not a fringe system. By industry estimates these algorithms are applied to approximately 200 million patients in the United States annually.
As AI becomes more embedded in healthcare, the risk is not only that an individual AI tool affecting a single patient could be biased. Widespread adoption of the same technology risks spreading uniform errors or biases across whole healthcare systems; making robust, continuous monitoring a necessity.
The Deskilling of Healthcare Professionals & Automation Bias
When a sophisticated tool does something better than an unaided human, humans stop practising that skill unaided. Aviation has documented this problem for decades. Studies of Airline pilots found that while basic aircraft control skills remained largely intact with heavy autopilot use, the cognitive skills needed for manual flight degraded meaningfully. The US Federal Aviation Administration considered the problem serious enough to issue a formal industry safety alert in 2017 (FAA). The concern was not that autopilot was unsafe. It was that pilots who had spent years relying on it might not have the underlying skills to take over when it failed.
Medicine could now be facing the same issue. A 2025 multicentre study in The Lancet Gastroenterology & Hepatology found that experienced endoscopists who had regularly used AI to assist polyp detection showed a decrease in performance during colonoscopy when the AI was taken away. Their adenoma detection rate fell from 28.4% before AI exposure to 22.4% once the AI was removed, a meaningful drop in how often they caught precancerous growths unaided. The findings suggest that routine reliance on AI support may influence unaided clinical proficiency among experienced practitioners.
The second risk is subtler, yet equally serious: automation bias. Even when clinicians know they should scrutinise AI outputs, evidence suggests they often do not.
A 2025 randomised controlled trial highlighted this vulnerability: physicians who had completed structured AI-literacy training still achieved only 73% diagnostic accuracy when an AI tool provided deliberately flawed suggestions, compared to 85% when the advice was accurate.
What makes this striking is that the physicians were completely free to ignore the AI and had been specifically trained to evaluate its outputs, yet the bias persisted. While the trial was small and remains suggestive rather than definitive, it signals a significant risk that healthcare systems cannot afford to ignore.
These two risks compound each other in an uncomfortable way. Deskilling would reduce the underlying expertise needed to catch AI errors. Automation bias would mean clinicians are less inclined to look for them. A clinician affected by both is in a poor position to act as the safety-net these systems depend on.
Do not use general-purpose AI for clinical decisions. Rely only on approved sources. You remain personally accountable for all clinical decisions made.
The erosion of the clinician-patient relationship
Medicine has always been a human relationship. There is a risk that as AI takes on more of the analytical work of medicine, what may remain for clinicians is a more procedural role. Approving AI outputs, implementing AI recommendations - becoming the Air Traffic Controller not the Pilot. If clinical work is reshaped this way the relational and interpretive dimensions of clinical care decrease.
For patients, the risk is a kind of care that is more precise analytically but poorer in human terms; correct in its technical outputs and absent in its attention to the person those outputs are about.
The risks described here: bias, harm at scale, deskilling, are structural problems with how AI is built and deployed. Does that change how you think about who is responsible when things go wrong? And who should be doing something about it?
The Accountability Gap: When Something Goes Wrong, Who Is Responsible?
When a healthcare worker makes a mistake, the lines of accountability are established and understood. The clinician is responsible for their clinical decisions. The hospital or institution may carry organisational responsibility. There are professional standards, legal frameworks, and mechanisms for investigation and remedy.
When an AI system is involved in harm, these lines become less certain. While existing legal frameworks provide guidance, there is no established UK case law on how liability should be assigned when AI contributes to patient harm.
Who Bears the Risk?
Consider a realistic scenario. A hospital deploys an AI system to assist in reading pathology slides. The system is purchased from a commercial vendor. It was trained on data from a research institution in another country. It was validated in a clinical trial before deployment. A pathologist reviews its outputs before finalising reports.
The system classifies a tumour incorrectly. A patient receives the wrong treatment. By the time the error is discovered, months have passed.
Who is responsible? The pathologist who approved the output? The hospital that deployed the system? The vendor who built it? The research institution whose data trained it? The regulator who approved it?
Although there may be no clear answer, in the UK today and most other current legal and regulatory frameworks, the pathologist and the trust would likely be the first line of accountability. Treating the clinician as the 'learned intermediary' responsible for final oversight, acting as the final safety net.
The problem is that this places responsibility on clinicians & hospital trusts for outputs they may have limited ability to evaluate. This concern becomes more significant when AI systems move beyond simply providing information and begin generating diagnoses, treatment recommendations, or other outputs that clinicians are expected to rely upon.
Holding clinicians accountable for AI errors they could not reasonably have detected appears neither fair nor effective at preventing future harm. It pushes the duty towards the person less able to fix the cause.
The ability to detect and correct such errors depends on clinicians being able to meaningfully scrutinise AI recommendations, a task that becomes more difficult when how a system produces a recommendation is opaque.
For now, in the UK, the professional position is clear: the clinician remains accountable for the decisions they make with AI support, which is exactly why the argument in this section matters.
Explain-ability and the black box
Many AI systems, particularly deep learning systems cannot easily explain how an output was produced. Deep learning systems are commonly used in medical imaging. A radiologist can usually explain why a particular shadow on an X-ray raised concern, whereas an AI imaging system may simply produce a probability score indicating the likelihood of concern.
This creates a challenge for both clinical decision-making and accountability. If a clinician is expected to use their judgement to review an AI output, they need enough information about how that output was generated to meaningfully scrutinise it. And if something goes wrong, investigators should be able to understand what the AI system did and where possible, why.
The field of explainable AI (XAI) is working on this problem, developing methods to explain what an AI system produced and what drove that output. However, the most powerful AI systems tend to be the least explainable & the companies building them are generally becoming less transparent over time. The 2025 Stanford Foundation Model Transparency Index found that average transparency scores among leading AI developers fell from 58 to 41 out of 100 in that year. With companies most secretive about the training data their models were trained on: potentially the thing that matters most in healthcare.
What Good Accountability Would Look Like
Although there is no consensus on a single model of AI accountability, many researchers and regulators emphasise several principles: allocation of responsibilities before deployment; transparency about how systems are developed; mandatory reporting of significant incidents; ongoing monitoring of performance in real clinical settings; and effective mechanisms for patients to seek explanation, and obtain redress when harm occurs. (WHO, OECD, MHRA)
If you were harmed by a clinical error that involved an AI system, what would you want to happen? Would your answer be different depending on whether you were a patient or a clinician?
Unit 3 Summary: The Outcome Isn't Inevitable
Why the outcome isn't inevitable
The future of AI in healthcare, including its benefits and risks, is not determined in advance. It will be shaped by choices made in research labs, in boardrooms, in regulatory agencies, in hospital trusts, and at the bedside. The people best placed to shape those choices well are those who are well-informed; this course hopes to provide the starting point.
Think about one specific concern from this unit that you would want addressed before an AI system was deployed in your care, or in your workplace. Hold that concern in mind as you go into Unit 4. It will help you identify what kind of action feels most meaningful to you.
BONUS - The Systemic Risks: Power, Data, and a Two Tier System
Beyond the clinical risks of individual AI tools lies a set of larger, systemic questions. These concern not whether a specific AI system works, but what kind of healthcare we are building as AI becomes embedded in medicine at scale.
The concentration of power & dependency on a handful of companies
The infrastructure of the most capable AI systems is built by a few very large technology companies, creating a form of structural dependency. As hospitals integrate AI into clinical workflows, they may come to rely on infrastructure they neither own nor control. Such dependence is not new; healthcare already leans on pharmaceutical companies, device manufacturers, and software vendors. But AI could become embedded across diagnosis, documentation, resource allocation simultaneously, concentrating in a few providers what was once spread across many.
At an international scale, the implications are serious. Healthcare systems deeply dependent on AI infrastructure they don't control may find that control exercised in ways that place commercial priorities above public health.
Data: who owns it, who profits from it, who is protected by it
Healthcare data is among the most sensitive personal information that exists. It is also, in the context of training powerful AI systems, extraordinarily valuable. The question of who owns health data: Individuals, healthcare providers, insurers, technology companies is contested and inconsistently answered across different legal systems.
In 2019, it emerged that Google had obtained data on approximately 50 million Americans through a partnership with a major US health system, without patients being informed. This is not an argument that patient data should never be used to train AI, the benefits of doing so are real. It is an argument that the terms of that use need to be transparent and the consent mechanisms genuine.
The risk of a two-tier system
In healthcare systems with both public and private provision (which includes most countries in the world) advanced AI risks reinforcing a two-tier dynamic. Well-funded private systems and wealthy patients gain early access to the most capable diagnostic and treatment AI. Under-resourced public systems and lower-income patients receive whatever is left.
This is not inevitable. But it is the path of least resistance. Preventing it requires deliberate policy choices about how AI in healthcare is funded, regulated, and distributed. These choices need to be made early, before the market settles into patterns that are hard to reverse.
Of the systemic risks described here: concentration of corporate power, data governance, a two-tier system. If you had to nominate one of these systemic risks as the most urgent to address, which would it be, and why?