How Does It Work?
Introduction: Why This Matters
Unit 1 showed you what AI can do in healthcare. But how do these systems actually work?
You don't need to understand the engineering to use AI tools, any more than you need to understand internal combustion to drive a car. But having a basic mental model of how these systems are developed and work makes it much easier to understand what their limits are, and why the risks coming up in Unit 3 can be structural rather than incidental.
This unit covers four things.
- The core idea: how AI learns from examples rather than rules
- How an AI tool gets built: the steps taken to develop a clinical AI system
- Two types of AI that matter in healthcare: Discriminative AI versus generative AI
- The technical terms you will encounter: machine learning, deep learning, and natural language processing - and why training data is the most important variable of all
A reminder that none of this requires a technical background.
The Core Idea: Learning from Examples
At its heart, AI is a broad umbrella covering any attempt to get computers to do things that would otherwise need human judgement. Nearly everything you'll come across in healthcare labelled "AI" is built by machine learning.
Machine learning is the part of the AI umbrella where nobody tells the computer how to reach its answer. Rather than a programmer writing explicit rules: "if this, do that", an AI system is shown thousands or millions of examples and develops its own rules by finding patterns across them.
The steps of developing these tools are broadly consistent across different applications, and are explained below.
From Data to Deployment: How an AI Tool Gets Built
Understanding how an AI tool gets built is not just a technical curiosity. It is the foundation for trusting one in a clinical setting. At each stage of development, decisions are made that will shape whether the finished tool helps patients and can be used safely in real-world practice.
Step 1: Data Collection and Organisation
Everything begins with data. For a tool that reads chest X-rays, that means gathering thousands of X-ray images. For a tool that drafts discharge summaries, it means large volumes of clinical text. For a sepsis prediction model, it means many patient records including vital signs, lab results, and outcomes.
This data is then organised: labelled, cleaned, and sorted into categories the system can learn from. An X-ray labelled "pneumonia present" or "pneumonia absent." A patient record labelled "developed sepsis" or "did not develop sepsis."
This step is crucial. If the data collected reflects only certain patient populations, certain hospital settings, or certain time periods, the finished tool will reflect those limitations too. A tool can only be as good as the data it was built with.
Step 2: Training
The system is then exposed to this labelled data repeatedly, learning to identify which patterns predict which outcomes. It adjusts its internal rules after each pass through the data, getting progressively better at distinguishing one label from another.
Training produces a statistical model. One that predicts outcomes from patterns in the data it was given. It has not learned to understand; it has learned to predict. That distinction matters when it encounters a situation the training data didn't prepare it for.
Step 3: Testing on Unseen Cases
Once trained, the system is tested on cases it has not seen before, data held back from the training process specifically for this purpose. This is where the real performance is revealed: not how well it memorised its training examples, but how well it generalises to new ones.
A tool that performs brilliantly in this internal test may still fail in clinical practice. The test data was collected in the same way & from the same kinds of patients, as the training data. Not from the clinical environment you may be planning to use it in.
Step 4: Refinement
Where the system makes errors, those errors are used to adjust its rules. This iterative process continues until performance reaches an acceptable threshold.
In practice, no system is perfect: the question is whether its errors are acceptable in type, frequency, and distribution across different patient groups.
A tool that errs at the same error rate for all patients is a different proposition from one that performs well on some groups and poorly on others. Generalised use of the latter is not acceptable, even if the average performance looks acceptable.
Step 5: Assessment and Deployment
Before a clinical AI tool can be used on real patients, it should be assessed against a defined standard: a clinical trial, a regulatory review, or an independent validation study. Regulatory bodies like the FDA in the US, the MHRA in the UK, and notified bodies under the EU Medical Devices Regulation in Europe have frameworks for approving AI as a medical device, though these frameworks are still catching up with the pace of tool development.
A tool that performs well in controlled testing may behave quite differently once it meets the messy real world. This gap between testing and deployment is one of the most common failure points in clinical AI. This is why assessment should not end when a tool goes live. A model validated in one hospital population may under-perform in another.
Post-deployment monitoring: tracking how the tool performs in practice, for real patients, over time is as important as everything that came before it. It is also in most organisations, the least resourced step.
What this means when you evaluate a tool
Understanding how training data works changes how you ask questions about clinical AI. The question is not only "how accurate is it?" it is "what was it trained on, and does that match my patients?". A system with excellent trial performance may under-perform in your clinical population for reasons that were entirely predictable from its training data.
Share an example of a time when a standard clinical "rulebook" or guideline didn't quite fit a patient sitting in front of you (or a time when you as a patient, didn't quite fit the standard "rulebook"). Now, translate that experience to AI: What steps should you take to verify that the tool aligns with the patient before trusting its recommendations?
A Way To Distinguish Types of AI in Healthcare
There are many different types of AI systems, powered by different complex mathematical processes, going into the detail of this is beyond the scope of this course. But a useful distinction for AI in Healthcare is whether the AI system is sorting or creating information.
Discriminative AI: Sorting Information
Most clinical tools developed over the last decade fall into this category. These systems are designed to look at data (like a skin lesion or a lab result) and discriminate between different possibilities. It asks: "Is this Category A or Category B?"
An imaging AI that flags a potential fracture is a sorter. It doesn't understand what a bone is. It has simply learned to recognise the mathematical pattern of a bone that is "broken" versus "intact" from the thousands of labelled examples it was trained on. An AI that predicts sepsis from vital signs is a sorter. An AI that grades diabetic retinopathy from a retinal photograph is a sorter.
Discriminative AI tends to be the most rigorously validated type deployed in clinical settings. Its outputs are bounded; it produces a classification or a risk score, which makes it easier to test, audit, and hold accountable.
Generative AI: Creating Information
This is the more recent technology behind tools like ChatGPT and ambient voice technology (AVT). Instead of just labelling what it sees, generative AI uses its pattern-matching skills to create new content. In healthcare, this might look like an AI taking the raw data from a consultation and generating a clinical note, a formal discharge summary, or a response to a patient's question.
Generative AI carries a specific and important risk: the generation of incorrect or unsupported information. Because these systems work by predicting what token or phrase is most likely to come next, they are optimised to produce outputs that sound right, not outputs that are right. A generative AI can state or omit a drug dosage, clinical fact, or patient history detail with complete confidence while being entirely wrong. This is not a bug that will be fixed in a software update; it is a consequence of how the technology fundamentally works, although the frequency of such errors can be reduced.
For clinicians reviewing AI-drafted documents, and for patients reading AI-generated health information, this is likely the most important thing to understand.
Never enter identifiable patient or staff data (names, numbers, photos, etc.) into unapproved public AI tools. The rule is zero tolerance. Use organisation-approved tools according to local policies instead.
Why Distinguishing Types Of AI Matters
Separating tools based on how they were designed shifts how healthcare leaders and clinicians assess safety, governance, and workflow use.
- A sorting tool requires strict validation of diagnostic accuracy, sensitivity, and specificity.
- A creating tool requires strict oversight to ensure outputs are factually sound, education regarding clinical liability, and controls for safe communication.
Key Terms
You will encounter several terms used to describe AI development and capabilities that appear frequently in conversations about clinical AI.
is the subfield of AI that underlies most clinical tools; it's using mathematical algorithms to find patterns in data without being explicitly told what to look for.
takes this further, using structures called 'neural networks' to process large amounts of complex, unstructured data. Like medical notes, audio files, video footage, medical scans, PDFs.
is what enables AI to read and write clinical text, "understanding" the language of discharge summaries, referral letters, and clinical notes.
Some others:
Think about the AI tools described in Unit 1. Which of them are sorters and which are creators? For which tools do you naturally feel more comfortable placing your trust?
Unit 2 Summary
Here is what we covered:
Units 1 and 2 have been about understanding. What AI is doing. How it works.
The next section shifts from the technology itself to its practical impacts on patients, clinicians, health systems, and society.
If you read one part of this course carefully, make it the next one.