The Radiologist And The Reading List

AFor a decade the reading of medical images has been the field in which artificial intelligence was expected to arrive first and to arrive completely. A much-repeated remark made at a conference in 2016 — that hospitals should stop training radiologists altogether — is now quoted mainly by people who wish to ridicule it, but at the time it captured a genuine expectation, and the systems that followed have turned out to be narrower than that forecast implied. One flags suspicious densities on a mammogram; another marks a bleed on a head scan and pushes the case up the reading list; a third measures the volume of a tumour more consistently than a tired human can. What they have in common is their raw material: each learns from images that clinicians have already labelled, and each therefore inherits whatever those labels happen to record. Vendors describe this as the distillation of expert judgement. Sceptics reply that it is the fossilisation of it.

BPriya Raghunathan, a radiologist at the Ferndale Institute of Diagnostic Medicine, has spent seven years asking how much difference the software actually makes. Her group assembled 214,000 chest and head examinations reported between 2017 and 2023, each with the eventual diagnosis attached. Judged alone, the commercial systems identified 87 per cent of the abnormalities later confirmed, against 84 per cent for the doctors who had reported the same images at the time. On the other side of the ledger the machines were the clumsier readers: they called 9 per cent of healthy scans abnormal, where the reporting doctors called 4 per cent. Raghunathan's reading of her own figures pleases neither camp. The tools are not the flawless second opinion the sales literature promises, nor the menace to patients their critics describe; they are readers of roughly human quality who never tire and never queue. What makes them matter, she argues, is not what they see but when they see it.

CThat last sentence is a claim which can be tested, and in 2024 she tested it. Eleven hospitals agreed to let a triage program reorder their unread scans for eighteen months, so that images the software judged urgent were presented to a doctor first. Across 46,000 examinations the median interval between a scan being taken and treatment beginning for a major stroke fell from 94 minutes to 61. The gain, however, was confined to the six sites which kept a stroke specialist on the premises at every hour of the day; at the remaining five, where such cases waited for a doctor to be summoned from home, the reordering saved four minutes, a difference her team reported as indistinguishable from chance. The software had shortened one queue and exposed another. Raghunathan is careful about what the trial establishes: nothing in it measured whether patients treated sooner went on to recover better, only how quickly their treatment began.

DQuentin Ravindran, a health economist at the Braemar School of Public Health, accepts the trial and quarrels with the lesson drawn from it. A diagnostic tool, he says, is never usefully compared with an ideal unit; it is compared with the department that existed the week before, and that department was one in which the scans taken at three in the morning were read by whoever happened to be awake. Following fifty-eight hospitals which installed automated triage between 2019 and 2022, he found the proportion of urgent findings picked up outside normal working hours rising from 64 per cent to 79 per cent. Ravindran offers two qualifications without being asked. The hospitals chose to buy the systems, so they may have been the hospitals already working hardest to repair their night shifts; and repeat examinations, ordered when a flagged scan turned out to be innocent, rose over the same period from 6 per cent of the workload to 11.

EThe dispute is no longer confined to journals. A rule in force in the Canadian province of Alberta since 2023 requires every hospital using diagnostic software to record each occasion on which a clinician sets aside its recommendation, and to make an annual summary of those records public. Ravindran treats the recording duty as the most valuable element of such rules, because a department obliged to log its disagreements can afterwards be asked what came of each one. Raghunathan, whose numbers are the ones campaigners quote, is here the more doubtful of the two. A log, she points out, shows only that a doctor and a program differed; it does not show which of them was right, and a hospital with incomplete follow-up records will never find out. Both object to the vendors' habit of advertising accuracy figures without stating which readers, and which patients, those figures were measured against.

FWhat is taking shape is less an answer than a partition of the question. Where the abnormality is a thing visible in the image itself — a fracture, a large bleed, a dense mass — the systems perform close to their published figures, and their mistakes can be caught by looking again. Where the diagnosis turns on what is not in the picture, such as a patient's history, medication or account of their own symptoms, the model is trained on labels which were themselves provisional opinions, and feeding it further images will not mend that. Which of the two kinds of work fills more of a radiologist's day is disputed, and neither researcher will put a figure on it. What both refuse is the framing the public argument has settled into: a choice between an impartial machine and a fallible doctor, when the machine was assembled out of what the doctors wrote down.