When Should An Expert Trust A Hunch?

AIn 1994 a charge nurse on a neonatal ward in Dunmore asked for a newborn to be treated for sepsis hours before any laboratory result supported her. She was right, and afterwards she could not say why: the baby's colour, she offered, had been 'off'. Stories of this kind are told in two quite different registers. In one, they are evidence that years of practice deposit in the mind a store of knowledge which cannot be put into words but can be acted upon. In the other, they are survivorship: the ward remembers the nurse who was right and forgets the dozens whose hunches were wrong and whose errors ordinary care absorbed. The disagreement is not about whether experts have intuitions — nobody doubts that — but about when an intuition deserves to be believed, and by whom.

BMateo Yamashita, a cognitive psychologist at the Lindholm Institute for Decision Research, has spent her career on the sceptical side. Her method is to set experts against a formula so crude that beating it should be trivial; a poor showing against such an opponent, she reasons, cannot afterwards be blamed on the sophistication of the rival. Between 2009 and 2017 she assembled 41 such comparisons: parole boards against a three-item scoring sheet, admissions tutors against a weighted average of prior marks, radiologists against a checklist of image features. The formula matched or beat the experts in 33 of the 41 contests. What interests Yamashita is not the margin, which is usually small, but its direction, and the fact that the experts were confident throughout. Intuition, she argues, is not a separate faculty but a substitution: asked a hard question, the mind quietly answers an easier one, and the answer arrives with a feeling of knowing attached, because that feeling is produced by the ease of the substitution rather than by the quality of the answer.

COwen Radcliffe, who studies decisions in the field rather than the laboratory, regards the tournament as rigged by its setting. His own work began with fireground commanders, 156 of whom he interviewed between 1996 and 2003 about incidents they had directed. He had expected to hear of options weighed against one another; instead he found that in 81 per cent of the incidents the commander acted on the first workable course of action that came to mind, and had rehearsed it mentally rather than compared it with any rival. Radcliffe's term for this is recognition: the commander does not calculate but classifies, treating the fire as a case of a type he has met before, and the classification arrives with a course of action already attached to it. The types recur: a commander of twenty years' standing has met most of them many times over. Where Yamashita sees a shortcut standing in for analysis, Radcliffe sees a form of perception which analysis could not have supplied in the ninety seconds available.

DThe two met in 2005 at the invitation of a journal editor who expected a duel and got a treaty instead. Both accepted that the argument had been conducted on separate ground: Yamashita's professionals worked in domains where outcomes are slow, noisy or contested, and Radcliffe's in domains where a wrong reading of a burning building announces itself within minutes. The agreement they published sets two conditions. An environment must be regular enough that its cues genuinely predict what follows; and the practitioner must have had enough opportunity to learn those regularities, which means feedback that is both prompt and unambiguous. Where both conditions hold, intuition is to be trusted; where either fails, it is not. Neither condition, they stressed, is a property of the person. A chess player and a share-picker may have identical experience, temperament and confidence, and the first will develop skilled intuition while the second cannot, because the board rewards pattern and the market does not.

EThe framework's most awkward consequence concerns confidence. If validity is a property of the environment, then no amount of introspection will tell practitioners which side of the line they are standing on. Yamashita has tested this repeatedly, most pointedly with 60 clinical psychologists asked in 2014 to predict which of their patients would still be in treatment a year later; those who described themselves as very confident were correct 47 per cent of the time, and the rest 45 per cent, a difference well inside the noise. The reason, on her account, is that confidence tracks the coherence of the account the mind has assembled, and coherence is easiest to achieve when evidence is thin, since there is less to reconcile. Radcliffe does not contest the finding but declines the inference. Confidence is uninformative in the aggregate, he replies, which is not the same as saying that an experienced person can never tell a familiar case from a strange one.

FWhat follows in practice is less tidy than either position. Yamashita's remedy is structural: where the environment is poor, replace the judgement with a rule, or at least oblige the expert to record separate ratings before forming an overall impression. Radcliffe's is educational: build simulators that compress a decade of rare events into a fortnight, and make the feedback arrive while the decision is still fresh. The remedies address different weaknesses, and several professions now run both at once. What neither can supply is a verdict on the many fields — much of law, senior management, most of psychiatry — where feedback exists but arrives years late and muddled by everything that happened in between. Yamashita treats these as low-validity by default; Radcliffe holds that the question is open. Until it is settled, the practitioner in such a field is left with the least comfortable of the two camps' shared conclusions: that the feeling of knowing carries no information about whether one does.