Sorting The Applicants By Machine

ARecruitment has always been an exercise in guessing, but the guessing is now done at a scale no human reader could manage. A large employer advertising forty graduate posts in 2024 might receive twelve thousand applications, and almost none of them will be opened by a person until a program has ranked them. The programs differ. Some parse the wording of a curriculum vitae; some score a recorded interview by phrasing and pauses; a handful, now largely withdrawn, measured facial movement. What they share is a method. Past hiring decisions are treated as a record of what a good employee looks like, and the pattern drawn from them is applied to people who have not yet been hired. Vendors present this as the removal of tired and inconsistent human judgement. Critics answer that it preserves that judgement in a form which cannot be argued with.

BIngrid Halvorsen, an occupational psychologist at the Vestbury Institute of Work, has spent a decade asking whether the ranking is any good. Between 2016 and 2022 her team persuaded nineteen firms to release the scores their screening tools had produced, together with the performance reviews the same candidates received once employed: 34,000 pairs in all. The correlation was 0.21, weak but not nothing, and a little above the 0.17 she recorded for unstructured interviews conducted by managers at the same firms. Her yardstick, she is the first to say, is imperfect, since a performance review is itself an opinion written on a form. Halvorsen's conclusion suits neither camp. The tools are not the impartial instruments their vendors advertise, nor the engines of exclusion their opponents describe; they are mediocre predictors which happen to be mediocre very quickly. What makes them consequential is not accuracy but repetition: one flawed model, licensed to hundreds of employers, delivers the same rejection everywhere at once.

CThat last claim yields a prediction, and Halvorsen has tested it. If one model is copied across an industry, a candidate turned away by a single employer should be turned away by the rest for the same undeclared reason. In 2023 her group submitted 2,400 fictitious applications, matched in qualifications and differing in one respect only, to firms using three widely licensed systems. Applicants whose employment history contained an unexplained interruption of more than eleven months were advanced to interview in 9 per cent of cases, against 31 per cent for otherwise identical applicants. The penalty appeared only where the interruption was recent: breaks more than six years old made no measurable difference. No system, Halvorsen stresses, had been instructed to consider employment history in this way at all; the rule had been learned from the firms' own past decisions and was invisible to the people relying on it.

DTomas Beltran, a labour economist at the Kelso School of Public Policy, accepts the audit but disputes what follows from it. A screening tool, he argues, is never fairly compared with a perfect process; it is compared with the process it replaced, and that process was itself a machine for converting a recruiter's fatigue and preferences into decisions. Studying 71 firms that adopted automated screening between 2018 and 2021, he found that the share of interview slots going to graduates of institutions outside the country's twenty most selective universities rose from 22 per cent to 38 per cent. Beltran volunteers two limitations. The firms in his sample chose the software for themselves, so they may have been firms already intending to widen their intake; and he counted who was interviewed, not who was hired or who stayed.

EThe argument has left the journals. A statute in force in New York City since 2023 obliges employers using automated screening to commission an annual audit of its results and to publish a summary, and several other jurisdictions have drafted similar measures. Beltran regards the duty of disclosure as the most valuable part of such laws, because it compels firms to keep records that can be examined afterwards. Halvorsen, whose findings are the more often quoted by campaigners, is here the more sceptical of the two. An audit, she observes, reports what a model did without asking whether the thing it predicts is worth predicting, and a firm whose own performance reviews are unreliable will pass one comfortably. Both dislike the vendors' habit of publishing accuracy figures without stating what those figures were measured against.

FWhat is emerging is a division of the question rather than an answer to it. Where a job produces an outcome that can be counted, such as parcels sorted, calls closed or defects missed, the models predict performance tolerably well, and their mistakes can at least be detected afterwards. Where success is a matter of judgement, as in most professional work, the model is trained on a target which is itself an opinion, and enlarging the training set will not repair it. Whether the balance of employment is moving towards the countable or away from it is disputed, and neither researcher claims to know. What both reject is the framing that has dominated public argument: that the choice lies between an impartial machine and a prejudiced human, when the machine's raw material is a record of what the humans did.