The engineer wants the trace, the auditor the file. The court wants someone it can name. The claimant wants a while.
Ask four people what an explanation of the March determination should contain and you will receive four incompatible specifications, each entirely reasonable.
The engineer wants a causal trace: which inputs moved the score, by how much, through which features. The auditor wants evidence that the process followed its own rules, that the threshold in production matched the threshold approved, and that nothing was changed without a record. A court wants reasons attributable to an identifiable decision-maker, stated at the time, capable of being tested against the law. The claimant wants grounds they can understand and dispute, in language they can use, delivered to them and not about them.
Institutions typically produce one of these and send it to everybody. Usually it is the engineer's version, because that is the one the system can generate automatically, and it arrives at the claimant's kitchen table as a bar chart of feature attributions with a covering letter.
The forum decides the form
Mark Bovens, in a much-cited 2007 article in the European Law Journal, defined accountability as a relationship between an actor and a forum. The actor is obliged to explain and justify their conduct. The forum can pose questions and pass judgement, and the actor may face consequences.
The definition is dry, and its usefulness lies precisely in the dryness. Accountability, on this reading, has no content until you know who the forum is. An explanation that satisfies an engineering review board is an account rendered to the wrong forum when it lands in front of the claimant. It may be accurate in every particular and still fail as an account, because the relationship it serves is with someone else.
The four people at the start were four forums. Each can hold the institution to account in a different way, and each needs a different account to do so.
Excuses and justifications
Sociology got here earlier, and from a less institutional direction. Marvin Scott and Stanford Lyman's 1968 paper "Accounts" treated the account as a statement a person makes to explain untoward behaviour when others call it into question. They sorted accounts into two families. Excuses admit the act was wrong and deny full responsibility for it (I was misled; the system said so). Justifications accept responsibility and deny the act was wrong (it was the right call, and here is why).
Run the October hearing through that sorting and something uncomfortable appears. Every one of the four true denials from The Right Question to the Wrong Defendant is an excuse. The retired model, the software-only vendor, the successor with no record, the reviewer who relied on the system: each denies responsibility without saying a word about whether the determination was right. Nobody in the room has offered a justification, because a justification requires someone willing to own the decision.
A claimant who receives only excuses has been told a great deal about the organisation chart. They have learned nothing about their case.
What people actually ask for
Tim Miller's 2019 survey, "Explanation in Artificial Intelligence: Insights from the Social Sciences," brought this literature to bear on machine learning and found the field had been building for the wrong audience. People asking "why" are usually asking a contrastive question (why this outcome rather than that one?) and they select a small number of causes from it while ignoring the rest. Explanation, Miller stressed, is social: a transfer of understanding between people, shaped by what the asker already knows and believes.
A feature-attribution chart is none of those things. It answers every "why" at once and addresses nobody in particular.
The claimant's contrastive question is simple enough to state. Why was I declined when someone like me would have been approved? Answering it requires deciding which differences mattered, and that is a judgement about grounds, the very thing A Reason Is Not a Reconstruction found missing from the log.
One file, four accounts
None of this argues for producing less. The engineer's trace remains essential for debugging, and the auditor's evidence for assurance. The error lies in treating one artefact as though it discharged every obligation, and in letting whichever account is cheapest to produce stand in for the one actually owed.
A workable design starts from the forums. Name them before deployment. Decide what each is owed, and who inside the institution is responsible for giving it. For the affected person, the account has to be contrastive and addressed to them, and it has to come from someone able to offer a justification when an excuse will not do.
Job, notably, is surrounded by people offering accounts. Every one of them is addressed to the wrong forum.
Companions
- The four excuses: The Right Question to the Wrong Defendant.
- The log and its limits: A Reason Is Not a Reconstruction.
- Mark Bovens, "Analysing and Assessing Accountability: A Conceptual Framework," European Law Journal 13(4), 2007.
- Marvin B. Scott and Stanford M. Lyman, "Accounts," American Sociological Review 33(1), 1968.
- Tim Miller, "Explanation in Artificial Intelligence: Insights from the Social Sciences," Artificial Intelligence 267, 2019.
These notes come out of Sociable Systems, a practice that reads AI-shaped documents the way a hostile reviewer will, before a lender or a court finds the gap. The argument has an operational form: the Interim Protocol sets out four rules for AI use in environmental and social deliverables, covering disclosure at touch-point grain, evidence custody, the phrases no automated screening may settle, and a hostile read before anything ships. Free, and written to be cited or retired once institutional guidance arrives.