Auditability and the Decision Record

01

The problem

A consequential institutional decision is rarely contained in one system.

The evidence sits in documents: contracts, valuations, technical reports, correspondence. The assumptions sit in spreadsheets, often in one analyst's copy. The judgement is formed in meetings and survives as a slide or a memory. The approval arrives as an email, a signature or a row in a workflow tool. The resulting action lands in a system of record that stores the outcome and almost nothing about how it was reached.

Increasingly, AI takes part in the preparation. It reads the documents, extracts candidate figures, drafts the summary a committee will read, and answers questions about the material along the way.

Each of those systems logs something. None of them records the decision.

This is not a failure of logging discipline. It is a gap in what is being logged. An application log answers questions about software behaviour: which user opened which record, which endpoint was called, which field changed. Those questions matter for operations and for security. They are not the questions asked when a decision is challenged a year later.

02

Activity logs versus decision records

The distinction we find useful is between two artefacts with two different purposes.

An activity log

answers: what happened in the software? It is event-shaped, high-volume, scoped to a system, and organised around time. It is the right structure for detecting intrusion, diagnosing failure and reconstructing system state.

A decision record

should answer: why was this decision authorised? It is decision-shaped, comparatively small, scoped to one decision rather than one system, and organised around justification rather than chronology.

The two are not substitutes. An institution can hold a complete activity log across every system it operates and still be unable to explain a decision, because no log was ever asked to carry what justifies one: which facts were treated as authoritative, which assumptions were accepted, which calculation was official, which policy applied, and who was permitted to approve the result.

The underlying concern is not new. Institutions have long kept partial approximations of such a record — credit memoranda, committee minutes, procurement evaluation reports — though these were built for other purposes, and what any one of them preserves varies widely between organisations and over time. What is new is that the preparation is increasingly automated while the record around it stays manual, so the two drift further apart.

Regulation has begun to formalise parts of this. The EU AI Act requires high-risk AI systems to allow automatic recording of events over their lifetime, and to be designed so that human oversight can be exercised effectively. The NIST AI Risk Management Framework treats accountability as presupposing transparency, and names documentation and provenance among the mechanisms that support it. Both point at the same gap. Neither specifies what a decision record should contain, which is one reason this remains an open question rather than a settled compliance requirement.

03

Proposed components of a decision record

What follows is Inversiq's working architectural hypothesis. It is not a standard, and it is not a claim of completeness. We state it in this much detail because a vague proposal cannot be argued with.

A decision record should be able to hold:

Source evidence

the documents, datasets and systems the decision drew on.

Provenance

the link from each material figure back to the document, page or system it came from.

Accepted facts

what the institution treated as true, held distinct from what was merely extracted.

Assumptions

the judgements that were not derived from evidence, recorded as judgements.

Model and calculation version

the identity and version of the software that produced each material figure.

Validation results

which automated checks ran, and what they returned.

Policy and rules applied

the thresholds, requirements and constraints in force at the time.

Exceptions

where a rule was not satisfied, and how that was handled.

Reviewer actions

what a named person examined, changed, questioned or accepted.

Authority

who was permitted to approve this class of decision, and under what limit.

Approvals

the approvals actually given, by whom, and when.

The final decision

the outcome, stated as a decision rather than as an output.

Subsequent governed action

what the institution then did, and under whose authority.

Temporal state

timestamps and version state sufficient to reconstruct the above as it stood at the time.

The claim is not that every decision needs all fourteen. It is that a record missing several of them cannot answer the question a decision record exists to answer. Which of the fourteen are load-bearing is one of the open questions in section 06.

04

Reconstructability

The practical test of a decision record is whether an institution can later reconstruct six things.

• —What was known.
• —What was assumed.
• —Which calculation ran, in which version.
• —Which exceptions existed, and who reviewed them.
• —Who held the authority to approve.
• —Why the decision was permitted to proceed.

Reconstruction is a harder requirement than retrieval. Retrieval returns the document as it exists now. Reconstruction returns the state as it stood then — including evidence that has since been superseded, assumptions that have since been revised, and policy that has since changed. A record that silently reflects current state cannot support reconstruction, because the version of the world that justified the decision has been overwritten.

Provenance work outside AI is directly relevant here. The W3C PROV data model formalises provenance as a record of the entities, activities and agents involved in producing something. It is a general vocabulary rather than a decision-record specification, but it makes explicit an assumption institutional software usually leaves implicit: that how a value was derived is itself information worth storing.

05

Why AI raises the stakes

We do not argue that AI makes institutional accountability impossible, or that it introduces a problem that did not exist before. Decisions were poorly documented long before language models.

The argument is narrower. As AI performs more of the preparation, interpretation and coordination behind a decision, the distance grows between two statements that are easy to conflate.

The first is a system output. The second is an act with an accountable author. When human analysts performed more of the preparation, accountability was often easier to associate with identifiable people — even where the underlying record itself remained incomplete. As more of that preparation is automated, that association has to be created deliberately in software rather than inferred from who did the work.

Two existing literatures sharpen this. Work on internal algorithmic auditing argues that accountability mechanisms are considerably more effective when designed into a system's development and operation than when reconstructed from outside afterwards. Work on human use of automation shows that people over-rely on automated output in predictable ways — which suggests that a record of what a reviewer actually examined carries more weight than a record that a review step occurred.

06

Open research questions

We do not have settled answers to any of the following.

1. 01

   What is the minimum sufficient decision record?

   The fourteen components in section 03 are a maximal hypothesis. Which are structurally necessary, and which are merely useful?

2. 02

   Which artefacts must be immutable?

   Immutability is costly and not obviously required everywhere. Where is it necessary, and where is versioning sufficient?

3. 03

   How much model reasoning should be preserved?

   Reasoning traces are large, unstable between model versions and of uncertain evidential value. Does structured proposed state carry more of the justification than prose does?

4. 04

   How should corrections and superseded evidence be represented?

   A record that hides its own revisions is misleading. A record that surfaces all of them may be unreadable.

5. 05

   How does the required record change with materiality?

   A major acquisition and a routine renewal plainly do not warrant the same record. Is that a difference of degree, or of structure?