Audit Evidence Architects Can Actually Produce

The question arrives eventually

In a regulated organisation, someone will eventually ask about the architecture: who approved this design, when did this control appear in the model, who had access to this project in March.

These are reasonable questions. In most architecture practices they trigger a week of archaeology through email, shared drives and people's memories, and the answer is qualified with "as far as we can tell".

Figure 1: The four questions, and what has to have been recorded to answer them
Figure 1: The four questions, and what has to have been recorded to answer them

Four things to record

Actor and action

Not just "the model changed" but who did what: opened, locked, published, restored, granted, revoked. The set of auditable actions is small and it is worth enumerating it deliberately rather than logging whatever happened to be convenient.

Timestamp and resulting revision

An audit event that says a publish happened, without saying which revision it produced, cannot be joined to the content. The link between the event log and the revision chain is what makes both useful.

The prior state

"What did it look like before" is answerable only if the previous revision still exists. This is where immutable revisions and audit stop being two features and become one capability.

The permission context at the time

The one most implementations miss. Asked "who could have changed this in March", a current permission list is the wrong answer — grants change. Either version the grants or log the effective permission at the moment of each action.

Tamper-aware, not tamper-proof

Be careful with the vocabulary. Anyone with database administrator rights can alter rows, and claiming otherwise will not survive contact with a competent auditor.

What is achievable is tamper-evidence: hash chaining, so that altering an event invalidates everything after it; append-only permissions at the application layer; and export to a system outside the repository's own control. The honest claim is "modification is detectable", not "modification is impossible", and the honest claim is the one worth making.

If your audit trail lives in the same database as the data it audits, with the same administrators, say so. Auditors are used to this and mostly accept it. What they do not accept is discovering it themselves after being told otherwise.

Export matters more than the UI

An audit view in the administration portal is convenient. The thing actually needed during an audit is an export: a defined date range, a stable format, delivered to someone who will open it in a spreadsheet.

Design for that. It should be possible to hand over "every action against this repository between January and June" without an engineer writing a query, because the person asking has a deadline and the engineer is on another project.

What not to log

An audit trail that records every read produces enormous volume and obscures the events that matter. It also creates a surveillance dataset about your architects, with the privacy obligations that implies.

Log state changes comprehensively. Log reads selectively — access to sensitive projects, exports, administrative views — and be able to justify why each one is recorded. The test is whether you could explain the logging policy to the people being logged without embarrassment.

Retention

Audit events are small and they are the cheapest insurance in the system. Keep them for at least as long as the decisions they support remain relevant — which in most regulated environments means years, not months.

Set the retention explicitly rather than letting it emerge from a database maintenance job. The failure mode is discovering, during an audit, that events older than ninety days were being pruned by a housekeeping task nobody remembered.

Correlation identifiers

One implementation detail worth building in from the start: a correlation identifier that ties together everything arising from a single user action.

A publish generates an authorization check, a lock validation, a concurrency check, a database transaction, a revision, an audit event and several log lines. When something goes wrong at 3am, the difference between a five-minute diagnosis and an afternoon is whether those can be gathered with one query.

It also matters for the audit itself. "Show me everything that happened as part of this publish" is a reasonable question, and answering it by correlating timestamps across three logs is how you end up unsure.

Who may read the audit

A question that is easy to get wrong in both directions.

Making audit readable by all administrators seems natural and means the people whose actions are most consequential can read their own trail — which is not fatal, since they cannot alter it, but it is worth being deliberate about.

Making it readable only by a separate audit role is cleaner and creates an operational problem: administrators diagnosing an incident genuinely need to see what happened.

The workable answer is usually both, with a distinction: administrators can read operational events, a separate role can read the complete trail including administrative actions, and reads of the audit itself are logged.

What an auditor is actually testing

Architects prepare for audits by assembling evidence that the architecture is good. That is not what is being examined. The test is whether the control you described operates as described, and the finding comes from the gap between the two.

This reframing changes what is worth building. A beautifully modelled estate with no record of who approved what will fail. A modest estate with a complete decision trail will pass. The evidence is procedural, and the architecture quality is somebody else's conversation.

It also changes what to claim. A documented control that says architecture changes are reviewed monthly, when they are reviewed when someone remembers, is a finding you wrote yourself. Describing the cadence you actually sustain is not lowering the bar — it is the difference between a clean report and a remediation plan.

Reconstructing a point in time

The question that separates a real audit trail from a log file is some version of: on this date, what did the architecture say, and who could see it? Both halves are harder than they look.

The first half needs immutable revisions with reliable timestamps, which most repositories have. The second needs the permission history — not the current grants, but who held which grant on that date — and almost nothing records it. Current state cannot be run backwards: a person who left the company has no grants today and had all of them in March.

Recording grant and revocation as events, with timestamps, is a small amount of work at build time and the only way to answer the question later. It is worth checking a product for this specifically before buying it, because "full audit logging" in a datasheet frequently means model changes only.

Evidence for things that did not happen

Some of the most useful evidence is negative: nobody outside this group could read the model, no change bypassed review, no publish occurred between the baseline and the submission.

Negative claims are hard to evidence from a log, because the absence of an entry could mean the event did not occur or that logging was off. What closes that gap is a record that logging was continuously operating — a heartbeat, a sequence number with no gaps, or a periodic integrity check that is itself logged.

A sequence number is the cheapest of these and the most convincing. If every audit event carries a monotonically increasing number, a missing range is visible, and an unbroken range across the period under examination supports the negative claim in a way that "we found nothing" never will.

Getting evidence out of the tool and into the pack

Audit evidence is consumed outside the repository, by people who will not be given access to it, in formats they can annotate and attach. A repository whose audit trail can only be viewed in its own interface has evidence nobody can use.

The export needs three properties. It has to be filterable by date range and scope before export, because nobody wants a hundred thousand rows. It has to be a plain format — CSV or JSON, not a rendered report — so it can be sorted and pivoted by whoever received it. And it has to carry enough context to be read standalone: an entry saying user 4471 modified element 88213 is useless outside the system that can resolve those numbers.

Resolving identifiers at export time, rather than storing resolved names in the log, is the right trade. Names change; identifiers do not; and the export is where the two need to meet.

The evidence that comes from publication

A published portal is itself evidence, and it is the kind auditors find most persuasive because it demonstrates the control rather than describing it.

What it demonstrates is that the architecture was available to the people who were supposed to have it, in a form they could read, at a specific date. That is a control operating, witnessed. A repository that only architects can open supports a claim that the architecture exists; a published register supports a claim that it was communicated, which is usually the actual control being tested.

For this to hold, the publication has to be dated, scoped and retained. A portal regenerated in place every month, overwriting its predecessor, cannot evidence what was visible in March. Keeping the published output for each cycle — they are static files and cost almost nothing — turns the publication history into an evidence archive that needs no further work.

Preparing in the two weeks before

Notice of an audit usually arrives with enough time to assemble but not enough to build. What can genuinely be done in a fortnight is narrower than teams hope, and knowing the list stops the panic spending effort in the wrong place.

  1. Export the access grant list and have someone accountable sign it. If a periodic review was supposed to happen and did not, doing it now and saying so is better than an unreviewed list.
  2. Run the restore drill. An untested backup is a finding, and testing it takes an afternoon.
  3. Reconcile the documented cadence against reality and correct the document where it overstates. Amending a control description before an audit is legitimate; discovering the gap during one is not.
  4. Assemble the decision trail for the two or three changes most likely to be sampled — the largest, the most recent, and anything touching a regulated system.

What cannot be done in a fortnight is retrofitting a permission history or a decision record that was never kept. Those are the reasons to build them on a quiet day.

Evidence that survives a change of tooling

Audit periods outlast products. A repository bought in 2022 and replaced in 2027 leaves five years of decisions, approvals and access history that a regulator may still ask about, and the usual outcome is that the old system is decommissioned and the evidence goes with it.

Keeping the old instance running as a read-only archive is the obvious answer and the expensive one — it means maintaining a database, an application and a set of credentials for a system nobody uses, and patching it for years.

The cheaper approach is to export the audit trail on a schedule, in a plain format, to wherever the organisation keeps long-lived records. Monthly, filtered to nothing, resolved identifiers, compressed. It is a few megabytes a year and it is independent of any product. If the repository is ever replaced, the evidence archive is already outside it and the decommissioning is a technical exercise rather than a compliance one.

Do the same for baselines. A baseline is a content hash and a revision; exporting the model at each baseline alongside its metadata means the states someone approved remain readable even when the system that produced them is gone.

The evidence calendar

Audits are treated as surprises and almost never are. The regulator's cycle is published, internal audit plans a year ahead, certification renewals fall on anniversaries, and the supplier questionnaires cluster around contract renewals. A practice that writes these dates into one small calendar converts evidence production from an emergency into a season — and seasons can be prepared for on quiet Fridays instead of frantic Sundays.

The calendar needs only three columns: the event, the evidence it historically requests, and the check that proves the evidence is currently producible. Before each event's window, run the checks — is the export current, do the retention jobs show green, does the point-in-time reconstruction still work for a date in the relevant period, has anything in the tooling changed since the pack was last assembled. Most checks take minutes; the ones that fail are precisely the ones worth discovering a month early. Over a few cycles the calendar also accumulates the most useful institutional memory an audited practice can have: what was actually asked last time, by whom, and which answers landed well — knowledge that otherwise lives in the head of whoever survived the previous audit and leaves with them.

The deeper effect is cultural. A practice with an evidence calendar has admitted that audits are part of its normal operating rhythm rather than hostile weather, and that admission changes how everything upstream is built — records designed to be exported, workflows designed to leave traces, tooling chosen with the March question in mind. Evidence stops being something produced about the practice and becomes something the practice simply has, which is the end state every section above has been describing from a different angle.

The thread joining every section here is that evidence is a design property, not a retrieval skill. Records that were written at the moment of action, by the system that performed it, into a store that cannot be quietly amended, answer questions by existing; everything else answers questions by heroics. Build the four records, keep them exportable, rehearse the export once a year — and the arrival of the question, whenever it comes and whoever asks it, becomes an administrative event rather than a bad month. That is as close to audit comfort as an architecture practice gets, and it is entirely purchasable in advance, at Tuesday prices rather than crisis ones.

And when the question finally arrives, answer it slightly faster than expected — nothing builds an auditor's confidence in a control like a practice that was visibly ready.