Every AI workflow needs a black box.

The big story

Let me say up front what this issue is not.

I am not on the bandwagon that worries AI will end the world. The largest labs have real safety responsibilities. They also have liability and market incentives that do not always line up with the needs of a smaller business, and I do not need to settle anyone's motives to run my own company well.

My concern is more practical. I am giving software agents authority to change things: records, schedules, code, messages. When one of them gets something wrong, I want to trace exactly what happened without asking the agent to remember.

Here is what taught me that. I recently found a monitoring job that looked complete on paper. The scanner existed. The schedule existed. The reporting path existed. The controls had even passed a manual test.

But the scheduled run had been failing silently.

Nothing was missing from the feature list. What was missing was the ability to tell configured from working. A dashboard would have shown the last good result. It would not have proved that the current run happened.

What I needed was the business equivalent of an aircraft's flight recorder: an append-only record of what the agent intended, the authority it relied on, what it changed, and the evidence that the result actually occurred. My agents have since built the tamper-evident backbone for that inside my own company, and I am now building the fuller receipt around it so each consequential action carries the business context needed to explain it.

We are giving probabilistic systems access to deterministic business consequences. In plain terms, software that is sometimes wrong now has the power to do things that are hard to undo. The record between those two worlds cannot be an afterthought.

The record

A log is not a black box

Most software already produces logs. A log can show that an API returned 200 or a job exited cleanly. After an incident, those facts rarely answer the questions that matter: What was the agent trying to do? Who authorized it? What changed outside the model? Was the result verified, or just reported as done? Can it be reversed?

The useful record is not a dump of the model's hidden reasoning. That material is noisy, sensitive, and a poor audit artifact. The useful record has two layers: a structured action receipt that explains the business event, and a set of linked fingerprints that makes a rewritten history detectable.

Framework spotlight · The control standard I am building around my existing ledger. A pattern, not a product.

Nothing makes a file impossible to change. The linked fingerprints make a change impossible to hide. In my own company the rule is simple: if the record fails its own check, agents that work through my controls stop consequential work until I understand why.

One field in the receipt does more work than any rollback plan: can this be undone? Most agent work is reversible for free, a file edit with history or a draft that has not gone anywhere. Some is reversible with effort, and for those the trace is enough to design the rollback if the day comes. Some cannot be undone at all: a sent email, a payment, a deletion, a public post. Nothing recovers those, so they get a gate in front of them instead of a plan behind them. A gate is anything that makes the action wait for a decision the agent cannot make alone: a person who approves it, a dry run that shows what would happen first, a staging copy someone reviews, or a limit the agent cannot raise on its own.

Most small businesses do not need an exotic ledger on day one. They do need the entries linked from day one if they want to call the record tamper-evident. Signed snapshots and independent time-stamping can come later. Write the receipt in language a person can read, not compressed agent shorthand. If a customer or a vendor ever disputes what your system did, this record is what you fall back on.

The release decision

Now you can see what happened. Should it have shipped?

A ledger explains the past. It does not tell you whether the next change is ready to go out. That is where a release score comes in.

I have been evaluating AI code-review tools that map a whole repository and score a pull request against it. That is a useful outside challenge, and every business should evaluate the category for itself. But a code reviewer cannot prove the whole business workflow. It cannot prove that a scheduled job is still alive, that a vendor deleted the right data, or that production matches the branch it reviewed.

So the decision is not "buy a reviewer or build a framework." It is: own the release standard, attach evidence to every score, use outside reviewers to challenge the evidence, and let a failed dimension block the release.

Here is what that looks like in my own shop. Agents work around the clock and never wait on me. But a newsletter, an email, or a code change does not leave the building until a person has reviewed the finished work in a staging copy and approved it. The agents keep producing. The release waits.

This is not a five-star rating. The threshold follows the consequence.

A drafting assistant can ship at a lower score than an agent that moves money, edits a customer record, or sends a message in someone else's name.

Continuity

Keep one lane you control

Everything above assumes you can keep operating. Being unworried about the end of the world does not mean being unworried about depending on one provider. Call it concentration risk. Providers change prices. Models get retired. Access rules narrow. A capability that is acceptable today may fall outside a provider's risk posture tomorrow.

Maintain one useful lane the business controls: a local model for bounded internal work, owned hardware or a portable environment, narrow credentials, explicit budgets, and an action ledger that no single provider can rewrite.

Local does not mean safe. A local model with administrative permissions can damage a business as efficiently as a remote one. What you gain is control over whether you keep operating, where your data and models live, and what evidence you hold.

Worth watching

What I am watching

The next durable advantage will not belong to the business that gives agents the most freedom. It will belong to the business that can increase autonomy without losing the ability to explain, stop, and recover the work.