What are the five trust pillars for AI generated code?

Provenance, consistency, enterprise safety, user safety, and debuggability. Provenance means the artifact carries evidence of what produced it and which checks it cleared. Consistency means the same intent yields the same class of result with no silent drift. Enterprise safety means policy, licensing, and audit are enforced by the system. User safety means correctness and privacy are gates rather than promises. Debuggability means a human can still read, trace, and own the code years later.

Why is debuggability a verification concern rather than a style concern?

Because code nobody can read is code nobody can verify. If an artifact cannot be traced and understood by a human, no one can be meaningfully accountable for it, and the evidence attached to it stops being checkable the moment the original author is unavailable.

Is a passing test suite enough evidence?

No. Tests establish that code matches its own specification. They say nothing about provenance, licensing, or whether the implementation still matches what the business asked for. An agent that writes an implementation can also write the tests that confirm it, which makes the suite a statement of intent rather than independent evidence.

What is the root cause behind all five pillars?

Intent was never captured in a system. Requirements live in one document, architecture lives in a senior engineer's head, and the implementation lives in an agent's context window. They diverge silently. Every pillar is a different person discovering that divergence at a different moment.

← All posts

What a factory must prove

4/7/2026 · 4 min · OpenFactoryAI · Read as Markdown

trust pillarsprovenanceauditmaintainabilityverification

TL;DR

Before you trust machine written software it has to prove five things: provenance, consistency, enterprise safety, user safety, and debuggability. Each is evidence attached to the artifact, not a claim made about the pipeline.

Trust is earned with evidence attached to the artifact, not asserted about the pipeline.
Consistency means the same intent produces the same class of result, with no silent drift.
The factory cannot ship what the enterprise never permitted.
End user safety is enforced upstream, not promised downstream.
Debuggability is a verification property: code nobody can read is code nobody can own.

A factory that ships code is common. A factory that can prove its code is the rare and valuable thing.

Proof is not a feeling and it is not a badge on a marketing page. It is a set of specific claims, each one attached to the artifact itself, each one checkable by someone who was not in the room when the code was written. There are five.

1. Provenance: where did this come from?

How do I know I can trust the output?

Every artifact carries evidence: what produced it, which checks it passed, and who or what is accountable. Not a log entry buried in a build system, but a record that travels with the change.

Trust is earned with proof, not asserted. The moment a pipeline says "our agents are safe" instead of "here is what this artifact cleared", you are being asked for faith.

2. Consistency: will it do this again?

How do I ensure consistency?

Same intent, same class of result. Two engineers pointing agents at the same requirement should not get contradictory implementations, and neither of them should find out in production.

This is the pillar teams underestimate. Non-determinism is fine at the token level and fatal at the decision level. What must be deterministic is the gate: which checks run, what counts as passing, and what happens when it does not. No silent drift between what was approved last month and what is approved today.

3. Enterprise safety: were we allowed to ship this?

How do I safeguard the enterprise?

Policy, security, licensing, audit trail, accountability. The factory cannot ship what the enterprise has not permitted.

An agent that pulls in a dependency with an incompatible licence, or that reaches a system it was never scoped to touch, has produced working code and an unacceptable liability. Working is not the bar. Permitted is the bar, and permission has to be enforced by the system rather than remembered by a person.

4. User safety: is the person on the other end protected?

How do I safeguard the end user?

Safety, correctness, privacy, recourse. Enforced upstream, not promised downstream.

This is the pillar that gets dropped, because the end user is not in the room and does not sign the contract. They are also the only party with no way to inspect what was built for them. If correctness and privacy are not gates in the pipeline, they are marketing copy.

5. Debuggability: can a human still own this?

How do I keep code debuggable and maintainable?

Output a human can read, trace, and own years later. No black boxes in production.

Debuggability sounds like a code quality concern. It is a verification property. Code that nobody can read is code nobody can verify, which means nobody can be accountable for it, which means the other four pillars quietly collapse the first time something breaks at 3am. Longevity is not a nice to have. It is what makes the proof survive the person who produced it.

Why five and not fifty

Because each one maps to a person who signs.

The engineer feels consistency, because their agent and a colleague's agent shipped contradictory changes. The finance lead feels the cost of rework, which is what inconsistency costs when you meter it. The compliance lead feels drift, because the code stopped matching what the business asked for and it surfaced in an audit rather than in development. Security feels enterprise safety. The person on call feels debuggability, every time.

Underneath all of it sits one root cause: the intent behind the code was never captured in a system. It lived in a senior engineer's head, a stale document, and a comment from 2014 that says do not touch this. AI does not fix tribal knowledge. It amplifies it, at speed, in every repository at once.

The pillars are a bar, not a product

These five are not features. They are the contract that any machine on the floor has to honour before its output is trusted, and they are the bar the certification ladder is built to enforce: certify the engineer, then the agent, then the code.

We have not finished that ladder. The rung that runs today certifies the people who direct the agents, because you cannot hold a machine to a standard you have not yet written down.

If a verification tool oversells, you should not trust it. So we do not.

What are the five trust pillars for AI generated code?: Provenance, consistency, enterprise safety, user safety, and debuggability. Provenance means the artifact carries evidence of what produced it and which checks it cleared. Consistency means the same intent yields the same class of result with no silent drift. Enterprise safety means policy, licensing, and audit are enforced by the system. User safety means correctness and privacy are gates rather than promises. Debuggability means a human can still read, trace, and own the code years later.
Why is debuggability a verification concern rather than a style concern?: Because code nobody can read is code nobody can verify. If an artifact cannot be traced and understood by a human, no one can be meaningfully accountable for it, and the evidence attached to it stops being checkable the moment the original author is unavailable.
Is a passing test suite enough evidence?: No. Tests establish that code matches its own specification. They say nothing about provenance, licensing, or whether the implementation still matches what the business asked for. An agent that writes an implementation can also write the tests that confirm it, which makes the suite a statement of intent rather than independent evidence.
What is the root cause behind all five pillars?: Intent was never captured in a system. Requirements live in one document, architecture lives in a senior engineer's head, and the implementation lives in an agent's context window. They diverge silently. Every pillar is a different person discovering that divergence at a different moment.