What Assured AI Development Looks Like: Evidence from Our Own Delivery

Why does AI development need assurance?

AI tools can write most of the code in a project, quickly and plausibly. That makes the code cheaper to produce. It does not make it correct, secure or maintainable, and it does not tell you whether it does what was asked for.

The risk is easy to misjudge. Our own Inside the Harness article puts it this way: ‘the risk is not that the code is bad. It is that the code is good and answers a slightly different question than the one the work item posed.’

Assurance is the set of checks that catches both kinds of problem before the code reaches you.

What does ‘assured’ mean in practice?

We look for four properties. Each produces evidence a buyer can see.

• Reviewed. A person other than the author reviews every change before it merges, whoever or whatever wrote it.
• Gated. Automated checks run at several points, from every edit to every night, and a change cannot ship until it passes them.
• Traced. Every requirement is linked to a test that covers it, so the code is checked against what was asked for, not just against itself.
• Measured. Time, cost and quality are recorded, so claims about speed and quality can be checked.

What does assurance look like in our delivery?

We publish our delivery data quarterly in the AI Velocity Report. These are the figures and practices that make it assured, with links to where each is published.

How much is AI-written, and who reviews it?

In Q2 2026, 91.6% of our code is AI-authored, and 100% of it is reviewed by senior engineers and validated by ISTQB-qualified QA. On our own code, the reviewer is never the person who wrote the change. The Q2 2026 report explains why the share rose from a deliberate plateau of 83%.

Which checks does every change pass?

Our checks are tiered by cost. Cheap checks run constantly, and expensive ones run less often, after the cheap ones have filtered out the obvious problems:

1. Post-edit lint, on every edit the model makes
2. Pre-commit, with pattern review against our rules, lint and type checks
3. Continuous integration, on every pull request: the release build, unit tests and the acceptance criteria coverage check
4. Pre-release, with database integration tests and end-to-end suites
5. Nightly, the full regression suite

Inside the Harness describes each tier, and the architecture rules and reusable skills that shape the code before the gates check it.

How do we check the code does what was asked?

Every acceptance criterion is linked to a test that covers it, and each test is tagged with its work item. The continuous integration pipeline fails the build when an acceptance criterion has no covering test. Coverage of the requirements is a property of each work item, checked automatically, rather than a report assembled at the end of a sprint.

What did one change actually take?

In Anatomy of an AI Change, we measured one real change from first question to second merge. It took 15 hours 35 minutes elapsed, with Claude working for about three hours of that, and about $71 of AI usage. The unit test suite of about 2,400 tests ran twice with no failures, and around 160 end-to-end tests passed.

We estimated the people time for the same scope three ways:

All three are estimates against the scope that was delivered. The as-delivered figure includes a 4 to 6 hour manual session with a QA analyst and a business analyst, which no tool replaces. It is one well-specified change on a mature harness, not a benchmark, and it excludes what the harness cost us to build.

What does it mean for cost?

Our delivery is 40 to 50% faster, a figure we measure and publish ourselves. Recent proposals for large, business-critical, greenfield cloud-native Azure builds are landing at 28 to 34% of what the same work would have cost before AI, at the same quality or better.

What are the limits?

Assurance does not remove the need for judgement. Someone still has to design the rules, decide what good looks like, and own the work no template covers. As our harness article notes, traceability is only as good as the acceptance criteria: a green build means the code matches the specification, not that the specification was right. That is why business analysts and QA analysts stay in the loop.

What is not evidence?

Some claims sound like assurance but tell you little on their own:

• ‘We use AI.’ It says nothing about what happens to the AI’s output.
• A percentage of AI-written code, without a review rate alongside it
• Speed claims without a measured example and the assumptions behind it
• A demo of a tool generating code quickly
• A built-in scanner’s clean result, which shows that a rule exists, not that it is correct

None of these is a reason to rule a supplier out. Each is a reason to ask for the evidence behind it.

What evidence should you ask a supplier for?

Ask for artefacts, not descriptions. Redacted examples are fine. Here is what to request and what to look for in each:

Our guide to choosing a software development partner in the age of AI covers the wider evaluation. For contract terms, see our guide to who is accountable when AI writes the code.

How is this different from vibe coding?

The model can be the same. The difference is everything around it. Vibe coding accepts what runs, without review, gates or traceability. Our article on vibe coding versus harnessed delivery compares the two on a real change, and our guide to security checks for vibe-coded apps covers what to check when the assurance was skipped.

What if you build in-house?

The same four properties apply to your own team. Review by a non-author, tiered automated checks, requirement-to-test traceability and measurement do not depend on a supplier.

If your team uses AI tools and you cannot yet tell whether it is working, our AI Delivery Assessment benchmarks your delivery against our published Velocity Report data. It is £9,500, fixed, over three weeks, and ends with a prioritised 90-day roadmap. For systems already built without these checks, our Production Readiness Review gives you an evidence pack on the code as it stands.