Checking Code Against Acceptance Criteria in CI/CD with AI
Even strong teams occasionally ship work that misses a clearly specified requirement. Building an AI check into the CI/CD pipeline catches those obvious misses before work reaches QA, so testers spend their time on judgement, not on logging missing functionality.
- The check reads the acceptance criteria and evaluates whether the code meets them.
- It runs before the changes arrive in testing, catching obvious misses early.
- It is not designed to catch everything, and that is the point.
- QA then focuses on edge cases, user experience, performance, and interactions.
This is one practice within AI-augmented QA.
The cost of catching misses late
Even in high-performing teams, requirements occasionally get missed and obvious issues slip through. It is not a reflection of quality or care. It is the reality of complex software and tight deadlines.
QA picking up the occasional miss is a normal, expected part of the process. The problem is the cost of each one. When a clearly specified piece of functionality is missing, time gets spent on both sides: the tester raises a defect, the developer fixes it, and the work goes back round for retesting.
That is time spent on something a checklist could have caught, instead of on the testing that genuinely needs expertise. If we can catch the obvious misses earlier and automatically, we free that time up.
Where does the AI check sit in the pipeline?
The check runs in CI/CD (continuous integration and continuous delivery), after the code is committed and before the work reaches manual QA. It reads the acceptance criteria, evaluates the code against them, and routes the result.
What should the acceptance-criteria check actually do?
Keep its job narrow and clear: confirm that the functionality described in the acceptance criteria is present in the code. That is it.
In practice it is good at catching things like:
- A specified field, action, or state that is simply not implemented.
- A criterion the change appears to ignore entirely.
- Clearly defined behaviour that the code contradicts.
These are the cheap-to-catch, expensive-to-miss issues. Finding them in CI, in seconds, is far better than finding them in manual testing, hours or days later.
What should it not do?
It should not pretend to be a tester. The check catches obvious, specified misses. It does not catch subtle behavioural bugs, poor user experience, performance problems, or unintended interactions between features.
That narrowness is deliberate, not a shortcoming. Treating an acceptance-criteria check as full coverage would be the real risk. Its purpose is to clear the obvious so people can concentrate on everything else.
On gating: start advisory. Let the check warn and flag rather than hard-block the pipeline while you build confidence in its accuracy. As you learn where it is reliable, you can decide whether specific high-value checks should gate. Tighten deliberately rather than by default, and keep a human in the loop, the same principle behind why we do not let AI ship code unsupervised.
How does this change what arrives in QA?
When work reaches testing, the basic acceptance criteria have already been validated. The QA engineer is not writing defects about missing functionality that was clearly specified.
Instead, the time goes to the work that needs a person:
- Edge cases the criteria never anticipated.
- User experience and whether the feature actually feels right.
- Performance under realistic load.
- Interactions between features that only appear in combination.
That is the work that genuinely needs a QA engineer. Clearing the obvious misses earlier is what makes room for it.
For how this fits with test generation, AI bug reports, and release risk analysis, see the AI-augmented QA guide and Quality at the Speed of AI. To talk through your pipeline and QA process, get in touch with the Talk Think Do team.