The Risks of AI-Augmented Development

This guide is the governance spoke of the AI-augmented development pillar guide. It is written for CISOs, general counsel, COOs, and procurement leads who need to write a risk register entry, not for engineers who need to choose a tool. The engineering view sits in the AI-augmented software development lifecycle guide.

What are the genuine risks of AI-augmented development?

Eight risks come up consistently in the engagements we see. They are not the only risks. They are the ones that show up in every contract review, every ISO audit, and every procurement panel.

1. IP and code ownership.
2. Attribution and audit of AI contribution.
3. Regulatory exposure (EU AI Act, sector regulators, data protection).
4. ISO 27001 controls for AI tools as cloud services.
5. Data residency and prompt leakage.
6. Quality drift when review discipline slips.
7. Skills decay in junior engineers.
8. Contractual ambiguity in supplier engagements.

Each has a section below with the underlying risk, how it shows up, and the control that contains it.

Who owns the code and the IP?

The risk is that the supplier retains IP, grants a licence, or carves out AI-generated code as a separate class. The control is the contract.

Under English law (CDPA 1988 s.9(3)), computer-generated works are authored by “the person by whom the arrangements necessary for the creation of the work are undertaken”. In practice this means the engineer who directed the AI to produce the code, not the AI vendor. The default position should be that the client owns the code, the IP, and the repository from day one, with AI-generated code treated identically to human-written code. Talk Think Do contracts state this explicitly; some suppliers do not. Confirm before signing.

A separate concern is whether the AI tool itself can train on your code. Most commercial offerings (Claude for Work, Cursor Business, GitHub Copilot Business, Azure OpenAI) contractually do not train on customer prompts or completions. Free and consumer tiers often do. The supplier should disclose which tools are used and on which licence tier, and the answer should not include consumer accounts on client code.

How is AI-generated code attributed and audited?

The risk is that AI contribution becomes invisible after the fact, so an auditor or a buyer cannot reconstruct what was AI-authored and what was reviewed by whom.

The control is metadata, not vibes. Every commit and pull request should record which AI tool produced the change, who reviewed it, and which gates passed. Pre-commit and PostToolUse hooks in Cursor and Claude Code can attach this metadata automatically. CI status checks can refuse to merge if the metadata is missing.

The full picture sits in the AI Code Attribution and Procurement guide, including the CycloneDX and SPDX 3.0 SBOM extensions that increasingly carry AI provenance.

When a buyer asks for an attribution evidence pack, the supplier should be able to produce one. If the answer is “we don’t track that”, the audit posture is incomplete.

What does ISO 27001 require of an AI-augmented team?

The risk is that AI tools sit outside the Information Security Management System (ISMS) and create unmanaged data flows.

ISO 27001:2022 does not prohibit AI tools. It requires them to be inside the ISMS. The relevant controls are:

• A.5.23 (information security for use of cloud services). AI providers are cloud services for this purpose. Document them, assess them, and treat them like any other supplier.
• A.5.19 (information security in supplier relationships). AI vendors go in the supplier register.
• A.8.28 (secure coding). AI-augmented coding is still coding. Standards still apply.
• A.5.10 (acceptable use of information and assets). The data-in-prompt policy lives here.
• A.8.11 (data masking). Customer data and PII should not appear in prompts. Masking and synthetic test data are the practical controls.

Our ISO 27001 and AI development tools blog post gives a ten-point checklist for bringing AI usage into compliance. Failing to disclose AI tool usage to an auditor is increasingly cited as a non-conformance; doing the work in advance is cheaper than fixing it during certification.

Where does the EU AI Act apply?

The risk is that a UK business assumes Brexit creates an exemption. It does not. The EU AI Act (Regulation EU 2024/1689) has extraterritorial reach under Article 2.

The Act applies to any AI system placed on the EU market, put into service within the EU, or deploying outputs used by people in the EU. Most AI-augmented development work is not high-risk because the act of writing code with AI tools is not in itself a regulated AI system. The Act regulates the resulting software, not the development practice.

The shape of the risk depends on what you build:

• High-risk systems (Annex III). Employment decisions, creditworthiness assessment, educational scoring, access to essential services, law enforcement, migration. Full obligations enforceable from 2 August 2026.
• Limited risk systems. Chatbots, deepfakes, emotion recognition. Transparency obligations only.
• Minimal risk systems. Most internal productivity, content generation, code assistance. Voluntary codes of practice.
• Prohibited practices (Article 5). Social scoring, manipulative AI, certain biometric uses. Banned outright.

The provider versus deployer distinction in Article 25 determines who carries which obligations. A UK business commissioning custom AI-powered software may become the provider, especially if the system is placed on the market under its name or brand. The EU AI Act guide covers this in detail, including what to put in the supplier contract to allocate responsibilities cleanly.

SME concessions exist. Reduced fees for conformity assessments, regulatory sandboxes, and simplified documentation requirements are all available. Concessions are not exemptions.

How does quality drift show up and how is it caught?

The risk is that AI output is accepted with progressively less scrutiny. The first month, every change is reviewed line by line. By month six, the team is rubber-stamping. By month twelve, the codebase has the architectural coherence of a kit-built cupboard.

Quality drift is the most preventable risk on the list. The control is gates that AI cannot satisfy on its own.

• Senior engineer review on every change. Not a sample. Not a quota. Every change.
• ISTQB-qualified QA on every release. Test authoring is AI-augmented; test approval is not.
• Static analysis and security scanning as CI status checks. Required to merge.
• Acceptance-criteria checks tied to user stories. MCP-driven validation, not a human checking after the fact.
• Architecture review on changes that touch architecture. Triggered by a list of paths in the repository, not by reviewer judgement.

Talk Think Do’s Q1 2026 data shows that AI-augmented changes through this gate produce fewer defects than manually written code in the same systems. That is the headline outcome of getting the gates right. Teams that skip the gates produce the opposite outcome, faster.

What is the skills risk for junior engineers?

The risk is that junior engineers become AI-output operators rather than engineers. They accept changes they cannot explain, fail to debug their own systems, and lose the ability to make independent judgement calls.

The control is the same as it has always been: deliberate practice. Junior engineers need structured time writing code without AI assistance, paired sessions with a senior engineer, code walkthroughs, and feedback on the reasoning behind their choices. AI tools should be a force multiplier on the learning curve, not a substitute for one.

This is a real risk and an emerging one. Teams that pretend it does not exist tend to produce a generation of engineers who are productive on familiar tasks and helpless on unfamiliar ones. Teams that take it seriously build the discipline early and end up with stronger engineers, faster.

How do you handle prompt leakage and data residency?

The risk is that customer data, PII, secrets, or commercial information ends up in an AI provider’s training corpus or in a log that the provider retains.

Three controls in combination handle this risk:

• A data-in-prompt policy. Define what data classes can appear in prompts. Customer data and PII generally cannot. Internal source code and architectural information generally can, on a commercial licence.
• Tooling that masks or rejects. IDE plugins and pre-commit hooks can scan prompts for known data patterns (credit card numbers, NHS numbers, sample customer records) and refuse to send. This is the difference between a policy and an enforced policy.
• Provider selection. Use providers with contractual no-train commitments, EU or UK residency where required, and the right tenant model. Azure OpenAI and the commercial Anthropic, Cursor, and GitHub Copilot enterprise tiers all qualify on the no-train condition.

For sensitive workloads, self-hosted models (running on your own Azure tenant via Azure AI Foundry, or on dedicated infrastructure) remove the residency risk altogether. The trade is model capability versus residency posture. Most current self-hosted models are good enough for code work; they are not always good enough for deep reasoning tasks where a frontier model would otherwise be used.

What controls should appear in a contract?

The risk is that the contract is silent on AI, the supplier has an AI practice the buyer cannot see, and the obligations fall on whoever is unlucky when a problem arises.

A workable contract clause set covers:

1. Code and IP ownership from day one, with AI-generated code treated identically to human-written code.
2. EU AI Act allocation. Explicit statement of who is provider, who is deployer, and for which AI components.
3. AI tool disclosure. Which tools, which licence tier, which data classes pass through them.
4. Data residency and processing. Where prompts and completions are processed, who retains them, and for how long.
5. Attribution and audit. Standing obligation to record AI contribution and produce evidence on request.
6. Liability. No softening of liability because the code is AI-augmented. The supplier is responsible for what they deliver, full stop.
7. Training data. Explicit statement that client code, prompts, and completions do not train any public model.
8. Termination and exit. Standard exit clauses, plus an obligation to provide the full attribution and metadata corpus on exit.

A supplier who asks for a softer position on any of these because the code is AI-augmented is asking the buyer to carry a risk that the supplier should be carrying. That is a procurement warning sign.

How does this map to existing certifications?

Most clients ask whether their supplier should hold AI-specific certifications. The honest answer in 2026 is that the existing certifications still cover most of the ground.

• ISO 27001:2022. Covers AI tools as cloud services and as part of secure development. Sufficient for most engagements.
• Cyber Essentials Plus. Covers the endpoint and network controls around AI tool usage.
• ISO 42001. The AI-specific management system standard. Useful but not yet a procurement default in UK mid-market.
• G-Cloud Lot 3. Public-sector framework. Includes AI services. Covered by Crown Commercial Service supplier listings.

Talk Think Do holds ISO 27001 and Cyber Essentials Plus, is a Microsoft Solutions Partner across Azure Infrastructure, DevOps and GitHub, and Digital and App Innovation, and is on the Crown Commercial Service G-Cloud framework. The procurement question is rarely about new certifications; it is about whether existing ones are applied properly to AI tools.

Where to start

Three steps, in order:

1. Run an AI tool inventory. Every tool, every licence tier, every data class. Most teams find tools they did not know they had.
2. Add the AI tools to your ISMS. Treat them as cloud services and apply your standard controls. The ISO 27001 blog post linked above gives a checklist.
3. Update your supplier contracts. Add the AI clauses above to your standard procurement template. They are easier to introduce as a default than to negotiate one engagement at a time.

For the wider picture, the AI-augmented development pillar guide sets out the practice the controls are protecting. The AI Readiness Checklist covers the broader organisational preparation.

Book a consultation if you would like a review of an existing supplier’s AI posture, or if you are scoping a new engagement and want to align the contract clauses to your risk appetite. Our AI Development and Implementation service covers the delivery side.