Does Draft Annex 11 Prohibit Generative AI?

The question is increasingly appearing in quality and digital transformation discussions:

Does the draft Annex 11 say that a GMP organisation cannot use generative AI?

The short answer is no blanket prohibition is identified in the draft text. That does not mean every use of generative AI is acceptable, low risk, or automatically compliant. It means the regulatory question is more specific:

What is the organisation relying on the AI system to do?

That question leads to the matters that Annex 11 is concerned with: intended use, risk, data integrity, validation or verification, security, supplier oversight, controlled change, and accountable human review.

This article provides a practical reading for GMP teams. It is not legal advice, a validation protocol, or a customer-specific regulatory determination. The draft consultation text and the organisation's own quality-system context remain the controlling references.

Start with intended use

Generative AI can be used for very different purposes. A person may use M365 Copilot to improve grammar in a draft deviation report. Another team may use an AI system to summarise evidence, suggest a root cause, recommend a CAPA, or classify product impact. These uses may look similar from a technology perspective, but they do not have the same quality or data-integrity implications.

The first step is therefore to describe the intended use precisely:

• What task is the AI assisting with?
• What information does it receive?
• What output does it produce?
• Who reviews that output?
• What decision or process step follows?
• Can the output become part of a controlled record or approval?

A general statement such as "we use Copilot for documentation" is not enough to define the control boundary. The organisation needs to explain what the tool is allowed to do and what remains outside its role.

Support function or GMP decision-making?

A useful first distinction is between an AI support function and AI being relied on for GMP decision-making.

AI as writing support

Examples may include:

• improving grammar and readability;
• helping structure a draft;
• suggesting consistent terminology;
• identifying missing sections for an author to check;
• helping a person prepare a document for review.

These uses can still create risks. AI-generated text may introduce an incorrect statement, change the meaning of an investigation, omit important context, or expose information to an unauthorised service. But the AI remains a support tool if the responsible author checks the output and remains accountable for the final document.

AI influencing a GMP decision

Greater control is needed when an organisation relies on AI to:

• assess product impact;
• determine or recommend a root cause;
• classify a deviation;
• recommend or assess CAPA effectiveness;
• interpret evidence for a quality decision;
• recommend batch disposition or release;
• approve an investigation or other controlled action.

The issue is not simply whether the model is described as "assistive". The issue is what influence the output has on the process and whether a qualified person can understand, challenge, accept, reject, or escalate it.

Human review is necessary, but it is not the whole answer

Human review is a central control for AI-assisted GMP work, but "a human looked at it" is not a complete governance model.

A meaningful review process should define:

• who is authorised to review the output;
• what the reviewer is expected to check;
• which source evidence must be considered;
• what acceptance criteria apply;
• how the reviewer records accept, reject, rework, or escalate;
• how amendments and reasons are captured;
• what happens when the output is uncertain, incomplete, or wrong.

The original AI output and the human disposition may need to remain understandable and retrievable, depending on the process impact and the applicable quality-system requirements. Oversight is a workflow, not merely a signature at the end of a document.

Our guide on human oversight patterns for GMP AI workflows explores this distinction in more detail.

What controls should a GMP team consider?

A proportionate assessment of a generative AI use case should consider at least the following areas.

Risk and process impact

Assess the possible effect on product quality, patient safety, data integrity, and the decisions or records involved. Revisit the assessment when the intended use, data, workflow, or system changes.

Data integrity and traceability

Consider whether the organisation can explain what information was provided, what the AI produced, what the reviewer changed, who made the decision, and which relevant versions or configurations were involved. The appropriate evidence chain depends on the use case and process impact.

See evidence and traceability for GMP AI workflows for the practical questions this raises.

Security and information handling

The team should understand what information is sent to the service, how access is controlled, how prompts and outputs are handled, and whether the use is consistent with the organisation's data-governance requirements. These questions are especially important when prompts contain confidential product, patient, batch, investigation, or supplier information.

Supplier and service oversight

Where a cloud or SaaS service is used in the workflow, the organisation remains responsible for assessing whether the service is suitable for the intended use and for retaining the evidence needed to support that assessment. Using a well-known supplier does not remove the need to understand the service boundary and relevant changes.

Validation, evaluation, and verification

The level of validation or verification should follow the intended use and risk. A narrowly bounded writing-support use may require a different evidence approach from an AI output that influences a quality decision or enters a controlled record. The organisation should define how it will evaluate the use before release and how it will monitor effectiveness after release.

Read evaluating GMP AI before and after release for more on that lifecycle view.

Controlled change

AI services, models, prompts, retrieval sources, configurations, permissions, and surrounding workflows can change. A GMP team needs a way to assess whether a change affects the approved intended use, the output, the review process, or the evidence chain.

The controlled change guide for GMP AI workflows provides a starting point for that assessment.

What the answer does not mean

Saying that the draft Annex 11 does not create a blanket prohibition does not mean:

• generative AI can be used without a defined purpose;
• human review can be informal or bypassed;
• an organisation can put confidential GMP information into any public AI service;
• every AI-assisted document requires the same controls;
• M365 Copilot or another vendor is automatically validated for the customer's use;
• an AI system can make a regulated decision simply because a person remains nominally responsible;
• the use is compliant without a documented assessment and appropriate evidence.

The technology may be available, but the operating boundary still needs to be designed and maintained.

A practical starting point

For a proposed generative AI use case, a GMP team can begin with five questions:

1. What exactly is the AI allowed to do?
2. What must it never be relied on to do?
3. Where does its output go next?
4. Who must review and decide before the process progresses?
5. What evidence will show what happened and what changed?

If those questions cannot be answered clearly, the use case is not ready to move from interest to controlled adoption. Start by defining the process problem and the output boundary, then decide what governance and evaluation are proportionate.

The Practical, Governed AI for GMP Quality Operations hub brings those questions together, including use-case selection, output boundaries, human oversight, evidence, evaluation, and controlled change.

A more useful question than "Can we use AI?"

The most useful question is not whether generative AI is categorically allowed or forbidden. It is whether the organisation can define, control, review, evidence, and improve the particular use it is considering.

That is where a regulatory reading becomes an operating decision. Generative AI may support some GMP work, but its role should remain bounded by intended use, process impact, accountable human judgement, and evidence that can be understood when the organisation needs to explain what happened.

Discuss your GMP AI pathway.