AI Agent Governance Scenarios, Part 1: The New "Game Changer" Agent

This is the first in a five-part series of practical AI-governance scenarios for quality professionals working through what "good" looks like when an AI agent proposal lands on their desk.

The scenario

The production team has developed a new AI agent that automatically reviews batch manufacturing records to identify potential GMP deviations. They want to deploy it within the next month and believe it will significantly improve productivity and reduce the review backlog. They tell QA:

the agent was built using Microsoft Copilot Studio;
it reviews PDF exports of batch records;
it was tested on ten historical records and "worked well";
the prompts include reference SOPs to help guide the AI;
QA will still make the final decision, so they believe validation can be minimal.

They ask QA to approve rollout. Before approving, what would you ask about inputs, governance, validation, human oversight, documentation, and risk management?

Issue 1 — Uncontrolled SOP versions provided to the agent

If the prompt references a draft, superseded, or otherwise uncontrolled training document, the agent cannot determine document validity, and an incorrect SOP leads to incorrect reasoning. Controlled document management has to remain inside the quality management system regardless of what generates the prompt.

Issue 2 — Batch records provided as PDF exports

An exported PDF is not the system of record, and metadata can be lost in export. Ask whether the exports are complete, whether the export process is itself validated, and whether records can be altered after export — traceability requires knowing exactly what data was analysed.

Issue 3 — Insufficient testing

"We tried it on ten records" is not an evaluation method. Ask what the evaluation method was, whether known deviations were deliberately included in the test set, and what the miss rate was. AI-assisted processes must be risk-assessed and justified, not validated by anecdote.

Issue 4 — The agent summarises compliance issues

Summarising compliance issues edges into interpreting GMP compliance and inferring regulatory conclusions. AI is well suited to pattern recognition, comparison, and anomaly detection; it is not appropriate for making compliance determinations. Scope has to be defined precisely enough to keep the agent on the right side of that line.

Issue 5 — Production built the agent

A tool developed by the operations team and brought to QA for approval after the fact inverts the correct order. Every agent needs an owner, defined responsibilities, and controlled deployment established before operational use — QA governance has to exist before use, not be retrofitted onto it.

Issue 6 — SOP updates could break the agent

If prompt logic references SOP text directly, an SOP revision without a corresponding agent update creates silent drift. Prompt logic tied to controlled documents must be subject to the same change control as the documents themselves.

Issue 7 — Ambiguous human review

"QA will still make the final decision" is a common but vague reassurance. Verification needs to be explicit, documented, and meaningful — humans reviewing the underlying reasoning, not just the summary presented to them.

Issue 8 — No clear definition of agent scope

Is the agent checking calculations? Detecting missing data? Interpreting GMP compliance? Without an explicit answer, the agent has no defined tasks, boundaries, or known limitations — and no governance control can be properly assessed against an undefined scope.

The takeaway

None of these eight issues require deep AI expertise to identify — they require the same instinct a quality professional already applies to any new process: define scope, control inputs, verify outputs, and document the decision trail before granting operational trust.