What this looks like in practice
The problem
A customer-support agent that can read account data, issue refunds, or escalate a case is processing personal data and taking actions with real consequences for customers. Under the EU AI Act, a support agent that talks to customers falls under Article 50(1): it must be designed so people are told they are interacting with an AI system, unless that is obvious from the context. Whether it is also high-risk depends on its intended purpose, not on how consequential its actions are. Refunds and account changes are not, on their own, an Annex III use case. A support deployment comes within Annex III when it is intended for a listed purpose — for example evaluating a person's creditworthiness or setting their credit score (Annex III, point 5(b)), or risk assessment and pricing for life or health insurance (point 5(c)) — and even then Article 6(3) sets out when a listed system is not high-risk, except where it profiles people. Where Annex III does apply, the obligations apply from 2 December 2027. Most support-agent deployments were shipped to cut response time, not to produce the audit evidence a regulator or an enterprise customer's procurement team will eventually ask for, and that evidence is easier to collect from the start than to reconstruct later.
That gap shows up at the worst possible moment: during a security questionnaire, a compliance audit, or a regulator inquiry, when the team is asked to show what the agent saw, what it decided, and who approved a specific refund or account change — and the honest answer is that no one kept that record because the agent was built to move fast, not to be reviewed.
What good looks like
A well-governed support agent produces the audit evidence for free, as a byproduct of how it runs, rather than as a project bolted on after a customer or regulator asks for it. Personal data the agent reasons over — account numbers, names, payment details — is redacted or minimized before it reaches the model in both directions, so the agent has what it needs to help the customer without needlessly exposing raw personal data at every turn of the conversation. The decisions that govern the agent are recorded in an append-only, hash-chained, signed trail, so a later edit to a record is detectable, rather than in a log an admin could quietly change after the fact.
Consequential actions — a refund, a plan change, an account closure — go through a human approval gate rather than executing autonomously, at least until the organization has evidence the agent's decisions in that category are reliable enough to trust unsupervised. Policy controls apply per connection: which task types, tools and models it may carry, how many tasks per minute or hour it may run and in which hours, and how much it may spend in a month, so a single misconfiguration doesn't become an organization-wide exposure. Where a customer's data is processed is not a per-connection rule: each organization is assigned one residency region and its requests are served only by that region's deployment. None of this should require rebuilding the support agent from scratch — it's a governance layer the agent's existing logic runs behind, not a redesign of the agent itself.
How Praesidia helps
Praesidia applies PII redaction in both directions of a support conversation, so sensitive data is minimized before the agent's model sees it and before its response reaches the customer, without the underlying account record losing its data. Per-connection policy controls set how fast, when, and with which tools a given agent connection can operate, so a support agent's reach stays scoped to what the deployment actually intends.
Praesidia records the decisions it makes in an append-only, hash-chained audit trail, signed under the default configuration; an operator can turn signing off. That gives compliance and security teams a record of what was decided in a given conversation without reconstructing it after the fact. A signed record shows it was not changed afterwards. It does not show that every action was captured, and calls that do not pass through Praesidia are not recorded. Governance reporting maps that activity to the compliance frameworks a support-agent deployment is likely to be measured against, including the EU AI Act, so the evidence a regulator or an enterprise buyer's security team asks for already exists in a usable form. Human approval gates sit in front of consequential actions like refunds, so a support agent's judgment stays reviewable exactly at the point where a mistake would have real financial or account-level consequences. The AI governance guide covers how these controls fit into a broader compliance program beyond the support-agent use case specifically.
Getting started
- Inventory what the support agent can currently reach — which account data fields, which actions (refunds, escalations, account changes), and under what conditions it currently acts without a human step.
- Classify actions by consequence, separating "answers a question" from "changes something" — the second category is where audit evidence and approval gates matter most.
- Turn on PII redaction at the model boundary so the agent reasons over minimized data by default, before tackling any other control.
- Add a human approval gate on the highest-consequence action types first — typically refunds and account changes — rather than trying to gate everything at once.
- Confirm the audit trail actually answers the question a regulator or auditor would ask: what did the agent see, what did it decide, and who approved the resulting action, for any conversation on demand.
FAQ
Is a customer-support agent high-risk under the EU AI Act? Not because it issues refunds or changes accounts. High-risk status depends on the system's intended purpose: it applies when the agent is a safety component of a product covered by Annex I, or is intended for a use listed in Annex III, such as evaluating creditworthiness (point 5(b)) or life and health insurance risk assessment and pricing (point 5(c)), subject to the Article 6(3) exceptions. Answering questions and processing refunds are not themselves listed uses, but a support agent must still tell people they are talking to an AI system (Article 50). Record the classification and its reasoning per agent, and have counsel confirm it.
What evidence should we keep for each support conversation? A tamper-evident record of what the agent read, what it decided, what action it took (if any), and whether a human approved that action — the same evidence a regulator or enterprise auditor will ask for.
How do we redact personal data without breaking the agent's usefulness? Redact at the boundary the agent's model sees, not in the underlying account record, so the agent reasons over minimized data while the record it acts on stays intact.
Governing customer-support agents covers the broader control set for this category of deployment, the EU AI Act explained for engineers walks through the regulation's risk tiers in practitioner terms, and how to redact PII from agent prompts and tamper-evident audit logs go deeper on the two capabilities this use case depends on most.