Skip to content
Skip to content

Practical AI project guide · By Yes AI

What controls should an AI agent have before it acts?

Published 6 min read

An AI agent reads information and takes actions in other systems, such as updating records, sending messages or raising bills. That changes which controls matter, because a wrong answer becomes a wrong action. This guide sets out the controls to agree before any agent goes live, drawing on guidance from the Australian Signals Directorate and the OWASP Gen AI Security Project.

Quick answer

Before an AI agent acts, limit it to a low-risk, well-defined task; give it its own account with only the permissions that task needs, for a limited time; require a person to approve high-impact or irreversible actions such as payments, external messages and changes to bank details; log every action and approval; and make sure a named person can pause, override, roll back or shut it down, with a manual process ready if it stops.

Accounts officer in a Newcastle office checking a stack of supplier invoices against a blurred monitor
In a first agent project, people keep approval of anything involving money.

How is an agent different from a chatbot?

A chatbot produces text for a person to use. An agent is given tools, such as the ability to read an inbox, query a database, update an accounting system or send an email, and it decides which tool to use to reach a goal. The Australian Signals Directorate's guidance, co-authored with partner agencies in the United States, Canada, New Zealand and the United Kingdom and published on 1 May 2026, describes agentic systems as intended to operate without continuous human intervention. The OWASP Gen AI Security Project calls the resulting risk excessive agency: damaging actions taken because of unexpected, ambiguous or manipulated model output, whatever caused the model to go wrong.

Which tasks should an agent start with?

The ASD guidance recommends limiting initial use to low-risk, well-defined tasks, and expanding autonomy, privileges and scope only after that. It also suggests considering the full range of fixes for repetitive work, including reducing or removing low-value processes, which may carry less risk than an agent. Good first tasks have a clear input, a small set of permitted actions and an easy way to check the result. Collecting invoice details into draft bills for review fits that description, while approving payments does not.

What should the agent be allowed to access?

Apply least privilege. The ASD guidance recommends assigning only the minimum access each role needs, limiting entitlements to the exact resources, operations and timeframes required, and replacing long-lived secrets with credentials that expire when the job is complete. OWASP adds the practical rules listed below. In plain terms, an agent that summarises email should not be able to send or delete it, and an agent that reads a product table should not be able to change it.

  • Offer the agent only the tools it needs, and remove tools left over from development.
  • Give each tool only the functions it needs, and avoid open-ended tools such as running any command or fetching any web address.
  • Run actions with the permissions of the user it acts for, or a dedicated limited account, and never a shared administrator login.
  • Enforce permissions in the downstream system instead of trusting the model to decide what is allowed.

Which actions need a person to approve?

OWASP recommends human approval of high-impact actions before they are taken, and the ASD guidance includes mandatory human approval for decision-making steps among its human control points. Decide the list before launch, build it into the agent's configuration and test that the agent cannot bypass it. A useful starting rule is that anything irreversible, anything sent outside the business and anything involving money or a decision about a person needs approval until evidence supports relaxing it.

  • Payments, refunds and changes to bank or payment details.
  • Messages sent to customers, suppliers or the public.
  • Deleting or overwriting records.
  • Decisions about a person, such as eligibility, pricing or employment.

How do you stop or roll back an agent?

The National AI Centre's foundations guidance asks for clear intervention points where people can pause, override, roll back or shut down an AI system, and for alternative pathways so critical functions continue if the system malfunctions or is retired. The ASD guidance recommends fail-safe defaults, where the agent stops and escalates to a person when it is uncertain, and versioning so you can return to known-good behaviour. Before launch, rehearse a stop: pause the agent, find the work it left part-done and finish that work by hand.

What should be logged and reviewed?

Log every action the agent takes, the input that triggered it, the tool used, the result and any human approval or rejection. The ASD guidance recommends comprehensive logging by default, unified audit logs for interactions between agents and priority review of incidents where guardrails were triggered or a reviewer denied an action. OWASP notes that logging and rate limits reduce the damage from excessive agency by making problems visible sooner, though they do not prevent it. Assign someone to read the exceptions every week, as well as after anything breaks.

When can the agent be given more autonomy?

Expand only on evidence. Keep a record of the agent's actions and approvals for an agreed period, then look at how often reviewers changed or rejected its proposals and why. If a category of action has been approved unchanged many times and errors are easy to reverse, it may be reasonable to let the agent do it without approval, with spot-checks. The ASD guidance describes continuous evaluation to decide when to expand scope or roll back autonomy and access. Record each decision, and who made it, in the AI register.

Hypothetical example: permission plan for an accounts payable agent

ActionAgent allowed?ControlWho approves
Read supplier invoices in the accounts inboxYesRead-only access to one mailboxNot needed
Create draft bills in the accounting systemYesDraft status only, with no approval rightsAccounts officer reviews each draft
Match a bill to a purchase orderYesFlags mismatches instead of resolving themAccounts officer for flagged items
Email a supplier about a missing detailDraft onlyA person sends it from the shared mailboxAccounts officer
Change supplier bank detailsNoPermission not grantedFinance manager, after verifying by phone outside the agent
Approve or schedule a paymentNoPermission not grantedFinance manager in the accounting system

Practical checklist

  • The agent's task, permitted actions and out-of-scope actions are written down.
  • The agent has its own account with only the permissions the task needs, and credentials that expire.
  • Actions that need human approval are configured and tested so the agent cannot skip them.
  • Every action, input, tool call and approval is logged, and someone reviews the exceptions.
  • A named person can pause, override, roll back or shut down the agent, and has rehearsed it.
  • A manual process is ready for the agent's work if it is stopped.
  • The agent is recorded in the AI register with an accountable owner and a review date.

Common questions

Can we let an agent send emails to customers?

Start with drafts that a person sends. External messages are hard to recall and carry your business's name, so they belong on the approval list until you have evidence. Once reviewers have approved a category of message unchanged many times, you might allow automatic sending for that category alone, with sampling and a quick way to stop it.

What is prompt injection, and why does it matter for agents?

Prompt injection is text crafted to make a model follow instructions it should not, and it can arrive hidden in an email, document or web page the agent reads. OWASP lists direct and indirect prompt injection among the triggers for excessive agency, and the ASD recommends input checks and filters. Tight permissions and human approval limit what a successful injection can do.

Who is accountable when an agent gets something wrong?

Your business is. The National AI Centre's guidance says the organisation is ultimately accountable for how and where AI is used, and asks for a named accountable person for each AI system. Agree with the builder and any platform vendor who monitors, fixes and reports faults, but the decision to let the agent act sits with the business.

Do agents need their own accounts?

Give each agent its own account. A dedicated account, separate from any person's login, lets you limit permissions to the task, see the agent's actions in system logs and switch it off without locking a staff member out. The ASD guidance goes further and recommends treating each agent as a distinct identity with its own credentials.

Do these controls apply to agents built into Microsoft 365 or our CRM?

Yes. The same questions apply whether you build an agent or switch one on inside software you already use: what it can read, what it can change, what needs approval and who can stop it. Check the vendor's admin settings for each of these and record the answers in your AI register.

Sources and further reading

Continue planning

Turn the project decision into an agreed next step

Describe the business process, project stage and decision you need to make. We can discuss scope, evidence and who should own the next step. Keep confidential records, credentials and sensitive commercial details out of this enquiry.