Copilot or autopilot: how much control should an AI agent have?

In 60 seconds: Autonomy is not a general property of an agent. Decide it action by action. Use five levels: read, recommend, draft, approved execution, and bounded autonomy. Score error impact, reversibility, sensitivity, frequency, and policy clarity. Begin with the lowest level that creates value and move up only with evidence. Human approval is insufficient when nobody can establish what changed, stop the workflow, or restore the prior state. Payments, commercial terms, and hard-to-reverse commitments should retain separate controls.

An agent may summarize overdue invoices safely enough, then create a serious problem if the same credentials let it post payments. Calling it “supervised” does not explain what it can do, who decides, or what happens after an error.

Design around the individual action. For each one, define what the agent reads, which conclusion it may propose, and which system it may change. That distinction separates a useful copilot from an autopilot with excessive access.

Suggestions, decisions, and execution are different acts

A suggestion presents options or a priority. For example: “Contact these three accounts first.” It does not create an obligation.

A decision selects an option under a policy and assumes its consequences. A person, a deterministic rule, or a tightly bounded prior authorization may make it. The model should not hide this step inside persuasive prose.

Execution changes something outside the chat: it sends an email, updates CRM, confirms an order, issues an invoice, or initiates a payment instruction. It needs controls that a suggestion does not. The interface should name these three stages clearly and preserve a separate record for each.

The five autonomy levels

Level 1: read

The agent queries authorized sources and responds without changing records. It can gather opportunities with no recent activity, blocked orders, overdue invoices, or figures for a report. Every result should show its source and cutoff time.

Read access is still limited access. An executive assistant needs company, region, field, and role filters. Our guide to the ERP, sales, and collections assistant develops that architecture.

Level 2: recommend

The agent reads information, ranks options, and proposes the next step. It may flag an order for review, explain a report variance, or suggest which opportunity needs a follow-up. Label the recommendation as an inference and link it to the facts behind it.

A person or an external rule retains the decision at this level. Measure how often recommendations are accepted, corrected, or rejected. Pay close attention to important cases that never appear.

Level 3: draft

The agent prepares an artifact with no operational effect yet: an email in the drafts folder, a proposed CRM update, an unconfirmed order, a preliminary invoice, or an unsigned payment batch.

Keep the draft separate from the official record. The reviewer needs the source data, fields that would change, and failed validations. Copying text into an approval screen without showing the full effect produces shallow review.

Level 4: approved execution

The agent executes after an authorized person approves a specific action. The request must fix the recipient, amount, currency, affected records, content, and authorization expiry. A change to any of them invalidates the approval.

This level can work for sending a reviewed follow-up, updating a CRM stage, or confirming an order that passed stock, price, and customer checks. A generic approval such as “process pending items” should not authorize a list that can change while someone reviews it.

Level 5: bounded autonomy

The agent may execute without case-by-case approval inside a predefined envelope: allowed actions, amount thresholds, eligible recipients, schedule, volume, systems, and stop events. Everything outside that envelope goes to a queue with a named owner.

One reasonable example is filling a low-sensitivity internal field from a verified source with change history. Autonomous payment execution should not be the default recommendation for this level. An agent may detect duplicates and prepare the batch while bank authorization and segregation of duties stay outside its control.

A matrix for choosing the level

Score each factor from 1 to 5. A 1 means low risk or difficulty and a 5 means high. For policy clarity, 1 means a precise rule and 5 means ambiguous judgment. The score supports the process owner’s judgment by forcing the team to explain why an action receives more autonomy.

Factor	Question for the team	Signal to limit autonomy
Error impact	What happens if the agent acts on the wrong case?	It affects cash, a customer, compliance, or continuity
Reversibility	Can the prior state be restored, and how long does it take?	There is no undo, a third party has acted, or compensation is costly
Sensitivity	Which data and capabilities become available?	Credentials, payments, personal data, or confidential terms are involved
Frequency	How many times can the error repeat before detection?	Volume can expand the potential harm quickly
Policy clarity	Would two owners handle the same case alike?	Exceptions are common or commercial judgment is required

If any factor scores 5, keep the action at read, recommend, or draft until a specific barrier addresses it. A low total may permit approved execution but does not automatically justify level 5. High frequency creates automation value and expands the blast radius.

This table suggests starting levels, not permanent destinations:

Action	Error	Reversal	Sensitivity	Frequency	Policy	Starting level
Summarize a report from approved views	Low	Easy	Medium	Weekly	Clear	1, read
Prioritize sales follow-ups	Medium	Easy	Medium	Daily	Mixed	2, recommend
Draft an email to a customer	Medium	Before sending	Medium	Daily	Mixed	3, draft
Fill an internal CRM field	Low	Through history	Medium	High	Clear	3; later 5 within limits
Confirm an order	High	Depends on dispatch	Medium	High	Variable	3 or 4
Issue an invoice	High	Requires an auditable correction	High	High	Clear with exceptions	3 or 4
Prepare a payment batch	Very high	Difficult after submission	Very high	Variable	Strict	3, no autonomous execution

Assign the level to the step, not the whole process. One workflow can read invoices at level 1, recommend priorities at level 2, draft messages at level 3, and require level 4 to send them.

Human approval needs an architecture

A busy person may approve out of habit. They may also review an accurate summary while the executable payload contains another amount or more records. Approval adds control under these conditions:

• the approver has the authority and context to decide
• the screen shows the exact action, its sources, and the proposed change
• the authorization expires, cannot be reused, and is bound to that payload
• the system validates permissions, limits, and current state again before execution
• a separate identity executes and records the result, especially for financial tasks

Traceability supports investigation. Recovery limits harm. Both are necessary. An external email can rarely be undone, so the design must prevent a wrong send and define a compensating action instead of promising rollback that does not exist.

Minimum controls before execution

Apply least privilege by tool and action. Current OAuth security practice recommends restricting an access token to the minimum privileges and audience required. Give the agent its own credentials, separate them by environment, and make them expire or support revocation. Never reuse the process owner’s personal account.

The log should answer who requested the action, which data and policy were used, what the model proposed, who approved, which tool ran, and what happened. Store useful identifiers and changes without copying secrets or unnecessary personal data.

Place the stop control outside the agent’s reasoning. It should pause queues, block new tool calls, and support credential revocation. Add volume and rate limits so one failure cannot repeat hundreds of times before an alert arrives.

Define recovery for each action: restore a field from history, cancel an order before dispatch, issue a credit note through the accounting process, or stop a batch before bank authorization. Test those procedures. Our article on secure repository access for coding agents applies the same pattern to branches, credentials, and production.

Controls AC-6, AU-12, and CP-10 in NIST SP 800-53 Rev. 5 cover least privilege, audit record generation, and system recovery. RFC 9700, section 2.3 applies least privilege to access tokens. The NIST Generative AI Profile proposes lifecycle risk management and documented roles for oversight, measurement, and response. These are design foundations. Each company still has to map them to its systems and responsibilities.

How to increase autonomy with evidence

Start in shadow mode. The agent observes real cases and produces output that nobody uses to act. Compare it with the team’s decision and record errors, omissions, and exceptions.

Move to recommendations once sources are traceable. Then allow drafts for one action type. Before level 4, test duplicate events, stale data, changes between approval and execution, connector failures, and recovery. Level 5 also needs a low initial volume and automatic stop events.

Review results by time period and number of cases:

• corrections and omissions, grouped by exception type
• attempts outside the allowed scope
• actions executed with the approved payload
• time to detect, stop, and recover from a failure
• workload created for approvers and exception owners

A high approval rate does not prove quality when people approve without checking. Look for explained disagreements, covered edge cases, and rehearsed recovery. Any expansion to new systems, recipients, or amounts returns to a controlled phase.

Kiia designs these levels around one real workflow, including its permissions, owners, and failure modes. The desired result is a sequence of actions whose control can be demonstrated before autonomy expands.