AI Agent Autonomy Framework: Decide What Should Run Without a Human

Autonomy is a measure of how cheap your mistakes are.

The question of whether an AI agent should act on its own is not primarily a question about model capability. It is a question about two properties of the decision itself: how much damage a wrong output causes, and whether that damage can be undone.

Score those two, and the oversight tier follows. Low cost and reversible means the system runs. High cost or permanent means the system diagnoses and a person decides. The middle cases need an explicit rule rather than an assumption.

This matters because most agent deployments inherit their autonomy boundary from the architecture rather than choosing it. Whatever the integration made easy became the policy. That is how a system ends up with the authority to send customer-facing messages because the API happened to support it.

Key takeaways

• The two variables that determine autonomy are magnitude of harm and reversibility, not model accuracy.
• Model capability affects error rate. It does not affect error cost. Only the second one should set the boundary.
• Four oversight tiers cover most enterprise cases: autonomous, autonomous with audit, recommend-and-approve, and diagnose-only.
• The boundary has to be explicit and understood by the people operating the system. An implicit boundary is an assumption waiting to be tested in production.
• Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, or inadequate risk controls. The third cause is the one this framework addresses directly.
• Autonomy should follow process understanding, not precede it. You cannot scope a boundary around a process you have not mapped.

Why capability is the wrong variable

The instinct when scoping an agent is to ask how good the model is. If accuracy is high enough, let it run.

That reasoning has a structural flaw. Accuracy tells you how often the system will be wrong. It tells you nothing about what happens when it is. A model that is right 99.2% of the time is an excellent assistant for drafting internal summaries and an unacceptable one for issuing refunds, and the difference has nothing to do with the model.

Consider four decisions with identical error rates. Summarizing a meeting transcript, where a wrong output means someone reads a slightly inaccurate summary and the correction is trivial. Routing a support ticket, where a wrong output means a case sits in the wrong queue for hours and the correction takes effort. Approving a vendor payment, where money leaves the company and recovery is partial and costly. Sending a customer-facing communication, where the customer has already read it and no rollback exists.

Same model, same accuracy, four different answers on autonomy. This is why "is the model good enough" is not a governable question, and why the framework below scores the decision rather than the system.

The two questions

Before any agent runs without human review, answer both.

Question 1 — If this is wrong, how big is the damage?

Score the worst plausible outcome of a single wrong output, not the average one. Consider financial exposure, regulatory consequence, customer impact, employee impact, and reputational effect.

A score of 1 is negligible: a person notices and moves on, such as a draft with a wrong heading. A score of 2 is contained: one workflow is delayed or reworked, such as a misrouted internal request. A score of 3 is material: a customer, a payment, or a compliance record is affected, such as an incorrect entitlement applied. A score of 4 is severe: regulatory exposure, financial loss, or harm to an individual, such as a wrong credit or claims decision.

Question 2 — If this is wrong, can we undo it?

Reversibility is about the state of the world after the action, not about whether the log can be corrected.

A score of 1 is fully reversible: nothing left the system. A score of 2 is reversible with effort: internal state changed but a record can be restored. A score of 3 is partially reversible: external state changed and a payment can be clawed back at some cost. A score of 4 is irreversible: information left the organization or a decision was communicated.

A useful heuristic: if a human being outside the team has already seen or acted on the output, treat it as irreversible regardless of what the database allows.

The four oversight tiers

Plot the two scores and the tier follows.

The rule for the mixed cases is that the higher of the two scores governs. A decision with low magnitude and irreversible effect still goes to tier 3 or 4, because an irreversible small mistake compounds when it repeats a thousand times before anyone notices.

Autonomy boundary

How the two scores map to oversight

The boundary is a property of the decision, not of the model. Improving the model moves the error rate, not the tier.

Applying the framework: a worked example

Consider a finance team deploying an agent on invoice exception handling. The instinct is to treat "invoice exceptions" as one use case. It is six.

Decomposing the use case is where most of the value is. A single autonomy setting for "invoice exceptions" either blocks the classification step that could run freely, or authorizes the send step that should not.

Notice that the drafting and sending steps have the same magnitude but different reversibility, and that difference alone moves three tiers. This is the most common place where deployments get the boundary wrong.

Making the boundary explicit

A boundary that exists in the architecture but not in anyone's head is not a control. Four things have to be written down before an agent goes live.

Scope of action. What the agent is permitted to do, enumerated. Not "handle exceptions" but the specific set of actions it may take. Anything not on the list requires escalation.

Escalation trigger. The conditions under which the agent must stop and hand off: low confidence, an input pattern outside its training distribution, a value above a threshold, a flagged counterparty, a customer segment.

Named owner. One person accountable for the boundary, the error budget, and the decision to widen or narrow it. Gartner attributes agentic project cancellations in part to inadequate risk controls, and unowned boundaries are the most common form of inadequate control.

Reassessment trigger. What causes the tier to be reviewed: a change in the underlying process, a model update, an incident, a volume threshold, or a fixed cadence.

This is the difference between a system with governance and a system that has not failed yet.

What has to be true before you set a boundary at all

The framework assumes you know what the process does. That assumption fails more often than it should.

Setting an autonomy boundary requires knowing which exceptions exist, how often they occur, who currently handles them, what the escalation path is today, and where regional or customer-segment variation changes the answer. If that information is not available, the scoring exercise produces confident numbers built on guesses.

This is the sequencing point that most agent programs skip. Process understanding, then boundary, then autonomy. Reversing the order produces agents scoped around the documented process while the real work continues to run through exceptions nobody mapped.

The failure mode is specific: the agent handles the happy path correctly and the exception path is where the cost lives.

Reversibility is easiest to underestimate in regulated operations

The magnitude score tends to be assessed reasonably well. Reversibility is where teams get optimistic, because the system view suggests a correction is always possible.

Financial services makes the distinction concrete. In pension fund administration, an error in an international transfer can produce a financial loss that is difficult to recover. The record can be amended. The transfer cannot be unsent, and the recovery path runs through a counterparty institution, a regulator, and a customer relationship rather than through a database.

That is a reversibility 4 dressed as a reversibility 2 on any architecture diagram. The test that catches it is not technical: ask what the recovery path actually requires, in the world, and who has to agree to it.

Where Horizon fits

Horizon is an AI-powered continuous discovery platform. Its role in this framework sits before the scoring, at the point where teams need to know what the process actually does.

Discovery Cycles run AI-led interviews at census scale across the roles that operate a process, adapting to each role and following up on the gaps. The Process Library structures what comes back into navigable documentation: which exceptions recur, who owns each decision, which approvals live outside any system, and where the process varies by region or team. That is the input the two scoring questions require.

AFAP SURA, a pension fund administrator in Uruguay with 600 employees, used Horizon on its ANR process for international transfers and its BPC workflows. Those processes had required weeks of manual interviews reaching a small portion of the team, with information scattered across Floxu, UI, CAFAP, Treasury and banking systems, and a standing risk that an error in an international transfer would be difficult to recover.

In under seven days Horizon delivered 63 prioritized insights covering bottlenecks, repetitive tasks and operational risk points, with a complete view integrating all five systems, against a three-week baseline for the manual approach. The pilot returned 186% ROI.

That is one engagement under specific conditions rather than a projection for any organization. What generalizes is the sequence: the operational risk points were identified before anyone decided what could run without a person.

An agent scoped from documentation inherits an incomplete version of the company. An agent scoped from evidence has a boundary that reflects the exceptions it will actually encounter.

Autonomy readiness checklist

Run this before any agent moves from pilot to production.

1. Have you decomposed the use case into individual decisions rather than scoring it as one?
2. For each decision, have you scored worst-plausible magnitude, not average?
3. For each decision, have you scored reversibility based on external state, not database state?
4. For anything scored reversible, have you named the recovery path and who has to agree to it?
5. Does the higher of the two scores govern the tier?
6. Is the scope of permitted actions enumerated rather than described?
7. Are the escalation triggers specific enough to be implemented?
8. Is there one named owner for the boundary?
9. Do the operators of the system know where the boundary sits and why?
10. Is there a defined error budget and a monitoring signal against it?
11. Is there a reassessment trigger, including for model updates?
12. Was the process mapped from real evidence before the boundary was set?
13. Is there a documented path to narrow autonomy, not only widen it?

Common mistakes

Letting the integration set the policy. If the agent can send because the API supports sending, the boundary was chosen by an engineering convenience rather than a risk decision.

Scoring average impact instead of worst plausible. The average error is not what ends up in an incident report.

Treating a better model as a reason to widen the tier. A better model reduces frequency. It does not change what happens when the system is wrong.

Confusing database reversibility with real reversibility. If a person outside the organization has seen the output, no rollback exists.

Setting one autonomy level per use case. Most use cases contain decisions that belong in different tiers.

Only ever widening. A governance system with no path to reduce autonomy after an incident is not a control loop.

FAQ

How do you decide how much autonomy an AI agent should have?

Score the decision on two axes: how much damage a wrong output causes, and whether that damage can be reversed. The higher of the two scores determines the oversight tier. Model capability affects how often errors occur, but not what happens when they do, so it should not set the boundary.

What is human-in-the-loop AI?

Human-in-the-loop describes any arrangement where a person reviews, approves, or corrects AI output before or after it takes effect. In this framework it corresponds to tiers 3 and 4, where the system produces a recommendation or a diagnosis and a person makes the decision.

Why do agentic AI projects get canceled?

Gartner's June 2025 prediction attributes the projected cancellation of over 40% of agentic AI projects by the end of 2027 to escalating costs, unclear business value, and inadequate risk controls. In practice these are related: scope drawn around the documented rather than the real process drives cost up and value down, and unmapped exception paths are where risk controls turn out to be insufficient.

Should autonomy increase as the model improves?

Not automatically. A model improvement lowers error rate, which changes how often you exercise the recovery path, not what the recovery path costs. Autonomy should widen when the cost or reversibility of the decision changes, when monitoring proves the error budget holds, or when the process itself is redesigned to make errors cheaper.

How do you set a boundary for a process nobody has mapped?

You do not. Scoring magnitude and reversibility requires knowing which exceptions exist and how often, who owns each decision, and where the process varies. If that evidence does not exist, establish it first. A boundary set from documentation reflects the process as intended rather than as practiced.

Decide the boundary before the deployment decides it for you

Every agent in production has an autonomy boundary. The only question is whether someone chose it.

The most reliable systems are the ones where that boundary is written down, owned by a person, scored against the actual cost of being wrong, and grounded in evidence about how the process really runs. Not assumed, not inherited from the architecture, and not left to be discovered during the first incident.

See it. Fix it. Scale it.

See Horizon in action.