“Agent” is often used for any software that calls a language model. That label hides the design decision buyers actually need to make: how much behavior should be determined at runtime, and how much should remain explicit in software?
The safest and usually fastest rule is to choose the least autonomous system that can handle the variability of the workflow. Autonomy creates value when the path cannot be enumerated economically. It creates avoidable risk when a normal state machine would be clearer.
Three useful system classes
Deterministic automation follows known rules and transitions. AI-assisted workflows use a model for a bounded step—classification, extraction, drafting, or ranking—inside an explicit process. Agents decide which steps or tools to use as the task unfolds.
- Deterministic automation: stable inputs, explicit rules, high auditability, and predictable exceptions.
- AI-assisted workflow: unstructured inputs but a known process and clear review point.
- Agentic system: variable task paths, multiple tools, incomplete information, and a meaningful benefit from adaptive planning.
The decision is about variability and consequence
A workflow with many document formats may need model-based extraction but still follow a fixed approval sequence. A research task may need an agent because the next query depends on evidence found during the previous step. The second system is more autonomous because the problem demands it, not because autonomy is a product feature.
High consequence pushes the design toward explicit controls. If an agent is justified, constrain the action space, isolate credentials, and require confirmation for irreversible steps.
Why over-agentic designs fail
Allowing a model to rediscover a known workflow on every run increases latency, cost, variance, and testing difficulty. Teams then compensate with prompts that restate the missing software logic. That logic is harder to version, inspect, and enforce.
A hybrid design often wins: deterministic orchestration for the business process, model calls for uncertain interpretation, and limited agent loops only where adaptive exploration creates measurable value.
Run the same task through competing designs
Before committing, implement a rule baseline, an assisted workflow, and a limited agent for a representative task set. Compare completion quality, exception rate, operator time, latency, cost, and the severity of failures. The smallest design that clears the acceptance boundary is the strongest default.
Use autonomy only where variation creates value
Map the workflow into stable rules and judgment-heavy decisions. Stable routing, calculations, approvals, and record updates usually belong in deterministic automation. Language models are most useful where inputs vary, intent is ambiguous, or the system must synthesize unstructured information. A hybrid design often produces more value than making the entire process agentic.
Score each candidate step by variability, reversibility, consequence, observability, and frequency. High-frequency, irreversible actions with weak feedback should receive the least autonomy. Reversible drafting or classification with clear review can tolerate more. This produces an autonomy budget grounded in operating risk rather than enthusiasm for a new interface.
Define the escalation path before the happy path
A useful system knows when it lacks evidence, when policy conflicts, when a tool result is ambiguous, and when the user request exceeds its authority. Escalation must specify who receives the case, what context accompanies it, how urgent it is, and how work resumes after a decision. Sending every uncertainty to a generic queue simply transfers the bottleneck to people.
Measure automation by completed outcomes, not by the percentage of steps touched by AI. Track exception rate, rework, review time, incorrect actions prevented, and time to recovery. A less autonomous system can outperform an agent if it completes more work predictably and creates fewer expensive exceptions.
Executive decision record
The decision is where variable language or judgment creates enough value to justify autonomy beyond conventional rules and orchestration. Write that decision before selecting a model, vendor, framework, or implementation pattern. A written boundary keeps technical exploration connected to the operating outcome and makes it possible to explain why the organization advanced, revised, or stopped the work.
Approval should depend on a workflow decomposition, deterministic baseline, exception distribution, consequence analysis, and outcome comparison across autonomy levels. The evidence does not need to remove every uncertainty, but it should address the uncertainty capable of changing value, architecture, risk, or ownership. Record the baseline, assumptions, unresolved questions, and the person accepting the next stage.
Failure boundary and operating ownership
The central failure to guard against is making stable steps agentic, thereby increasing variance, cost, and review without improving the completed business outcome. Treat that condition as a testable scenario. Define how the system detects it, what users experience, which action is prevented or reversed, and what evidence reaches the person responsible for recovery.
Long-term accountability sits with the operating process owner, not an isolated AI team, because that owner controls policy, exceptions, staffing, and adoption. Supporting specialists can provide platforms, research, review, or delivery capacity, but they cannot substitute for an owner who controls policy and operating change. Name that owner before production and include the ownership path in release evidence and incident procedure.
A practical 90-day application plan
During the first 30 days, convert a workflow decomposition, deterministic baseline, exception distribution, consequence analysis, and outcome comparison across autonomy levels into a bounded evidence plan. Assign each artifact to a named contributor, identify the representative inputs required, and agree on the comparison baseline before implementation expands. The objective of this period is to expose the assumption most likely to invalidate the work while the cost of changing direction is still low.
During days 31 through 60, build or instrument the smallest complete workflow that can support the decision about where variable language or judgment creates enough value to justify autonomy beyond conventional rules and orchestration. Include the real data and authorization path where feasible, record exceptions, and review difficult cases with the people who own the underlying process. Resist adding breadth until the team can explain the measured behavior of this narrow slice.
During days 61 through 90, test the boundary represented by making stable steps agentic, thereby increasing variance, cost, and review without improving the completed business outcome. Exercise degraded dependencies, ambiguous inputs, recovery, and handoff rather than demonstrating only successful cases. End the period with a written advance, revise, or stop decision that cites evidence, residual exposure, expected operating cost, and the next authority boundary.
The review should be accepted by the operating process owner, not an isolated AI team, because that owner controls policy, exceptions, staffing, and adoption. That group should confirm not only that the system can work, but that ownership, support capacity, monitoring, and change control are credible. If those conditions are absent, the responsible outcome is another bounded learning stage rather than an unsupported production commitment.
Practical checklist
- Can the valid steps be enumerated and maintained?
- Do inputs require interpretation or only validation?
- Does the next step depend on newly discovered information?
- What is the maximum consequence of a wrong action?
- Can the action be reversed automatically?
- How will operators review exceptions?
- What simpler baseline will the agent need to beat?
Engagement scenario
An accounts-payable process uses a model to extract fields and classify exceptions, but deterministic code matches purchase orders and enforces approval limits. An agent is reserved for gathering missing supporting information across approved systems; it cannot release payment.
Continue reading
- [Enterprise AI agent architecture](/enterprise-ai-agent-architecture)
- [Human-in-the-loop agent design](/human-in-the-loop-ai-agent-design)
Sources and further reading
- [Anthropic: Trustworthy agents in practice](https://www.anthropic.com/research/trustworthy-agents)
- [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework)