A product team wants to automate a difficult task. The inputs are messy, the current process contains exceptions, and a language model appears able to explain what should happen. The next architectural decision is whether the application should control the steps or let an agent choose its next action.

That decision deserves a more precise answer than “use an agent because the task involves AI.” For teams moving from AI assistants to agentic workflows, the useful question is which decisions require model judgment, which can remain explicit rules, and how the resulting system will be evaluated.

Start with who controls the next step

In its foundational Building effective agents essay, Anthropic distinguishes predefined orchestration from systems in which a model dynamically directs its process and tool use. This is a useful architectural distinction, even though vendors use the word “agent” differently.

For a product discussion, separate four arrangements:

  • Conventional automation: application code performs defined calculations, validations and state transitions.
  • A workflow containing model calls: code controls the process, while a model performs bounded tasks such as extraction, classification or drafting.
  • An agent loop: a model selects the next permitted action, observes its result and continues until an exit condition is reached.
  • A hybrid: a defined process delegates one bounded investigation or planning step to an agent, then resumes application-controlled execution.

Calling the control flow predefined is more precise than calling the whole system deterministic: generated outputs can vary between runs even when application code controls the steps.

Separate uncertain inputs from an uncertain path

An incoming support message may be unpredictable while the required procedure remains stable: identify the account, retrieve the relevant record, classify the issue, and prepare an explanation. A model may help interpret the message without needing authority to invent the procedure.

OpenAI’s practical guide to building agents identifies nuanced decisions, difficult rule sets and unstructured information as candidate use cases. These are reasons to investigate agent value, rather than proof that every such task needs dynamic orchestration.

A practical test is to ask what the system learns after each step. If that evidence changes which investigation should happen next, an agent may add value. If it mainly changes the contents of a known output, a workflow with a model step is a credible baseline. Exceptions that signal missing product requirements should first be clarified with the process owner.

Compare three implementations of the same job

Consider a hypothetical B2B application with a support request: “Our weekly customer import did not finish.” The objective is to produce an evidence-backed diagnosis for an operator. No customer records will be modified automatically.

Option 1: A conventional diagnostic workflow

The application reads the import job status, checks documented validation codes, and returns the matching resolution instructions. For known failures such as a missing required column, this may be sufficient. Unrecognized conditions go to the support team with the collected evidence.

Option 2: A workflow with a language model

The same application-controlled sequence retrieves the job result and relevant documentation. A model explains the failure in language suited to the customer’s question. Validation checks ensure that the response refers to the correct job and does not introduce an unsupported action. The model improves interpretation and presentation without controlling the investigation.

Option 3: A bounded investigative agent

The agent can choose among permitted searches of job records, relevant service events and release notes. A timeout may lead to one investigation; a changed file format may lead to another. It returns a diagnosis with evidence, or states what remains unresolved when the investigation budget is reached.

These are alternative designs, not measured results. The third option earns its additional freedom only if it resolves meaningful cases that the simpler versions handle poorly. The comparison should include the effort required to verify its answer, rather than treating a longer explanation as a better diagnosis.

Use a decision rubric before choosing a framework

The following questions provide a practical design review. They are a proposed rubric for the import example, not an industry scoring standard.

  • Path variability: do observations determine a genuinely different next investigation, or only select a known branch?
  • Evidence quality: can the system retrieve authoritative records and distinguish a supported diagnosis from a plausible guess?
  • Action scope: will it only prepare a result, or can it change shared state or communicate externally?
  • Tool readiness: are the permitted operations clear, bounded and testable?
  • Operating limits: can the team define when to stop, wait, ask for missing information or escalate?
  • Measured benefit: does the additional freedom improve accepted outcomes enough to justify its operating and maintenance cost?

Microsoft’s agent orchestration guidance similarly starts with the lowest complexity that reliably meets the requirement, from direct model calls through single agents to multi-agent systems. Use that progression as an architectural option, not a maturity ladder every product must climb.

Define the execution envelope

Before testing a dynamic loop, specify its permitted tools, data boundaries, completion conditions and fallback. In the import example, a useful answer contains the affected job, supporting evidence and the next recommended operator action. An unresolved answer is a valid outcome when the evidence is insufficient.

The OpenAI Agents SDK runner documentation describes a model-and-tool loop with a configurable turn limit and an exception when that limit is exceeded. A turn limit is one control; applications should separately manage elapsed time, tool requests and spending. A small number of model turns does not necessarily imply a small total cost.

If the design later adds a repair operation, recovery needs its own engineering. The AWS Builders’ Library guidance on idempotent APIs explains how a caller-provided request identifier can support safe retries when the destination implements the corresponding contract. An agent should not assume a timed-out write failed or repeat it merely because the outcome is unclear.

Keep uncertain outcomes explicit and use the destination’s supported recovery process. For actions requiring review, define where human approval belongs separately from the model’s ability to propose an action.

Improve the tools before multiplying the agents

An agent cannot compensate reliably for tools whose purpose, parameters or results are ambiguous. Anthropic’s guidance on writing effective tools recommends a carefully selected tool set, clear functional boundaries, meaningful responses and evaluations grounded in actual tasks.

For the hypothetical investigation, expose operations that retrieve a specific job, search relevant events and locate applicable documentation. Return enough context to connect a result to the customer’s issue. Enforce access boundaries in application services; the model’s instructions should not be the mechanism that separates customer data.

Adding specialists becomes a separate experiment. A single investigative agent may already be sufficient. If the model repeatedly confuses tools from unrelated domains, a narrower tool set or a defined routing step may solve the problem. Multiple agents should address a demonstrated limitation and be evaluated for coordination failures as well as their individual outputs.

Compare accepted outcomes, not impressive demonstrations

Run the alternatives against the same representative cases. Include routine failures, missing identifiers, contradictory records, unfamiliar error messages, unavailable tools and cases where the correct answer is “more information is needed.” Keep development examples separate from cases used to judge readiness.

OpenAI’s evaluation guidance recommends task-specific evaluation and checks on tool selection and argument accuracy. For the import investigation, assess whether the diagnosis is supported, whether the correct customer and job were used, and whether the system respected its action boundaries.

Define acceptance criteria with the process owner before comparing results. Track successful resolutions, unsupported conclusions, operator correction effort, elapsed time and cost per accepted outcome. Report difficult cases separately so improvements on common cases do not conceal serious failures. Any example thresholds should come from the product’s actual service expectations.

Inspect the execution history when an answer fails. The Agents SDK tracing documentation describes traces that connect model and tool activity, and warns that captured inputs and outputs can contain sensitive information. Choose what to capture, retain and export deliberately; observability should help diagnose behavior without unnecessarily replicating customer data.

Expand autonomy where the evidence supports it

For this hypothetical product, a reasonable first release could keep the known diagnostic workflow, use a model to explain documented failures, and reserve the investigative agent for unresolved cases. The operator receives the same evidence format regardless of which route produced it.

Start with a limited set of failure types and compare the alternatives before expanding access. Revisit the decision when tools, models or the underlying process change. The durable product capability is the ability to choose and verify the right action; the amount of autonomy can change as the evidence improves.

Sources

Sources checked on 31 October 2026. Anthropic’s architecture essay was published in December 2024 and its tool-design essay in September 2025. These and the AWS retry guidance are used as foundational engineering material; implementation details should be checked against the deployed framework version.