An AI agent is a system that uses a model to choose actions, observe their results and decide what to do next. Tools give it capabilities such as retrieving a document, running a calculation or editing a file. The useful distinction is how decisions are made, not whether a product uses the word “agent”.
Agent, workflow or chatbot?
A chatbot can answer without taking any external action. A workflow can call a language model at several steps while the application still controls the order. An agent gives the model more discretion over the next step. These are design choices on a spectrum: a fixed workflow can contain an agent, and an agent can invoke a fixed workflow.
| System | Who chooses the next step? | Example |
|---|---|---|
| Single model call | The application | Summarize one supplied report. |
| Workflow | Mostly predefined application logic | Retrieve a report, extract fields, validate them, then format a brief. |
| Agent loop | The model, within application limits | Inspect a repository, choose files to read, propose a change, run checks and revise it. |
Anthropic’s Building effective agents makes a similar distinction between predefined workflows and systems where the model directs tool use. It is a useful design reference, not proof that an agent is always the better solution. For a predictable task, a smaller workflow can be easier to evaluate and operate.
How the tool-use loop works
- Provide a goal and context. The application supplies the request, available tools, relevant state and limits.
- Choose an action. The model may answer, request a tool call or ask for information. A tool call is a structured request; it does not execute itself.
- Validate and execute. Application code checks arguments and permissions, then invokes the tool. The model should not be the only component enforcing access limits.
- Return an observation. The tool result becomes evidence for the next decision. A timeout, partial result or failure needs to remain distinguishable from success.
- Continue or stop. The system checks completion, budgets and escalation conditions. A repeated failure should not lead to an unbounded retry loop.
Consider a research assistant asked to compare two papers. It might find each paper, inspect the evaluation section, extract the datasets and then compare the claims. A convincing paragraph is not enough to show that it completed the task: the references must resolve, the extracted numbers must match the papers and the comparison must acknowledge differences in setup.
Evaluate the whole task, not one fluent answer
Build an evaluation set from the work you actually want done. Include ordinary cases, ambiguous instructions, missing information and tools that return errors. Keep a held-out set so that repeated prompt tuning does not turn the test into a training exercise.
- Success: define an observable final result. For a code task, passing relevant checks is stronger evidence than the agent saying it fixed the issue.
- Reliability: repeat some tasks and record the distribution of outcomes. One successful demo can hide inconsistent behavior.
- Cost and latency: measure the complete run, including retries, tool calls and unsuccessful attempts.
- Action quality: record invalid calls, unnecessary writes and incorrect assumptions, even if the final answer looks acceptable.
- Escalation: check whether the system stops or asks for help when it lacks the required evidence or access.
When comparing systems, keep the task set, tool access, maximum time and attempt budget comparable. More sampling or a larger tool budget can improve results without demonstrating a better underlying model. Publish those conditions beside the score.
What changes in a real deployment?
Tool access makes failures consequential. A read-only document search and an agent able to send messages or change production data need different controls. Separate read and write capabilities, validate inputs, isolate untrusted documents from application instructions and require appropriate authorization for consequential actions.
Logs should show which evidence supported each action without unnecessarily retaining secrets or private data. Add cancellation, time and spending limits, and a recovery path for partial completion. A multi-agent arrangement adds coordination overhead; it does not remove these requirements.
Questions to ask when reading an agent announcement
Which tasks were tested? Were tools simulated or real? Was success graded automatically, by people or by another model? How many attempts were allowed? Was the test set public before development? Can the result be reproduced with the described model and tools? These questions turn an attractive demonstration into something you can assess.
Follow AI agent news for current reporting, or use the AI research reading checklist to examine the evidence behind a benchmark claim.
Sources and scope
- Anthropic: Building effective agents — workflows, agent patterns and tool design.
- NIST AI Risk Management Framework — a broader framework for evaluating and managing AI risks.
This guide explains architectural choices and evaluation questions. It does not rank vendors or report an independent benchmark. Send a correction or a source suggestion.
Follow the latest coverage
Otter.ai raises $320 million from General Catalyst’s growth fund for its meeting tool
Meeting note-taking tool Otter.ai announced on Tuesday that it has raised $320 million from General Catalyst’s Customer Value Fund (CVF), which provides capital to startups without diluting equity and gets returns on the results of marketing spend. Otter.ai said that it will use these funds for enterprise growth, go-to-market strategy, international expansion, and product innovation. […]
CoreWeave’s debt is becoming the whole AI industry’s problem
CoreWeave had a bad week last week. Its bonds slipped into distressed territory by a common definition - when investors demand 1,000 basis points more in yield than they'd get from government bonds. Last Tuesday Chamath "SPAC King" Palihapitiya noticed the distressed debt, and also asked whether the AI buildout was a private credit bubble. […]
A visit to Samsung’s massive new semiconductor hub
Generative AI models run on powerful chips known as GPUs, which depend on stacks of specialized memory chips packaged with the processor. To meet growing demand for memory, Samsung is building a massive new semiconductor manufacturing complex in Pyeongtaek, South Korea. Three fabs already operate around the clock, and a fourth is coming online. Construction…