An AI agent is a language model that works through a task in a loop: it decides on an action, uses a tool such as web search, code execution or an API, looks at the result, and repeats until the task is done. The difference from a chatbot is control: a chatbot answers one message, while an agent chooses its own sequence of steps to reach a goal.
Anthropic's engineering guide draws the line precisely. Agents are systems where "LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks." The OpenAI Agents SDK defines them more compactly as "LLMs equipped with instructions and tools", with "a built-in loop that continues until the task is complete."
What an AI agent is made of
- A model that reads the situation and decides what to do next.
- Instructions, usually a system prompt describing the goal, rules and when to stop. See system prompt vs user prompt.
- Tools: functions the model can ask to run, each with a name, a description and a schema for its inputs. Examples: search the web, read a file, run a shell command, query a database, send an email.
- A loop run by ordinary code that executes the tool calls and feeds results back.
- Memory, optionally: notes or files that persist beyond the model's context window.
Anthropic calls the combination of model plus retrieval, tools and memory "the augmented LLM", the basic building block of every agentic system.
How an AI agent works: the loop
The model never runs anything itself. It produces a structured request, and your code does the work. Anthropic's API documentation describes the round trip for a tool your application runs: the model responds with stop_reason: "tool_use" and one or more tool_use blocks naming the tool and its arguments; "Your code runs the operation and sends back a tool_result." The model then reads the result and either calls another tool or writes its answer. An agent is that round trip repeated:
messages = [user_task]
while True:
reply = model(messages, tools)
if reply has no tool calls:
return reply.text # task finished
for call in reply.tool_calls:
result = run_tool(call.name, call.arguments)
messages.append(result) # the model sees what happened
if too_many_steps(): stop_and_ask_human()Two parts of that sketch matter more than they look. First, the results go back into the conversation, so each decision is based on what actually happened, not on what the model expected. Anthropic stresses this: "it's crucial for the agents to gain 'ground truth' from the environment at each step." Second, the loop needs an exit besides "done". The same guide notes it is "common to include stopping conditions (such as a maximum number of iterations) to maintain control," and that agents "can then pause for human feedback at checkpoints or when encountering blockers."
An example: an agent fixing a failing test
Ask a coding agent: "The test_invoice_total test fails. Fix it." An illustrative run looks like this:
- Calls a shell tool:
pytest tests/test_invoice.py. Reads the failure: an assertion error, expected 107.0, got 100.0. - Calls a file-reading tool on
invoice.pyand sees that tax is never added. - Calls an edit tool to change the total calculation.
- Runs the test again. It passes.
- Runs the full test suite to check nothing else broke, then reports what it changed.
Nobody scripted those five steps. If step 4 had failed, the agent would have read the new error and tried something else. That ability to change course based on results is what makes it an agent, and it is also why agents can't promise a fixed number of steps or a fixed cost.
AI agent vs workflow vs chatbot
Anthropic separates agents from workflows, which are "systems where LLMs and tools are orchestrated through predefined code paths." A workflow might be: summarise this email, then classify it, then draft a reply. The model does each step, but the code decides the order.
| Chatbot | Workflow | Agent | |
|---|---|---|---|
| Who decides the steps | Nobody; one reply per message | Your code, in advance | The model, as it goes |
| Uses tools | Sometimes, once | At fixed points | Whenever it decides to |
| Number of steps | 1 | Fixed | Varies with the task |
| Cost and latency | Low, predictable | Predictable | Higher, variable |
| Good for | Questions, drafting | Repeatable, well-defined processes | Open-ended tasks where the path isn't known in advance |
The workflow patterns Anthropic lists (prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer) cover a lot of ground before a full agent is needed. Its advice is to start with "the simplest solution possible, and only increasing complexity when needed."
What AI agents are used for
- Coding agents that read a codebase, run commands and tests, and edit files.
- Research agents that search the web, read pages and compile a report with sources.
- Browser and computer-use agents that click through websites or desktop apps to complete forms or gather data.
- Customer-support agents that look up orders, check policies and take actions such as issuing a refund, often with a human approving the riskier ones.
- Data agents that write and run queries or analysis code against a dataset and explain the results.
The costs and risks of AI agents
Cost and speed
Every loop iteration is a model call, and the growing conversation, including every tool result, is sent each time. Anthropic's guide is direct: "Agentic systems often trade latency and cost for better task performance." A task that takes a chatbot one call can take an agent dozens. How to reduce AI token usage covers ways to keep that in check.
Compounding errors
A wrong conclusion in step 2 becomes the premise for step 3. The same guide warns that autonomy means "the potential for compounding errors." Models can also state wrong things confidently (why AI hallucinates), and in an agent a hallucinated file name or API field turns into a failed action rather than just a wrong sentence. Good agents check their work against the environment, such as by running tests, rather than trusting their own output.
Too much power
An agent can only do what its tools allow, so the tools define the damage a mistake or a manipulated instruction can cause. OWASP's Top 10 for LLM applications lists this as "Excessive Agency" and traces it to "excessive functionality," "excessive permissions," and "excessive autonomy." Its mitigations are concrete: give the agent only the tools it needs, give each tool the minimum permissions (for example a read-only database account), and require a human to approve high-impact actions such as payments, deletions or sending messages.
Text the agent reads is a risk too. A web page or email can contain instructions aimed at the agent, and a model that treats them as commands may follow them. Limiting tools and approving risky actions is the defence that still works when the model is fooled.
Should you build one?
Use an agent when the task is open-ended, the steps can't be predicted, and there is a way to check results automatically (tests, a validator, a human review). Use a workflow, or a single well-written prompt, when the steps are known. Most of the reliability of an agent comes from the boring parts around the model: clear tool descriptions, a step limit, good error messages fed back as tool results, and approval gates on anything that can't be undone.
Frequently asked questions
- Is ChatGPT or Claude an AI agent?
- The chat model on its own answers one message at a time, which is not an agent. The products built around these models increasingly include agent features: they can search the web, run code, read files or operate a browser over several steps before replying. When a model is choosing and calling tools in a loop to finish a task, it is acting as an agent.
- What is the difference between an AI agent and a chatbot?
- A chatbot replies to what you say. An agent works toward a goal: it decides on actions, uses tools such as search, code execution or APIs, looks at the results, and keeps going until the task is done or it needs your input. Many chat products now do both, switching into agent behaviour when a request needs several steps.
- Do AI agents learn from their mistakes?
- Not in the sense of permanently retraining themselves. Within a single task, an agent sees the results of its actions, such as an error message or a failed test, and can adjust its next step. Anything carried between tasks has to be stored explicitly, for example in a memory file or database the agent can read later.
- What is a multi-agent system?
- A setup where several agents, each with its own instructions and tools, split a task. A common pattern is an orchestrator that breaks the work into pieces and hands them to worker agents, then combines the results. The OpenAI Agents SDK calls passing work from one agent to another a handoff.