LLM vs AI Agents

While a LLM serves as the core engine, tools like Claude, Gemini, ChatGPT (chatbots), Claude Code (agentic coding), NotebookLM (retrieval-grounded) are comprehensive AI tools built by layering additional modules onto that core model.

Standalone LLMs have several inherent limitations:

• Static Knowledge: Their internal knowledge is fixed at the time of training.
• Action Limitation: They cannot inherently send emails, fetch real-time data, or interact with other software.
• Reasoning Gaps: They can be "distracted" by useless information or fail at multi-step logic without a structured reasoning framework.

An AI tool is composed of several additional modules, primarily an orchestrator (also called a router) that:

• determines how many times the LLM has to be invoked, with which inputs and how the outputs have to be used;
• determines which external commands have to be invoked, with which inputs and how the outputs have to be used;
• determines how the text produced by an LLM invocation has to be used for the next LLM or external command invocation;
• enforces safety/resource limits (number of iterations, safety guardrails, permission checks on tool calls)

How an LLM invokes a software

An LLM cannot invoke an external software command. The actual invocation is done by an orchestrator.

The invocation is triggered when the LLM generates a specific predefined marker or command string instead of standard natural language. This occurs when the LLM determines that its internal knowledge is insufficient and external data or computation is required. During text generation, the LLM outputs a structured string, such as {tool: calculator, expression: 10 * 4 * 2}. The orchestrator detects this specific pattern, pauses text generation, and executes the requested command.

LLMs are not naturally trained to produce these markers, they are typically fine-tuned using supervised data where parts of the desired output are replaced with these commands. Alternatively, they can be prompted with a list of available function APIs and instructions on how to call them.

The input for the command is constructed dynamically by the LLM based on the tool, the user's query and the current context. For example, if a user says, "Find my bear in the living room," and the tool is find_object(item, location), the LLM extracts "bear" and "living room" from the conversation history to populate the command.

Once the external software executes the command, the result of the command (e.g., the answer from a calculator or a database snippet or a value returned from an API) is integrated back into the workflow by the orchestrator. The result is usually fed back to the LLM so that the LLM then continues predicting the next tokens based on this new, grounded information.

If the external software returns an error or no response, the orchestrator or the LLM must be capable of realizing the failure and either trying a different approach or informing the user, though "hallucinating" a response despite a tool failure is a known risk.

Web search

This is an example of a chatbot that receives a question whose response requires executing and summarizing a web search. An orchestrator invokes several times the LLM and external commands for the actual execution of web searches.

We will consider this example: "What's the current inflation rate in the US, and how does it compare to a year ago?". This clearly needs live data.

#	Call	Purpose	Can be skipped/merged?
1	Query planning	Decide to search + generate queries	Sometimes merged into the main model deciding to invoke a "search" tool directly (tool-use pattern)
2	Relevance filtering	Pick best sources from raw results	Often replaced by simple heuristics (top-N by rank)
3	Per-source extraction	Compress each page to relevant facts	Often skipped; raw snippets fed directly to Call 4 instead
4	Final synthesis	Write the actual answer	Never skipped — this is the core call

Step 1 - LLM call: Query planning / search query generation

The orchestrator decides whether to search, and if so, it generates one or more effective search queries (the user's raw question is often a bad search query as-is).

LLM Input:

System: You are a query planner. Given a user question, decide if a web search is needed. If so, output 1-3 concise search queries as JSON.

User question: "What's the current inflation rate in the US, and how does it compare to a year ago?"

LLM Output:

{
"needs_search": true,
"queries": [
"US inflation rate August 2026",
"US inflation rate one year ago comparison"
]
}

Step 2 - External Command: Execute the search(es)

The orchestrator invokes an external command, (e.g., Bing API, Google Custom Search) with each query from Call 1. Returns, say, 10 results per query: titles, URLs, snippets.

Step 3 - LLM call: Relevance filtering / re-ranking

Assuming the command returned \~20 raw results, the orchestrator decides which 3-5 are actually worth reading in full, since fetching and processing every page is expensive. Then, it invokes the LLM with a dedicated prompt.

LLM Input:

System: Given these search results, select the 4 most relevant and authoritative sources for answering the question below. Return their indices only.

Question: "What's the current inflation rate in the US..."

Results:
1. [BLS.gov] "Consumer Price Index Summary" - bls.gov/news.release/cpi.nlm...
2. [Reuters] "US inflation eases to X% in July" - reuters.com/...
3. [random blog] "My thoughts on the economy" - medium.com/...
... (17 more)

LLM Output:

{"selected_indices": [1, 2, 5, 8]}

Step 4 - External Command — Fetch full page content

The orchestrator fetches the actual page content for the 4 selected URLs (via web_fetch-style tool), possibly stripping HTML/boilerplate.

Step 5 - LLM call: Per-source extraction/summarization

The orchestrator asks the LLM to extract just the relevant facts from each fetched page, so that the final synthesis LLM call will have a context small and focused. Alternatively, one could skip this and just truncate/concatenate raw snippets instead, trading quality for fewer LLM calls.

LLM Input (run once per source, so potentially 4 separate calls here):

System: Extract only the facts relevant to this question from the following article. Be concise.

Question: "What's the current inflation rate in the US..."

Article (BLS.gov): [~2000 words of CPI report text]

LLM Output (per source):

The BLS reports the Consumer Price Index rose 2.7% year-over-year
in July 2026, compared to 3.1% the same month last year.

Step 6 - LLM call: Final synthesis / answer generation

This is the "real" answer-writing LLM call — the one the user experiences as "the response."

LLM Input:

System: Answer the user's question using only the provided source summaries. Cite sources by name inline.

User question: "What's the current inflation rate in the US, and how does it compare to a year ago?"

Source summaries:
[BLS.gov]: CPI rose 2.7% YoY in July 2026, vs 3.1% a year earlier.
[Reuters]: Inflation eased for the third consecutive month, driven by...
[source 3]: ...
[source 4]: ...

LLM Output:

As of July 2026, the U.S. inflation rate (CPI) stands at 2.7% year-over-year, according to the Bureau of Labor Statistics. That's down from 3.1% a year ago, continuing a three-month easing trend reported by Reuters, largely attributed to...

This is the text actually shown to the user.

Tool-use pattern

Many modern chatbots use a different technique, based on the tool-use pattern. This is simpler to build and more flexible: the LLM adapts its own strategy — searching again if results are poor, stopping early if the first search already answers the question.

	Multi-stage pipeline	Tool-use pattern
Number of calls	3-6, mostly fixed	2 to N, dynamic — model decides when it has enough info
Flexibility	Rigid, predetermined steps	Model can decide to search again, fetch a full page, or stop early
Cost control	Easier — can use cheap models for intermediate steps	Harder — every call uses the same (often expensive) capable model
Query generation	Separate dedicated call	Folded into Call 1 (model writes the query itself as part of tool_use)
Relevance filtering	Separate call	Folded into Call 2 (model reads raw results and picks what matters itself)
Per-source extraction	Separate calls (parallel)	Folded into Call 2 (model reads snippets/pages directly)
Final synthesis	Separate call	Same Call 2

Before any conversation happens, the orchestrator defines what tools are available, as a schema (not natural language). This schema is sent alongside every request to the LLM.

{
"name": "web_search",
"description": "Search the web for current information",
"input_schema": {
"type": "object",
"properties": {
"query": { "type": "string", "description": "The search query" }
},
"required": ["query"]
}
}

Step 1 - LLM Call

LLM Input (full message sent to the model):

System: You are a helpful assistant with access to a web_search tool. Use it when you need current information you don't already know.

Tools available: [web_search schema above]

Conversation:
User: What's the current inflation rate in the US, and how does it compare to a year ago?

LLM Output (the model's raw response — note it's not plain text, it's a structured object):

{
"stop_reason": "tool_use",
"content": [
{
"type": "text",
"text": "I'll look up the current inflation data for you."
},
{
"type": "tool_use",
"id": "toolu_01A2b3",
"name": "web_search",
"input": { "query": "US inflation rate 2026 vs year ago" }
}
]
}

Two things happen in this single call: the model produces a short natural-language lead-in and a structured tool-call request, in the same generation pass. The stop_reason: "tool_use" tells the orchestrating code "pause here, I need a tool result before I can continue."

Step 2 - External Command: Execute the search(es)

The orchestrator sees stop_reason: "tool_use", extracts the input.query, and actually calls the search API. Returns something like:

[
{"title": "Consumer Price Index Summary", "url": "bls.gov/...",
"snippet": "CPI rose 2.7% over the 12 months ending July 2026..."},
{"title": "US inflation eases to 2.7% in July", "url": "reuters.com/...",
"snippet": "Inflation continued its downward trend, easing from..."},
{"title": "...", "url": "...", "snippet": "..."}
]

Step 3 - LLM Call

The orchestrator appends the tool result to the same conversation and sends it back to the LLM. The LLM invocation includes the entire prior conversation (the model has no memory between calls — the full history must be resent each time, as covered earlier for the Artifacts API).

LLM Input:

System: You are a helpful assistant with access to a web_search tool...

Conversation:
User: What's the current inflation rate in the US, and how does it compare to a year ago?

Assistant: [tool_use: web_search(query="US inflation rate 2026 vs year ago")]

Tool result (id: toolu_01A2b3):
[
{"title": "Consumer Price Index Summary", "url": "bls.gov/...",
"snippet": "CPI rose 2.7% over the 12 months ending July 2026..."},
{"title": "US inflation eases to 2.7% in July", "url": "reuters.com/...",
"snippet": "Inflation continued its downward trend, easing from..."},
...
]

LLM Output:

{
"stop_reason": "end_turn",
"content": [
{
"type": "text",
"text": "As of July 2026, U.S. inflation (CPI) is 2.7% year-over-year, according to the Bureau of Labor Statistics. That's down from roughly 3.1% a year earlier, continuing a multi-month easing trend reported by Reuters."
}
]
}

This time stop_reason: "end_turn" signals the model is done — no more tool calls needed, this text goes straight to the user.

The loop isn't hardcoded to exactly 2 calls — it repeats until the model stops requesting tools. E.g., if the snippets weren't detailed enough, Call 2's output might itself be another tool_use block (e.g., calling web_fetch on the BLS URL for the full report), producing a Call 3 with that page's content, and so on. Each iteration of this loop is one LLM call. Simple questions resolve in 2 calls (1 tool-request + 1 final-answer); complex research questions might take 5-10 iterations if the model chooses to search multiple times, fetch full pages, or refine queries based on what it finds.

Key differences

The tool-use pattern is simpler to build and more flexible: the LLM adapts its own strategy — searching again if results are poor, stopping early if the first search already answers the question, but it's harder to control cost/latency precisely since you don't know upfront how many iterations it'll take.

Furthermore, every LLM call typically has to use a model capable enough to reason about tool selection — you can't easily swap in a cheap model for "just the filtering step" the way you could in the explicit pipeline.

	Multi-stage pipeline	Tool-use pattern
Query generation	Separate dedicated call	Folded into Call 1 (model writes the query itself as part of tool_use)
Relevance filtering	Separate call	Folded into Call 2 (model reads raw results and picks what matters itself)
Per-source extraction	Separate calls (parallel)	Folded into Call 2 (model reads snippets/pages directly)
Final synthesis	Separate call	Same Call 2
Number of calls	3-6, mostly fixed	2 to N, dynamic — model decides when it has enough info
Flexibility	Rigid, predetermined steps	Model can decide to search again, fetch a full page, or stop early
Cost control	Easier — can use cheap models for intermediate steps	Harder — every call uses the same (often expensive) capable model

Chain-of-Prompting

Chain of Prompting requires an orchestrator software that breaks down a complex problem into a sequence of smaller sub-problems, each addressed in separate runs of the LLM. The orchestrator manages the flow of data between the multiple LLM calls. The output of one run (the answer to a sub-problem) is used as context for the next run, so that the context for each step is small and focused. Steps can be run in parallel (e.g., checking 50 functions independently). Verification checkpoints can be inserted so errors don't silently compound through the chain

General heuristic:

• If the problem is a single chain of logical implications
• CoT is natural and sufficient.
• If the problem is a search over a large space combined with a need to verify hypotheses against ground truth (code execution, a proof checker, a compiler)
• Chain-of-prompting (often paired with tool use) tends to outperform pure CoT.

In practice, sophisticated systems often combine both: chain-of-prompting at the macro level, with a CoT reasoning trace happening within each individual stage.

As an example, consider the task of finding a vulnerability in a codebase. Here, a single linear CoT tends to break down, because the task isn't a deduction chain:

• It is a search problem across a large space (many files, many possible vulnerability classes).
• It needs verification against ground truth (does the exploit actually trigger? what does the compiler/interpreter say?).
• Different parts of the analysis are logically independent (a SQL injection check and a buffer overflow check don't depend on each other's reasoning).

A more effective structure is chain-of-prompting, where each stage is a separate, focused prompt/tool call with its own context:

• Stage 1 — Recon: "List all user-input entry points in this codebase"
  → produces a list of functions/routes.
• Stage 2 — Triage: For each entry point, a separate prompt: "Does this function sanitize its input before using it in [SQL query / file path / shell command]?"
  → flags suspicious ones.
• Stage 3 — Deep dive: For each flagged function, a focused prompt with just that function's code and its call graph: "Trace how user_input flows through this function. Is it possible to inject unescaped content into the query at line 42?"
• Stage 4 — Verification: Feed the hypothesis to a tool (static analyzer, sandboxed execution, or a test harness) to confirm the vulnerability is real, not just plausible-sounding.
• Stage 5 — Report: Synthesize confirmed findings into a report.

Chain-of-Prompting in practice

• Consider chain-of-prompting. How is it implemented in real systems?
• Does a system have a predefined collection of prompt chains and chooses the most suitable for the specific problem?
• If so, how does it identify the most suitable?

Real systems implement chain-of-prompting in a few distinct architectural patterns. Many real systems do maintain a predefined library of chains, with selection typically done via a lightweight classification step (either an LLM call or an embedding-similarity lookup). However the field has been trending toward more dynamic plan generation, where the boundary between "picking a chain" and "writing a new chain" gets blurry.

Static, hand-written chains (no selection needed)

The simplest and still most common approach in production: a developer writes a fixed pipeline for a known task type. There's no "choosing" at runtime because the chain is the application.

Example: a customer-support triage tool might always run:

1. classify intent
2. retrieve relevant docs
3. draft reply
4. check draft against policy.

Every ticket goes through the same fixed sequence. You compose a DAG of prompt/tool steps at design time.

This works well when the task is known in advance and doesn't vary much in shape.

Router / dispatcher pattern (selecting among predefined chains)

In this case the system has a library of chains, each specialized for a task type, and a router decides which one to invoke. The router itself is typically one of:

• A classifier prompt: a lightweight LLM call (often a smaller/cheaper model) that takes the user's request and outputs a label like "code_debugging", "math_proof", "data_analysis", which then maps to a specific chain via a lookup table.
• Embedding similarity / semantic routing: the incoming request is embedded and compared (via cosine similarity) against embeddings of example requests representing each chain's domain — whichever chain's exemplar is closest gets selected. This is fast and doesn't need an extra LLM call, but is less flexible than a classifier prompt.
• Rule-based/heuristic routing: simple keyword or regex matching (e.g., "if the request mentions a filename ending in .sql, invoke the SQL-review chain"). Cheap, predictable, but brittle.

The "chains" are sometimes framed as tools or skills the orchestrator can invoke, by deciding which tool fits the request based on the request itself.

A few common approaches used to build a router:

• Confidence thresholds: if the classifier/router's confidence for any chain is below some threshold, fall back to a generic/default chain (or ask a clarifying question) rather than guessing.
• Cost/latency tradeoffs: cheaper routing methods (keyword rules, embeddings) are used for high-volume/low-stakes routing; more expensive routing (a full LLM classification call) is reserved for ambiguous or high-stakes requests.
• Human-in-the-loop fallback: some systems route to a "none of the above, ask a human or ask for clarification" path when no chain scores well — better than forcing a bad-fit chain.
• Evaluation against labeled examples: teams typically build a small benchmark of "which chain should handle this request?" examples and measure routing accuracy before deploying, since a bad router silently degrades every downstream chain's performance.

Dynamic chain construction (no fixed library at all)

Rather than choosing from predefined chains, many modern agentic systems have the model generate the plan itself at inference time — essentially writing a custom chain of prompts/tool calls specific to the problem, rather than picking one from a fixed menu. The model is asked to decompose the task into steps itself ("planner" LLM call), effectively writing a custom chain on the fly.

1. An initial prompt asks the model to output a step-by-step plan (e.g., "Step 1: find all API endpoints. Step 2: check each for auth. Step 3: ...").
2. Each step is then executed as its own sub-prompt (sometimes by the same model, sometimes handed to a smaller/cheaper model), with results fed back into a shared scratchpad or memory.
3. The plan can be revised mid-execution if a step reveals something unexpected (e.g., "step 3 found no auth vulnerabilities, but discovered a possible SSRF issue — inserting a new investigative step").

This is more flexible than a static chain but harder to guarantee correctness for, since the model can produce a bad plan.

Many production systems mitigate this by combining approaches: a constrained planner that must choose steps from a known set of tools/chains (bounding the space of possible "wrong turns") but can compose them in a novel order.

Instruction Hierarchy

Most modern LLMs, especially in chat or agent use cases, process structured inputs consisting of:

• System Messages define the general instructions, safety guidelines, and constraints for the LLM, as well as tools available to it. They define the expected behavior and the overall control flow. These messages can only be provided by the application developer.
• User Messages are an end user’s inputs to the model.
• Model Outputs refer to responses from the LLM, which may consist of text, images, audio, calls to a tool, and more.
• Tool Outputs may contain internet search results, execution results from a code interpreter, or results from a third-party API query.

Each serves a different purpose and is formatted with special tokens to enable the LLM to delineate between different message types.

The context provided to an LLM may contain multiple conflicting instructions, for example user messages could contain instructions that conflict with the expected behavior described by system messages.

Ideally, a privilege hierarchy should be established for these types of messages: system messages should take precedence over user messages and user messages should take precedence over third-party conten. The LLM should refuse to act in a way that subverts such a hierarchy. This objective is approximated with a specific training (fine tuning).

From a different point of view, the LLM acts as an operating system in which every instruction is executed as if it was in kernel mode, i.e., untrusted third-parties can run arbitrary code with access to private data and functions. Modern LLM are specifically trained to defer to higher-privileged instructions. Training can only approximate this desired behavior, though.

Attacks to an LLM

A typical use case of an LLM product involves up to three parties:

• the application builder, who provides the LLM’s instructions and drives the control flow,
• the main user of the product,
• third-party inputs from web search results or other tool use to be consumed by the LLM as extra context.

Attacks arise when conflicts between these parties arise, e.g., when users or adversaries try to override existing instructions. These conflicts can manifest in different forms:

• Prompt injections, where adversaries insert instructions that subvert the intent of the system designer. Prompt injections do not target the models themselves, but rather the applications built on top of them (e.g., if an application has access to confidential data or can take actions in the world, prompt injections can cause catastrophic harms such as exfiltrating user data or hijacking the LLM’s actions).
• Direct prompt injections occur when the end user of an application provides the injection into the input.
• Indirect prompt injections occur when a third-party input (e.g., from browsing or tool use) contains the prompt injection.
• Jailbreak, which specifically aims to escape the safety behavior that is trained into an LLM. As such, they often don’t specifically conflict with a model’s previous instructions. There are a myriad of variants aimed at performing malicious tasks.
• System Message Extraction The System Message defines the expected behavior of the model, and it may contain well-curated business logic, private information such as passwords, or safety related instructions. The prompt could thus be considered intellectual property, and having knowledge of the prompt can aid in other attacks. System message extraction attacks aim to reveal the entirety of the system message or a specific secret from it.

AI agents - ReAct

AI agents are based on the framework outlined above, with an orchestrator, an LLM, external commands.

Some systems break the request into subtasks that can potentially be executed in parallel by specialized "worker" agents (which might itself be a fixed chain, or another LLM with a narrow prompt/toolset). An orchestrator plans, spawns subagents to research pieces potentially in parallel, then synthesizes their output.

The overall framework is sometimes called ReAct (Reason + Act). The LLM "reasons" to create a "Plan," but it then "Acts" by generating a tool-calling command to obtain information. The overall execution functions as a chain of separate prompts managed by an orchestrator, because ReAct waits for an "Observation" (the result of the tool call) before continuing to the next reasoning step. It relies on specifically aligned models that have been fine-tuned to:

• recognize tool-use scenarios, and
• incentivized to reason through them step-by-step.

The specific training includes the generation of tool markers: standard LLMs are not naturally trained to output the specific markers or command strings (e.g., {tool: calculator}) required to trigger external APIs. Therefore, models are often fine-tuned on annotated data so they "realize" exactly when and how to call a tool.

Feature	Chain of Thought (CoT)	Chain of Prompting (CoP)	ReAct
Model Runs	Single prediction	Multiple separate runs	Multiple rounds (Loop)
External Actions	None (Internal only)	Possible, but focused on sub-tasks	Core requirement(Tool calling)
Primary Goal	Explain logic before answering	Tackle complex tasks step-by-step	Use reasoning to drive actions