How AI Agents Work: The Complete Mechanism
The Core Loop: ReAct
Most agents run ReAct: Reasoning + Acting. Loop:
Goal: "Find cheapest supplier for X"
1. Thought: Need to search suppliers
2. Action: web_search("wholesale X supplier")
3. Observation: 3 results returned
4. Thought: Compare prices
5. Action: open_url(result 2)
...
Until confidence > threshold or max steps reached
4 Mechanisms Explained
1. Tool Calling (Function Calling)
LLM outputs JSON like {"tool":"send_email","args":{...}}. Framework (LangChain, OpenAI) executes real API. Critical: tool descriptions matter more than model size.
2. Memory Systems
- Short-term: Last 10-20 turns in context window
- Long-term: Vector DB (Pinecone, Chroma) storing embeddings of past runs
- Working memory: Scratchpad for current plan
3. Planning
Techniques: Chain-of-Thought, Tree-of-Thoughts, Reflexion (agent critiques own output). For complex tasks, agents split into sub-agents (CrewAI).
4. Guardrails
Max loops, cost caps, human-in-the-loop approval for risky tools (send email, charge card).
Architecture Diagram (Build This)
User Goal → Planner (LLM) → Tool Router → Tools (Search, Code, Email, DB) → Observation Store → Evaluator → Loop or Finish. With sidecar: Memory DB + Logging
Why Agents Fail: 3 Science Reasons
1) Context overflow → forgets earlier step. Fix: summarize memory. 2) Tool error not handled → hallucinated success. Fix: explicit error observation. 3) Reward hacking → stops early. Fix: define done criteria.
Read next: Myths & Mistakes | Costs