How AI Agents Think: Planning, Memory, and Tool Use Explained

What is the ReAct pattern and why does it matter?

ReAct — short for Reason + Act — is the foundational framework published by Yao et al. at Google and Princeton (2022) that showed language models perform dramatically better on multi-step tasks when they interleave reasoning traces with actions, compared to acting without explicit reasoning. In benchmark tests across HotpotQA, FEVER, and ALFWorld, ReAct agents outperformed pure chain-of-thought and pure action baselines by meaningful margins.

In practice, each step of the loop looks like this in the model's output stream:

1. Thought: I need to find the current population of Tokyo to answer the user's question. I'll search for it.
2. Action: search("Tokyo population 2024")
3. Observation: Tokyo metropolitan area population is approximately 37.4 million as of 2024.
4. Thought: I have the number. The user's question is answered. I can stop.
5. Final Answer: Tokyo's metropolitan population is approximately 37.4 million.

The reasoning step is not decoration — it makes the agent's tool selection dramatically more accurate and gives the system a natural place to catch its own mistakes before committing to an action.

How do agents use chain-of-thought to plan?

Chain-of-thought (CoT) prompting asks the model to show its work — to write out intermediate reasoning before committing to a conclusion. For agents, CoT serves as the planning layer. Before calling any tool, the agent narrates its intentions: 'I have three sub-tasks here. I'll handle them in order: first get the data, then calculate the metric, then format the report.'

More sophisticated agents use Plan-and-Execute: the model first generates a complete task list, then executes each step, revisiting the plan when new information changes the picture. This is more reliable for tasks with 10+ steps because the agent commits to a structure rather than improvising at every turn. LangGraph supports this pattern natively through its graph-based state machine.

When planning breaks down

Planning fails most often in three situations: the goal is ambiguous (the model plans for the wrong objective), the model over-estimates its own knowledge (it skips a tool call it should make), or the plan is too rigid (new information contradicts step 3 but the agent plows ahead anyway). Defensive agent design adds a replanning step after every N actions or after any unexpected observation.

How does an agent decide which tool to call?

Tool selection happens inside the reasoning step. The model reads its tool schema — a list of available functions with names, descriptions, and parameter types — and picks the tool whose description best matches what it needs next. This means the quality of a tool's description is as important as the tool's actual implementation.

A poorly described tool gets ignored or misused. A rule of thumb: write tool descriptions the way you'd explain a function to a new engineer. Include what the tool does, what inputs it expects, what it returns, and when NOT to use it. For example:

• Bad: `search(query)` — searches the web.
• Good: `web_search(query: str) -> str` — performs a live web search and returns the top 5 result snippets. Use when you need current information not in your training data. Do not use for mathematical calculations or code execution.

What are the types of memory an agent can use?

Memory is the component that most distinguishes a capable agent from a brittle one. There are four types worth knowing:

In-context memory

This is the agent's active working memory — everything in the current token window. It's fast, but token limits mean it can't hold more than a few dozen tool call results before older information gets pushed out. Most agents manage this by summarizing or truncating older turns.

Working / scratch-pad memory

A structured data store the agent writes to and reads from within a session — for example, a JSON object tracking which sub-tasks are done. LangGraph agents store this in the graph state object, which persists across nodes.

Episodic memory

A log of past events: 'At step 4, I searched for X and got an empty result.' Episodic memory prevents the agent from repeating the same failed action in the same session. It can also persist across sessions so the agent doesn't retry strategies it already knows don't work.

Semantic (vector store) memory

Long-term facts stored as embeddings in a vector database. The agent retrieves relevant chunks via similarity search before each step. This allows agents to work with knowledge bases far larger than any token window — millions of documents — by fetching only what's relevant to the current reasoning step.

Why do agents sometimes loop or hallucinate?

Looping happens when the agent's termination condition is too weak. If the model doesn't confidently recognize that the goal is met, it keeps generating new actions. Common fixes: add an explicit 'FINAL ANSWER' token the runtime watches for, set a hard step-count ceiling, or include a separate evaluator agent that checks whether the goal is satisfied after each round.

Hallucination in agents is subtler than in a plain chatbot. The agent may hallucinate a tool call (invoking a function that doesn't exist), hallucinate tool output (inventing the result of a search it didn't run), or hallucinate a planning step (claiming to have done step 3 when it skipped it). The fix is strict tool call validation — the runtime should reject any tool invocation whose schema doesn't match and return an error observation, forcing the agent to self-correct.

For a deeper look at the memory side of this problem, AI Agent Memory Systems goes into implementation detail for each memory type.