ReAct: close the loop between actions and observations
Follow an external-evidence loop, then add operational limits that a research demonstration does not automatically provide.
ReAct: Synergizing Reasoning and Acting in Language Models · Yao et al. · arXiv:2210.03629
Original editorial analysis by Token Lab · Source checked 2026-10-25 · Paper claims and synthetic examples are separated.
01 · What the paper establishes
ReAct interleaves reasoning traces and actions so an agent can update its approach using external observations. The paper evaluates question answering, verification and interactive environments. Tool interaction is part of the method; a purely textual plan does not reproduce the same setting.
Primary source: paper abstract ↗Study guide & lesson outline
LEARNING CONTRACTUnderstand → implement → investigate
Goal: explain the mechanism in this lesson, reproduce one example and test a failure case. Reading alone does not mark mastery.
- Understand: state assumptions, shapes/units and a counterexample.
- Implement: reproduce a numerical example and inspect boundaries.
- Investigate: compare one alternative under a matched budget.
02 · The mechanism
Text / 原始公式
Observation → next-step decision → action → new observation Operational extension (not a paper guarantee): continue only if calls < budget AND action ∈ allowlist stop on success, budget exhaustion or unrecoverable failure
An action changes what evidence becomes available. The next decision should depend on the returned observation, including failure, rather than blindly replaying a prewritten plan. A readable trace may help inspection but is not proof that all relevant evidence was used correctly. Execution success should be checked from the external task state.
Notation and intuition are teaching explanations. Consult the original method for complete assumptions and implementation details.
Mechanism map & worked derivation
| Symbol | Meaning |
|---|---|
| sₜ | Observed task context, not necessarily full environment state |
| aₜ | Selected tool action with explicit arguments |
| oₜ₊₁ | Returned observation, including errors |
- Write a transition contract for every tool: input schema, output schema, timeout behavior, side effects and idempotency. This makes the loop testable without a language model.
- A two-call task cannot complete with a one-call hard budget. After lookup, the system should record budget_exhausted rather than inventing verification evidence.
- If a write times out, its commit state is unknown. The next action should check state or use an idempotent retry, not assume nothing happened.
Separate mock control-flow tests from real-policy evaluation. A readable trace is not proof of external task completion.
Change one variable. Explain what moves.
Original teaching calculation, not a reproduction of the paper’s benchmark. Bars are relative within this panel; numeric values and units are authoritative.
04 · Work through an example
Original deterministic mock: a task needs a lookup followed by a verification call. With budget 1, the loop must stop after lookup; with budget 2, it can reach verification. The demo reveals this state progression without calling a model or network. Raising a budget makes more actions possible, but it does not itself improve the quality of the action policy.
05 · Design your experiment
Build a deterministic tool mock with success, empty result, timeout and malformed-output cases. Record action arguments, observation summaries, termination reason and side effects. For a model experiment, compare prompting methods with the same tools, task set and action budget. Score external task completion, cost and unsafe-action attempts separately.
State one hypothesis; save the configuration and raw results; identify a competing explanation; report the narrowest supported conclusion. Use the linked lab as a local starting point, not a full paper reproduction.
Open related lesson / lab →06 · Limits & reviewer questions
Editorial caution: allowlists, idempotency, approvals and hard budgets are engineering extensions, not safety guarantees established by ReAct. An untrusted tool response must not acquire authority merely by appearing in the context. A mock trace tests control flow, not the competence of an LLM.
If a tool returns a timeout after a write, should the agent immediately retry?
Not without knowing whether the write committed. Use an idempotency key or query external state; otherwise a retry can duplicate the side effect. This is an operational requirement beyond the prompting pattern.
07 · Read the original
Reasoning/action trajectories, task-specific interfaces and comparisons with prompting baselines.
Year is the first arXiv submission year. Unversioned links follow revisions; record the exact version in your reproduction report. DeepSeek-R1 here is explicitly pinned to v1.