I've been thinking about what comes after Tree of Thoughts — if you can search over reasoning steps, why not search over actions too? That is exactly what LATS (Language Agent Tree Search) does, and why I'm reading it now. The paper by Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang (ICML 2024, arXiv:2310.04406) is the clearest synthesis yet of reasoning, acting, and planning in a single agent framework, and the results are genuinely hard to dismiss.
The paper
The core problem LATS addresses is a structural gap in prior agent work. ReAct interleaves reasoning and acting, but has no mechanism to reverse and try a different path when a trajectory goes wrong. Tree of Thoughts enables branching over reasoning steps, but operates on internal LM knowledge — it cannot call tools or receive external feedback during the search. Reflexion adds verbal self-correction, but its linear retry loop commits to a new trajectory without exploring alternatives. LATS fuses all three ideas with a proper Monte Carlo Tree Search (MCTS) backbone, letting LLM agents explore multiple branches, receive real environment feedback, and backtrack when a path fails.
The technical machinery is a six-step MCTS loop: Selection (choose the next node to explore via the UCT formula), Expansion (sample n candidate actions from the LM), Evaluation (score each node with a hybrid value function), Simulation (roll out to a terminal state), Backpropagation (update ancestor values), and Reflection (on failure, generate a verbal summary of what went wrong and store it as context). The value function deserves attention: V(s) = λ·LM(s) + (1−λ)·SC(s), where LM(s) is the LM's own estimate of trajectory quality after receiving environment feedback, and SC(s) is a self-consistency score based on how often that action is sampled across sibling nodes. This is not a trained reward model — the value function is entirely prompt-driven.
Key ideas
- On HumanEval, GPT-4 + LATS reaches 92.7% pass@1, versus 91.0% for GPT-4 + Reflexion and 56.9% for GPT-3.5 + ReAct alone. GPT-3.5 + LATS jumps to 83.8%.
- On HotPotQA, LATS (CoT + ReAct) reaches 0.71 Exact Match versus 0.32 for the ReAct baseline — more than doubling multi-hop accuracy.
- On WebShop (web navigation + purchase), LATS scores 75.9 (38.0% success) versus Reflexion at 64.2 (35.0%) — a meaningful gap on a task that requires managing state across many pages.
- On Game of 24 (a pure reasoning puzzle), LATS reaches 0.44 success versus ToT's 0.20, despite using the same GPT-4 backbone.
- Surprisingly, LATS expands fewer nodes to find a solution than ToT (average 66.65 vs. 84.05 nodes on HotPotQA at k=50) and uses fewer tokens (173,290 vs. 210,215), even though it looks more expensive in theory.
What holds up — and what doesn't
The benchmark numbers are real and the framework is conceptually clean. The UCT formulation provides a principled exploration–exploitation tradeoff that ToT's ad-hoc BFS/DFS lacks. Integrating external environment feedback into the value function — instead of pure LM introspection — is the right move, and the results show it.
But the paper carries a critical assumption that the authors acknowledge without fully stress-testing: LATS requires the ability to revert the environment to a prior state. Without checkpointing, you cannot branch the tree — once an action is taken, you are committed. The authors note that for LM tasks this is often manageable by "copy-pasting historical text inputs," but for real action environments (databases, filesystems, APIs with side effects) this is a hard requirement many production systems cannot meet. The WebShop results, while better than the baselines, show that in complex environments self-reflections tend to become generic rather than specific — agents can stall and repeat superficially different but structurally identical mistakes. The paper notes this but offers no remedy.
There is also no ablation isolating the contribution of the MCTS structure versus the value-function design. It is plausible that a simpler branching approach with the same hybrid value function would close much of the gap, and the authors do not test this directly.
Why this matters for finance AI
Beancount ledgers are an almost ideal environment for LATS-style tree search for one main reason: every ledger is backed by a git repository. The state-reversion requirement — the hard constraint that makes LATS impractical in many real settings — is trivially satisfied by git checkout or git stash. A write-back agent could propose candidate journal entries across multiple branches, score them against balance-sheet constraints (the value function), and commit only the highest-scoring path. Failed branches get a verbal reflection: "The posted entry violated Assets = Liabilities + Equity because the account type was misclassified."
The hybrid value-function design is directly applicable too. For a ledger agent, LM(s) would score a proposed entry on semantic fit (does this look like the right category?), while SC(s) would track how consistently the agent classifies similar past transactions — a natural self-consistency check rooted in the ledger's own history.
State reversion is the one place I would push back on the finance analogy. Real ledgers often have downstream effects: a posted entry triggers an invoice that triggers a payment workflow. In those cases LATS's assumption breaks. Specifically for Beancount, where the ledger is a plain-text file under git control and changes happen locally before any downstream trigger fires, the assumption holds — but this is a design constraint to keep explicit.
What to read next
- MCTS-based planning without environment models: "Reasoning with Language Model is Planning with World Model" (Hao et al., 2023, arXiv:2305.14992) — RAP, which LATS builds on and improves.
- How well does the LM value function generalize? "Let's Verify Step by Step" (Lightman et al., 2023, arXiv:2305.20050) — process reward models as an alternative to prompt-based value functions.
- Safe multi-step planning under irreversibility: "Decision-Making with Language Models via Successive Prompting" (Creswell et al., 2023) — a simpler planning approach that avoids the state-reversion requirement.
