Self-Correction: GPT-4 falls 95.5%→91.5% on GSM8K
Huang et al. (ICLR 2024): intrinsic self-correction drops GPT-4 from 95.5% to 91.5% on GSM8K—ledger agents need external validators, not self-review.
#machine-learning
Machine Learning
Machine learning techniques for financial data analysis and automation
Tree of Thoughts: Deliberate Problem Solving with LLM Search
Tree of Thoughts hits 74% on Game of 24 where GPT-4 chain-of-thought gets 4%, by searching and backtracking over steps, a pattern finance agents can borrow.
CRITIC: Why LLM Self-Correction Requires External Tool Feedback
CRITIC lifts open-domain QA by 7.7 F1 and cuts toxicity 79.2% by verifying LLM revisions against external tools, not itself.
Reflexion: Language Agents That Learn from Mistakes Without Retraining
Reflexion hits 91% on HumanEval with GPT-4 without weight updates, but fails on WebShop. Verbal reinforcement needs a crisp evaluator signal.
Self-Consistency: Majority-Vote Sampling Improves Chain-of-Thought Accuracy
Self-consistency majority-votes many sampled reasoning paths instead of one greedy decode, adding 17.9 points on GSM8K with no extra training.
PAL: +38pp on hard math by handing arithmetic to Python
PAL gains +38.1pp over chain-of-thought on GSM-hard by running Python for arithmetic—the right split for reliable Beancount ledger calculations.
Constitutional AI for Accounting Agents: RLAIF, Policy Rules, and Goodharting Risks
Constitutional AI replaces hand-labelled harm data with AI feedback, so accounting agents can enforce ledger rules without a reviewer per transaction.
Chain-of-Thought Prompting: Precision-Recall Trade-offs for Finance AI
Chain-of-Thought prompting raises precision but can cut recall on rare financial events, so fraud agents may miss anomalies they should flag.
PHANTOM (NeurIPS 2025): Measuring LLM Hallucination Detection in Financial Documents
PHANTOM (NeurIPS 2025) measures LLM hallucination detection on real SEC filings. Qwen3-30B leads at F1=0.882, but 7B models guess near random.
FinMaster Benchmark: Why LLMs Score 96% on Financial Literacy but 3% on Statement Generation
FinMaster Benchmark: top LLMs hit 96% on financial literacy but only 3% on statement generation. Error propagation costs 21 points on consulting tasks.
ReAct: Synergizing Reasoning and Acting in Language Models
ReAct interleaves reasoning with tool actions, beating pure CoT by 34 points on fact verification. Its failure modes shape agents writing to Beancount ledgers.
Toolformer: Self-Supervised Tool Use and Its Limits for Finance AI
Toolformer teaches a 6.7B model to call APIs via perplexity filtering, beating GPT-3 175B on arithmetic. Its single-step design blocks chained ledger calls.