BloombergGPT and the Limits of Domain-Specific LLMs in Finance
BloombergGPT trained on 569B financial tokens, yet GPT-4 matched it with no finance pretraining. For accounting agents, tool-use beats model internals.
#finance
Finance
Financial research, analysis, and domain knowledge for accounting AI
AutoGen: Multi-Agent Conversation Frameworks for Finance AI
AutoGen's two-agent conversation lifts MATH accuracy from 55% to 69%, and its SafeGuard agent adds up to 35 F1 points on unsafe-code detection.
MemGPT: 92.5% multi-session recall vs 32.1% baseline
MemGPT's OS-style memory tiers push GPT-4 multi-session chat accuracy to 92.5% versus 32.1% fixed-context—needed for multi-year ledger agents.
Self-Correction: GPT-4 falls 95.5%→91.5% on GSM8K
Huang et al. (ICLR 2024): intrinsic self-correction drops GPT-4 from 95.5% to 91.5% on GSM8K—ledger agents need external validators, not self-review.
CRITIC: Why LLM Self-Correction Requires External Tool Feedback
CRITIC lifts open-domain QA by 7.7 F1 and cuts toxicity 79.2% by verifying LLM revisions against external tools, not itself.
Self-Consistency: Majority-Vote Sampling Improves Chain-of-Thought Accuracy
Self-consistency majority-votes many sampled reasoning paths instead of one greedy decode, adding 17.9 points on GSM8K with no extra training.
PAL: +38pp on hard math by handing arithmetic to Python
PAL gains +38.1pp over chain-of-thought on GSM-hard by running Python for arithmetic—the right split for reliable Beancount ledger calculations.
Can LLMs Reason Over Tabular Data? What Four Benchmarks Tell Us About Finance AI
Four benchmarks show GPT-4 at 42% on real-world table QA versus 86% for humans, and 19.6% on complex aggregations, so finance agents need clean table input.
Chain-of-Thought Prompting: Precision-Recall Trade-offs for Finance AI
Chain-of-Thought prompting raises precision but can cut recall on rare financial events, so fraud agents may miss anomalies they should flag.
PHANTOM (NeurIPS 2025): Measuring LLM Hallucination Detection in Financial Documents
PHANTOM (NeurIPS 2025) measures LLM hallucination detection on real SEC filings. Qwen3-30B leads at F1=0.882, but 7B models guess near random.
FinBen: Benchmarking LLMs Across 36 Financial Tasks — Implications for Accounting AI
FinBen finds GPT-4 at 0.63 exact match on FinQA and 0.54 on stock forecasting, barely above random, so accounting agents need validation, not raw LLM math.