PHANTOM (NeurIPS 2025): Measuring LLM Hallucination Detection in Financial Documents
PHANTOM (NeurIPS 2025) measures LLM hallucination detection on real SEC filings. Qwen3-30B leads at F1=0.882, but 7B models guess near random.
#ai
AI
Artificial intelligence research and applications in finance and accounting
FinMaster Benchmark: Why LLMs Score 96% on Financial Literacy but 3% on Statement Generation
FinMaster Benchmark: top LLMs hit 96% on financial literacy but only 3% on statement generation. Error propagation costs 21 points on consulting tasks.
ReAct: Synergizing Reasoning and Acting in Language Models
ReAct interleaves reasoning with tool actions, beating pure CoT by 34 points on fact verification. Its failure modes shape agents writing to Beancount ledgers.
Toolformer: Self-Supervised Tool Use and Its Limits for Finance AI
Toolformer teaches a 6.7B model to call APIs via perplexity filtering, beating GPT-3 175B on arithmetic. Its single-step design blocks chained ledger calls.
FinBen: Benchmarking LLMs Across 36 Financial Tasks — Implications for Accounting AI
FinBen finds GPT-4 at 0.63 exact match on FinQA and 0.54 on stock forecasting, barely above random, so accounting agents need validation, not raw LLM math.