GraphRAG: 72–83% wins vanish after LLM-judge bias fix
Microsoft's GraphRAG posts 72–83% sensemaking wins over vector RAG; a 2025 audit collapses those after correcting judge bias—caution for multi-doc ledger QA.
#machine-learning
Machine Learning
Machine learning techniques for financial data analysis and automation
FinAuditing: LLMs Score Under 14% on Real SEC XBRL Auditing Tasks
FinAuditing shows top LLMs hit just 13.86% on financial math verification of real SEC XBRL filings, capping what AI accounting tools can automate unaided.
InvestorBench: Qwen2.5-72B tops stocks at 46.15% CR
InvestorBench: Qwen2.5-72B leads stock trading at 46.15% CR; finance-tuned Palmyra-Fin backfires on equities—size beats domain fine-tuning.
StructRAG (ICLR 2025): Picking the Right Document Structure Beats GraphRAG by 28 Points
StructRAG routes each query to a table, graph, catalogue, algorithm, or chunk structure, beating GraphRAG by 28 points and running 22× faster.
Single-Agent: equals MAS on multi-hop at equal tokens
Under equal thinking-token budgets, single-agent LLMs match or beat multi-agent systems on multi-hop reasoning—favor simpler finance agent designs.
M3MAD-Bench: Are Multi-Agent Debates Really Effective Across Domains and Modalities?
M3MAD-Bench finds Collective Delusion drives 65% of multi-agent debate failures, and adversarial debate cuts accuracy by up to 12.8%.
AGrail: Adaptive Safety Guardrails for LLM Agents That Learn Across Tasks
AGrail's two-LLM guardrail cuts prompt injection attack success to 0% while preserving 95.6% of benign agent actions on Safe-OS.
ShieldAgent: Verifiable Safety Policy Reasoning for LLM Agents
ShieldAgent hits 90.4% accuracy on agent attacks with 64.7% fewer API calls by using probabilistic rule circuits instead of LLM guardrails.
Atlas: Joint Retriever-Reader Pre-Training Beats 540B-Parameter LLMs with 11B Parameters
Atlas hits 42.4% accuracy on Natural Questions with 64 examples, beating PaLM 540B by 3 points at 11B parameters via joint retriever-reader pre-training.
Fusion-in-Decoder: How Multi-Passage Retrieval Improves Generative QA
FiD encodes retrieved passages separately and fuses them in the decoder, beating RAG-Sequence by 4–11 points on NQ and TriviaQA, the synthesis ledger QA needs.
GuardAgent: Deterministic Safety Enforcement for LLM Agents via Code Execution
GuardAgent enforces LLM agent policies by running Python code, hitting 98.7% accuracy with no task failures, versus 81% and up to 71% failure for prompt rules.
Multiagent LLM Debate: Real Accuracy Gains, Uncontrolled Compute, and Collective Delusion
Multiagent debate gained 14.8 points on arithmetic, but equal-budget single agents match it. That limits debate as a safe check before a ledger commit.