LLM Confidence and Calibration: A Survey of What the Research Actually Shows
Verbalized GPT-4 confidence hits only ~62.7% AUROC, barely above chance. Uncertainty-aware finance agents need better calibration than that.
#trust
Доверие
Надеждност, калибриране и халюцинации във финансови ИИ системи
Uncertainty-Aware Deferral for LLM Agents: When to Escalate from Small to Large Models
ReDAct defers from a small model to a large one only when perplexity signals uncertainty, cutting cost 64% at matching accuracy for agent workflows.
Verifiably Safe Tool Use for LLM Agents: STPA Meets MCP
STPA plus capability-enhanced MCP yields formal safety specs for LLM tool use, with Alloy proving no unsafe flows in a calendar case study.
AGrail: Adaptive Safety Guardrails for LLM Agents That Learn Across Tasks
AGrail's two-LLM guardrail cuts prompt injection attack success to 0% while preserving 95.6% of benign agent actions on Safe-OS.
ShieldAgent: Verifiable Safety Policy Reasoning for LLM Agents
ShieldAgent hits 90.4% accuracy on agent attacks with 64.7% fewer API calls by using probabilistic rule circuits instead of LLM guardrails.
GuardAgent: Deterministic Safety Enforcement for LLM Agents via Code Execution
GuardAgent enforces LLM agent policies by running Python code, hitting 98.7% accuracy with no task failures, versus 81% and up to 71% failure for prompt rules.
Self-Correction: GPT-4 falls 95.5%→91.5% on GSM8K
Huang et al. (ICLR 2024): intrinsic self-correction drops GPT-4 from 95.5% to 91.5% on GSM8K—ledger agents need external validators, not self-review.
PHANTOM (NeurIPS 2025): Измерване на откриването на халюцинации в LLM при финансови документи
PHANTOM (NeurIPS 2025) измерва откриването на халюцинации в реални SEC документи. Qwen3-30B води с F1=0.882, но 7B моделите гадаят почти случайно.