OpenHands: Open Platform for AI Software Agents and What It Means for Finance Automation
OpenHands' CodeAct agent scores 26% on SWE-Bench Lite, showing what AI agents reliably do today. Finance automation should start tightly scoped, not autonomous.
#beancount
Beancount
Beancount ledger format, tooling, and ecosystem research
FinDER: Real Analyst Queries Expose a 74% Recall Gap in Financial RAG
FinDER's 5,703 real analyst queries show top RAG recalls only 25.95% of 10-K evidence. Normalize abbreviations first, before swapping embeddings.
CausalTAD: Causal Column Ordering for LLM Tabular Anomaly Detection
CausalTAD reorders table columns by causal dependency before LLM serialization, raising average AUC-ROC from 0.803 to 0.834 over AnoLLM.
AnoLLM: Fine-Tuning LLMs for Tabular Anomaly Detection in Financial Data
AnoLLM beats classical baselines on mixed-type fraud data by scoring rows with LLM negative log-likelihood, but adds no edge on purely numerical tables.
LLMs Score 2.3% on Beancount DSL Generation: The LLMFinLiteracy Benchmark
LLMFinLiteracy finds five ~7B models write correct Beancount transactions just 2.3% of the time, failing on accounting reasoning rather than syntax.
TableMaster: Adaptive Reasoning for Table Understanding with LLMs
TableMaster hits 78.13% on WikiTQ with GPT-4o-mini, 13 points over Chain-of-Table, via table-of-focus plus adaptive reasoning for agents over Beancount ledgers.
Zero-Shot Anomaly Detection with LLMs: How GPT-4 Performs on Tabular Data
GPT-4 reaches 74.1 mean AUROC on ODDS zero-shot, near the 75.5 ECOD baseline, but fails on high-variance data. Ledger auditing is not solved yet.
DocFinQA: Long-Context Financial Reasoning on Full SEC Filings
DocFinQA swaps FinQA's 700-word passages for full SEC filings, a 175× longer context that nearly halves GPT-4 accuracy on long documents.
τ²-bench: Measuring the Cost of Dual-Control in Conversational AI Agents
τ²-bench finds active users with their own tools cut conversational agent success by 18–25 points. Beancount agents sharing write access lose the same way.
GAIA Benchmark: Measuring What Frontier AI Agents Can Actually Do
GAIA's 466 tasks show frontier AI agents at 74.55% versus 92% for humans, with Level 3 coordination still the hardest gap to close.
WebArena: The 812-Task Benchmark That Measures What Web Agents Actually Can and Cannot Do
GPT-4 finishes just 14.41% of WebArena's 812 web tasks versus 78.24% for humans; agents mostly fail by falsely declaring a task impossible.
WorkArena: How LLM Web Agents Perform on Real Enterprise Knowledge Work
WorkArena's 33 ServiceNow tasks show GPT-4o at 42.7% overall but 0% on list filters, a wall structured UI interaction must clear.