One recorded agent run took a synthetic three-row ledger from a bank CSV to checked books. The opening balance was 1000 USD, the statement held three rows, and the expected answers were derived from arithmetic before any report ran: checking ends at 2958.50 USD, September profit is 1958.50 USD. The run hit those numbers exactly, after one real failure it diagnosed itself.
Inputs
The challenge is three published files under /downloads/agent-accounting/, with the full scenario in the agent accounting guide. Books open on 2026-09-01 in USD with 1000 USD in Assets:Checking. The statement carries three September rows: a 2000.00 USD client payment on 2026-09-02, a -29.00 USD hosting bill on 2026-09-03, and a -12.50 USD cafe visit on 2026-09-04. The rules map client to Income:Consulting, hosting to Expenses:Software, and cafe to Expenses:Dining, and every account the rules name is opened in the starter ledger at the required date.
The expectations come from the rows plus the opening, independently of any report output: 1000 + 2000 - 29 - 12.50 is 2958.50 USD of checking, and 2000 - 29 - 12.50 is 1958.50 USD of September profit. A changed input or a wrong expectation fails verification instead of printing a plausible-looking report.
Methodology
Two layers, kept separate. First, an installed-CLI verifier (scripts/check-agent-accounting.py, pinned to bea 0.1.0) runs preview, apply, repeat apply, check, balance and income-statement queries in a fresh temporary directory with isolated config. It asserts preview reports 3 ready and writes nothing, the first apply writes 3 entries, the repeat apply reports 0 ready with 3 exact duplicates and writes nothing, byte identity survives preview and rejected writes, check is clean, and both totals match the arithmetic above. It also attempts an unbalanced transaction (Assets:Checking -5 USD against Expenses:Dining 4 USD), requires exit 1, and verifies the ledger bytes are unchanged.
Second, one real agent run on a fresh copy of the downloads. The client was muse 1.1.1 (Muse Code) with model muse-spark-1.3-contributor, launched headless with shell and file-write tools inside the workdir, web tools disabled, and no human interventions during the run. Critically, no rules.toml was supplied — the agent derived its own categorization from the ledger's open accounts.
Observed results
The agent made 22 tool calls (21 shell plus 1 file write) and exited 0. It inspected the CSV and the 16 open accounts, read the CLI help surface, wrote its own rules.toml (literal capitalized matches, equivalent to the canonical patterns because matching is case-insensitive), previewed 3 ready with nothing written, applied 3 entries, and ran a clean check. Its reported totals were 2958.50 USD in checking and 1958.50 USD of September profit. The operator re-ran both read-only queries against the agent's ledger afterwards and confirmed the same figures against the independent expected values.
The short demo below re-enacts that working path deterministically from the canonical downloads. It is a replay, not the live run:
$ bea --version
bea 0.1.0
$ bea import (preview)
csv → Assets:Checking: 3 ready, 0 exact duplicates, 0 possible duplicates
$ bea import --apply
Wrote 3 entries to $WORK/books/main.bean.
$ bea check
$WORK/books/main.bean: no errors
$ bea balance Assets:Checking
Checking 2,958.50 USD
$ bea report income-statement -t 2026-09
Net Profit: 1,958.50 USD
checking: 2958.50 USD (expected)
profit: 1958.50 USD (expected)
REPLAY OK: preview 3/0, applied 3, check clean, 2958.50 / 1958.50 USD.Failures and interventions
The transcript keeps the original sequence, failures included. Two exploratory calls failed harmlessly (pip was not installed; an editable-install path probe found sources outside site-packages) and the agent moved on. One failure was real: the first --apply died with Operation not permitted on a cache lock outside the workspace, because the sandbox blocked lock files under the home cache. The agent searched the product sources, found that the cache directory honors XDG_CACHE_HOME, re-ran with it pointed inside the workdir, wrote the same 3 entries the preview had shown, and removed the temporary cache afterwards. No ledger was ever edited by hand; every entry came from bea import --apply. Interventions during the run: none — it was headless with approval off, and a human reviewed the event log only after the fact.
Limitations
This is CLI-output verification of one run of one client on synthetic data — not a model benchmark. It claims nothing about other clients, about accounting accuracy in general, or about unattended production use. Two distinctions matter. First, category judgment versus structural validation: the agent chose which account each row belongs to, and that judgment is only as good as its reading of three unambiguous rows. Everything bea verified afterwards — the balance, the zero-sum structure, the duplicate detection — is structural: a passing check proves the ledger balances, never that Expenses:Dining was the right account for the cafe. Second, the ledger is a toy: three rows, one currency, no categorization ambiguity, no conflicting history. A harder statement would test judgment; this one tests the loop.
Downloads
All challenge inputs and the full run record, as static files shared by every locale:
- main.bean — starter ledger, validates as published
- statement.csv — the three synthetic rows
- rules.toml — deterministic categories
- agent-run.transcript.md — the actual prompt, full tool sequence with failures kept, and independent verification
- demo-replay.sh — labeled deterministic replay of the working path (needs
bea 0.1.0on PATH) - demo-replay.txt — captured replay output with caption, the text alternative to watching the demo
Reproduce it: download the three inputs, preview first, apply, and check the two totals against 1000 + 2000 - 29 - 12.50. The agent guide walks the same steps by hand.
