We gave Claude access to our corporate QuickBooks. It committed accounting fraud.
LLMs are on the verge of replacing data scientists and investment bankers. But can they perform simple accounting tasks for a real business?
The answer is no.
We built AccountingBench, a test where LLMs must "close the books" for a real SaaS business using 1 year of @stripe, @tryramp, @mercury, and @Rippling data:
Millions of accountants do this every month, making sure internal records match external reality across every account.accounting.penrose.com