CUSTOMER ACCOUNT / ATTRIBUTED EVIDENCE
What this case establishes
This is a commercial publisher’s customer account. Roseram checked the source and wrote the analysis; it did not interview the customer, run the product, audit the figures or establish that AI caused the reported business result. The pilot below is a proposal, not a completed experiment.
Primary material reviewed 2026-10-12. Analysis and proposed evaluation by Roseram Editorial.- 01Synthetic workbook and rules
- 02Traceable agent edits
- 03Recalculated reference comparison
Basis: the documented workflow
OpenAI’s September 2026 account describes Basis evaluating an accounting agent on a 50-tab tax workbook and adapting reasoning effort to the task. It reports a faster completion and an internal evaluation improvement for the compared model route. These are scoped internal results, not a universal productivity claim for accountants or a verified guarantee of correct tax treatment.
The mechanism worth examining
A workbook is a network of inputs, formulas and linked outputs. Editing one cell can affect a summary several sheets away. The useful agent must preserve that dependency structure and explain its changes. A completed file with no visible errors can still contain a broken reference, overwritten formula or assumption that does not match the supplied task.
The implementation decision
Use a synthetic workbook with a known answer and no real taxpayer information. Define the permitted edits, required reconciliations and protected formulas. Keep model interpretation separate from deterministic arithmetic checks. Domain-specific treatment needs qualified review; the pilot tests workbook handling and traceability rather than providing tax advice.
Where the workflow can fail
An agent can hardcode a total, replace a formula with a value or use a stale cached result. A partial repair may make one sheet balance while breaking another. Record changed cells and recalculate in the intended spreadsheet engine. Include a contradictory input and a hidden dependency so the test rewards identifying uncertainty rather than forcing a balanced-looking output.
A practical evaluation, designed for this use case
Compare the recalculated workbook with the reference, checking both values and formulas. Count broken links, unsupported assumptions, review minutes and total request cost. Repeat after changing a key input to see whether the workbook still propagates correctly. A model comparison should name the specific task set, effort and acceptance criteria instead of presenting a general speed multiplier.
Primary sources
Figures and product details above are dated and attributed; they are not current guarantees.
- OpenAI: Basis primary account ↗The attributed workflow and reported result. Proposed pilots and decisions below are original Roseram analysis.
The decision matrix
Original evaluation criteria for this workflow. These are questions to test, not results measured by Roseram.
| Decision | Evidence to collect | Stop or revise when |
|---|---|---|
| Dependencies | Cross-sheet formulas retained | A local repair breaks downstream totals |
| Verification | Recalculation and reference checks run | Cached display values pass as correctness |
| Scope | Task and effort documented | One benchmark becomes a universal claim |
YOUR EVALUATION / NOT A REPORTED RESULT
A small pilot you can inspect
Use these steps as a planning checklist. Checking a box records your progress in this view; it does not run or validate an experiment.
Questions this case answers
Does an internal benchmark guarantee correct tax work?
No. It is scoped evidence and does not replace qualified review.
Why test a changed input after completion?
It exposes hardcoded totals and broken dependency chains that a single output comparison may miss.
Tools & capabilities referenced
Operators studying this story often evaluate adjacent AI products. Explore Roseram guides where available:
PRACTICAL NEXT STEPS
Take the evidence into your own project.
Calculate the whole project cost
Compare subscriptions, usage, launch work and renewals.
Test an existing-project workflow
Check import, source loading, preview and publication.
Verify the output you buy
Require evidence tied to a real customer task.
Wiki brief
Basis is examined here for multi-sheet accounting workbook verification. The article answers “How can AI agents handle complex accounting spreadsheets reliably?” using linked primary material and original evaluation design. It is not a customer testimonial for Roseram or an endorsement by Basis.
Operator takeaways
- Protect formula structure.
- Test dependency propagation.
- Report benchmark scope.
Related case studies
Plan → build → review with routed models and transparent usage.