Composite scenario, not a real company. Assembled from common industry patterns; every number is illustrative and its assumptions are listed at the end.
This case belongs to Agents, Tools and MCP.
Context
A regional parts distributor receives about 8,000 supplier invoices a month, mostly PDFs attached to emails and some scans. Accounts payable clerks read each one, key the header, line items and tax into the ERP, and match it to a purchase order. A clerk spends about 6 minutes per invoice. The ERP's posting interface returns numeric error codes and has no idempotency support, and a posted document can only be undone with a credit note.
The team builds an agent that reads an invoice, matches it to a purchase order and posts it.
Problem statement
A 400-invoice pilot of the first design went wrong in two ways: 9 invoices were posted twice (2.25%) and 14 were posted with a wrong amount (3.5%). At an assumed average invoice of 2,400 currency units, the 9 duplicates were 9 x 2,400 = 21,600 units of payment exposure.
Goals for the redesign: zero duplicate postings, no invoice posted whose lines do not add up to its total, and 75% of invoices posted without a clerk touching them, with the rest in a review queue.
Constraints
- Fixed: the ERP interface cannot change, and posting is expensive to reverse.
- Fixed: bank details come from the supplier master, never from an invoice.
- Fixed: invoice text goes only to providers on the approved list.
Why the traditional approach fell short
Template extraction is fast and exact for the 20 suppliers that send 60% of the volume, and breaks when any of them changes a layout. The remaining 400 or so suppliers each have their own layout, and a template per supplier is not affordable. Rules match purchase-order numbers but cannot read a line such as "freight surcharge per contract". Manual keying costs 800 clerk hours a month.
Options considered
| Option | Idea | Verdict |
|---|---|---|
| A. Templates and rules for the top suppliers, manual for the rest (non-AI) | Covers about 60% of volume at low risk | Kept for the top suppliers while the new path proves itself, then reviewed |
| B. Model extracts into a strict schema; code validates and posts | The model reads and matches with read-only tools; deterministic code checks the result and makes the single posting call with an idempotency key; anything doubtful goes to a human queue | Accepted |
| C. Agent composes ERP calls freely | The model chooses ERP functions and arguments and reads error codes | Rejected: this was the pilot design |
B is mostly a workflow with one model step and a short tool loop, by intent: the model decides how to match, and code decides whether a posting is allowed.
Solution architecture
Figure 1. Invoice agent components
How to read this: the model is reached only through the Orchestrator, and the ERP only through the Tool server.
Figure 2. One invoice, with the validation failure and the ERP timeout
How to read this: the first alt block is the validation gate, and the nested block is the timeout rule that prevents a duplicate.
How AI solved it, step by step
- An email arrives and an event lands on the Event bus. A Worker passes the invoice text to the Orchestrator.
- The Orchestrator sends the text to the model with three tool definitions:
find_supplier,find_poandsubmit_invoice. The last one has the invoice schema: amounts as integers in minor units, dates in ISO form, a currency code, and each line's quantity and unit price. - The model calls
find_supplierandfind_po. The Tool server answers from the ERP. - The model calls
submit_invoicewith the fields it extracted. The Orchestrator does not execute it; it validates first. - The validator checks the schema, that the supplier id is the one
find_supplierreturned in this run, that line amounts plus tax equal the total exactly, that the lines match the purchase order, and that no invoice from this supplier has this number already. - On a failure, the model receives the name of the failed rule and gets one correction attempt. A second failure goes to the clerk queue.
- On success, the Tool server records the idempotency key
supplier_id + normalized invoice numberin its own store (the ERP has no idempotency support) and then posts. On a timeout it looks the invoice up before any retry. - The Orchestrator records the outcome and the source link.
Under the hood
The key step is the function-calling round trip. Anthropic's tool-use documentation (as of 2026-10-21) describes it: the request carries tool definitions with a name, a description and an input_schema; the model replies with stop_reason: "tool_use" and one or more tool_use blocks; the application runs the operation and sends a tool_result block back with the matching tool_use_id. For client tools, the model does not execute anything. The documentation also notes that for a missing required parameter some models may infer a value instead of asking, and that strict: true makes calls match the schema exactly. The MCP specification's tools page states that servers MUST validate all tool inputs.
Put together: a schema guarantees shape, never truth. A total of 123450 is valid against the schema and still wrong if the invoice says 1,234.50 in a locale the model misread, which is what happened in the pilot. So the tool_use input is treated like a form post from an untrusted client: validated by code that knows the business rules. The message flow is in the lesson's function-calling section.
What went wrong / failure modes
- Timeout after commit. The ERP committed, then timed out, and the retry posted again. Mitigation: look up by supplier and invoice number before any retry, and treat the pair as the idempotency key.
- The model "fixes" an error by changing data. In the pilot it swapped a supplier id. Mitigation: the supplier id must equal one returned by
find_supplier, and error messages name the rule, not a value to use. - Ambiguous values. "03/04/2026" can be two dates. Mitigation: ISO dates in the schema, and ambiguity with no supplier locale goes to the queue.
- Injected text such as new bank details. Mitigation: bank details come only from the supplier master.
- Opaque ERP codes. Mitigation: the Tool server maps known codes to a short set; an unknown code goes to the queue, never to a retry.
- Layout drift. A new layout lowers accuracy silently. Mitigation: a monthly run over a golden set of 100 invoices and a 5% audit sample.
Results
These are illustrative targets, not measurements. At 8,000 invoices a month and 6 minutes each, today's effort is 48,000 minutes, or 800 hours. If 75% post automatically (6,000), the queue holds 2,000 invoices at the same 6 minutes, 12,000 minutes, plus a 5% audit of 300 invoices at 2 minutes each, 600 minutes. That is 12,600 minutes, or 210 hours, so 590 hours are saved, about 74%. The correctness targets are zero duplicate postings and zero postings where lines plus tax differ from the total.
Tradeoffs and when NOT to use this approach
- A strict validator sends more invoices to clerks at first; tune it with the audit sample.
- Do not automate posting where errors are costly to reverse and no validation catches them.
- Keep templates for the top suppliers while they work.
- Route on the validator's checks, not on a model's own confidence score.
Design rules extracted
- Validate every tool input in code against business rules, in addition to the schema.
- Give the posting tool an idempotency key based on a business key, and look up before retrying.
- Return errors that name the rule, so the model corrects the extraction and not the data.
- Take values such as bank details only from systems of record.
- Send doubtful items to a human queue and audit a sample of the rest.
Assumptions
- All numbers are chosen for illustration. Pilot: 400 invoices, 9 duplicates (9 / 400 = 2.25%), 14 wrong amounts (14 / 400 = 3.5%); average invoice 2,400 currency units; exposure 9 x 2,400 = 21,600.
- Volume: 8,000 invoices a month; 6 clerk minutes each: 8,000 x 6 = 48,000 minutes = 800 hours. The top 20 suppliers send 60% of volume.
- Target: 75% auto-posted = 6,000; queue 8,000 - 6,000 = 2,000 at 6 minutes = 12,000 minutes; audit 5% of 6,000 = 300 invoices at 2 minutes = 600 minutes; total 12,600 minutes = 210 hours; saving 800 - 210 = 590 hours = 590 / 800 = 73.75%.
- The golden set of 100, the 5% audit rate and the one-correction limit are illustrative choices.
- The distributor, its ERP and its suppliers are invented.
Last verified: 2026-10-22