Skip to content
    All guides

    Forward Deployed Engineer

    Lab: Invoice and Claims Document Processing Design

    An optional blank-page drill: design an invoice processing pipeline with extraction, checks and human review in 30 minutes, then score it against the rubric.

    By Published 6 min read

    • fde
    • solution-architect
    • mastery-lab
    • document-processing
    • extraction
    • optional
    Share

    This lesson is part of AI Solution Architect: Estimate, Propose and Defend. Enroll for free, then sign in, to read it in full.

    Enroll to unlockAlready enrolled?

    What you'll learn: how to run the seven-step design drill on a document-processing brief: why arithmetic and matching belong in code, how a review queue sets the automation level, and how a scored answer looks.

    At a glance

    Figure 1. The drill: seven outputs in 30 minutes

    flowchart LR accTitle: Document processing drill accDescr: Seven outputs in order: requirements, patterns, architecture sketch, failure path, evaluation plan, threat list and MVP scope, produced in four timed boxes and then scored against the rubric. A["Frame and requirements"] --> B["Patterns"] B --> C["Architecture and failure path"] C --> D["Evaluation and cost"] D --> E["Threat list"] E --> F["MVP scope"] F --> G["Score with the rubric"]

    How to read this: the same four timed boxes as the blank-page defense.

    • The model reads the document; code checks the numbers and matches the order.
    • Text inside a document is data, even if it says "pay this account".
    • A claims intake has the same shape: a claim form instead of an invoice.

    Scenario

    Composite scenario, not a real company. Quillfield Distribution has 9 accounts-payable clerks and about 6,000 supplier invoices a month (illustrative), arriving as PDFs by email. About 20 percent are scans, and about 400 suppliers use different layouts.

    Clerks key the supplier, invoice number, date, purchase-order (PO) number, lines, tax and total into the ERP system (enterprise resource planning software) and match each invoice to its PO. Month-end creates a backlog; wrong amounts and duplicate payments have occurred. The finance director wants less keying and earlier detection of mismatches, with nothing above a set limit posted unreviewed.

    What you will be able to do

    • Run the seven-step drill on a document brief.
    • Separate what the model extracts from what code verifies.
    • Set automation from measured accuracy.

    Prerequisites

    Core ideas

    The seven-step checklist for documents

    Step What is hard here Usually fits
    1. Frame The error rate today is unknown; some suppliers may send structured files, needing no model Measure first; ask which suppliers can send data
    2. Requirements Scans, 400 layouts, month-end peak, audit trail, bank details Batch processing with a per-document status
    3. Patterns Extraction is probabilistic, checks are not Document-processing pipeline; structured extraction; review queue; batch
    4. Architecture A posted payment cannot be recalled Post only after checks pass or a clerk fixes the invoice, with an idempotency key; approval above the limit
    5. Evaluation Field accuracy differs by field and by supplier Field-level labels; error rate of the auto-accepted set
    6. Threats Text in an invoice can address the model; changed bank details can redirect payment Schema-only output; bank changes never taken from extraction
    7. MVP (smallest first release) Posting is the risky step Extraction and review first, posting later

    In the interview: "How do you decide what is automatic?" Good: measure per-field accuracy, pick the threshold where the accepted set's error rate meets the business limit, keep sampling.

    Worked example

    Attempt the drill first, then open the answer.

    Open after your attempt: the model answer and its score

    Frame. Outcome: share of invoices posted without keying, and wrong-amount postings (target zero). Collect today's error and duplicate rates first; suppliers that can send structured files skip the model.

    Requirements. About 270 invoices a working day (illustrative). Users: 9 clerks, a finance approver. Data: PDFs, PO table, supplier master. Out of scope: payment runs.

    Patterns. Document-processing pipeline: schema-bound extraction, code checks, a review queue, asynchronous batches. Discarded: one model call that reads and posts (no check before paying) and manual keying (no saving).

    Figure 2. Model answer: invoice pipeline with checks

    Invoice processing model answerDocument store to Worker (PDFs); Worker to Model gateway (text); Model gateway to LLM provider (request); Model gateway to Orchestrator (fields); Orchestrator to Review queue (failed); Orchestrator to Tool server (passed); Review queue to Tool server (fixed)OUTSIDE OUR CONTROLDocument store: invoice PDFsDocument storeinvoice PDFsWorker: OCR, batchWorkerOCR, batchModel gateway: extract to schemaModel gatewayextract to schemaLLM provider: extractionLLM providerextractionOrchestrator: totals, PO, duplicatesOrchestratortotals, PO,duplicatesReview queue: clerk fixes fieldsReview queueclerk fixesfieldsTool server: ERP post, idempotentTool serverERP post,idempotentPDFstextrequestfieldsfailedpassedfixed

    How to read this: the model, outside our control, only extracts; code checks totals, PO and duplicates; failures go to a clerk; only checked or fixed invoices reach the ERP.

    Evaluation and cost. 200 invoices labelled field by field, spread across suppliers and scans. Metrics: per-field accuracy and the wrong-amount rate among auto-accepted invoices; 200 cannot prove a tiny rate, so 5 percent of accepted invoices are sampled in production. Cost at placeholder prices of 1.00 in and 5.00 out per million tokens: 6,000 x 2,500 = 15,000,000 tokens in and 6,000 x 400 = 2,400,000 out, so 15.00 + 12.00 = 27.00 a month, OCR (optical character recognition, which turns scans into text) priced separately. Review load (illustrative): 30 percent of 6,000 is 1,800 invoices at 1.5 minutes, 45 hours, against 300 hours to key all at 3 minutes.

    Threats. Instructions in an invoice: schema-only output, no tools on the model. Changed bank details: never taken from extraction, sent to a separate verified process. Duplicates: a supplier, number and total key. Wrong post: an idempotency key; above the set limit, even a passed invoice waits for approval.

    MVP. One supplier group, extraction and review only, no ERP posting for the first month; then posting after approval.

    Defend. "The model extracts and code verifies, with a clerk on every failure, because a payment cannot be recalled."

    Criterion Score Why
    Requirements quality 2 Baseline, structured-file question
    Pattern choice 2 Two alternatives discarded
    Architecture soundness 2 Flow, failure path, boundary, clerk review
    Risk and security 2 Injection, bank change, duplicates
    Evaluation plan 2 Labels, sampling, small-set caveat
    Cost awareness 2 Tokens and review hours
    Communication 1 Decision stated; no weakness conceded
    Total 13 of 14 Pass

    Assumptions: volumes, token counts, review share and times are illustrative; insert the provider's current published price and its date.

    Practice task

    Run the drill on the Quillfield brief or a claims variant.

    1. Work 30 minutes in four boxes (5 frame, 15 design, 5 risks and evaluation, 5 defend aloud). Produce six outputs: requirements, patterns, an architecture sketch, an evaluation plan, a threat list and an MVP scope.
    2. After a break, score each criterion with the rubric.
    3. Compare with the model answer and record your lowest criterion.

    Exercise

    The finance director asks to let small invoices post without review. Decide whether to allow it.

    Acceptance criteria: a measurable condition, a sampling rule and a switch-off. Done when you can name the accepted-set error rate at which you would revoke it.

    Hint 1: what evidence

    The error rate of the auto-accepted set, labelled and sampled.

    Hint 2: limiting the damage

    Cap the amount, keep duplicate and PO checks on, sample weekly.

    Common mistakes

    • Trusting the extracted total without summing the lines.
    • Posting without an idempotency key, so a retry pays twice.

    Knowledge check

    1. Why do totals and PO matching run in code and not in the model?

    Arithmetic and lookups have exact answers that code gives every time. A model can be subtly wrong, so code checks its extraction.

    2. How do you choose the threshold for automatic acceptance?

    Measure the error rate of accepted invoices at each threshold on the labelled set and pick the one that meets the business limit. Keep sampling in production: a small set cannot prove a rare error rate.

    3. Why must bank details on an invoice not be trusted?

    A changed account can redirect a payment, and document text is untrusted input. Changes go through a separate verified process, so a forged invoice cannot redirect money.

    4. What does the idempotency key protect against?

    A retry after a timeout would post the invoice twice. The key lets the ERP step recognise the repeat and do nothing.

    5. The model answer scored 1 on communication. What was missing?

    It stated the decision but conceded no weakness, so a reviewer cannot tell what the designer is least sure of. Naming one gap, such as unproven scans, earns the second point.

    You can now

    • Run a seven-step drill on a document brief.
    • Keep checks in code and extraction in the model.
    • Set automation from measured error rates.

    Further reading

    Last verified: 2026-11-03

    Next

    Continue with Lab: internal knowledge assistant with access control.

    Enroll to keep reading

    Lesson 13 of 15 · AI Solution Architect: Estimate, Propose and Defend

    An optional blank-page drill: design an invoice processing pipeline with extraction, checks and human review in 30 minutes, then score it against the rubric.

    In the rest of this lesson

    • Practice task
    • Exercise
    • Common mistakes
    • Knowledge check
    • You can now
    • Further reading

    Enrolling is free. It unlocks every lesson in AI Solution Architect: Estimate, Propose and Defend, tracks your progress across devices and keeps your quiz scores. The newsletter is optional.

    EnrollSign in

    Ask about this article

    Get answers grounded in this post. AI-generated — based on this article, and may be imperfect.

    AI Solution Architect: Estimate, Propose and Defend· Page 13 of 15
    PreviousLab: Support-Ticket Triage Assistant DesignNext Lab: Internal Knowledge Assistant With Access Control
    Scaled AI Weekly

    Enjoyed this? Get more like it every Monday.

    Real architecture decisions, LLMOps patterns that survive production, and engineering leadership advice — from 12+ years of building at enterprise scale. Free. No spam. Unsubscribe anytime.

    Get new articles by email

    Free. We email a confirmation link first, and you can unsubscribe from any email.

    Join engineers building production AI systems

    Comments

    Back to all guides