AI Architect Use Case Library
41 case files, each one a design problem solved with a named pattern. Some are based on an organisation's own published post or paper; the rest are labelled composite scenarios. Filter by industry, pattern or failure type.
Filter the cases
Industry
Failure type
Source
Showing 41 of 41 cases
- Case 1Composite
Case File: Claims Triage Copilot Without Requirements
An insurer's claims-triage copilot could not be judged a success or a failure because nothing was written down. How testable requirements fixed it.
- Industry
- Financial services
- Pattern
- Requirements definition
- Failure
- Requirement gap
- Lesson
- AI Architect Mindset: Requirements, ADRs, Design Reviews
- Case 2Composite
Case File: Contract Search, Build vs Buy vs Configure
A legal team started building AI contract search before costing it. A three-year comparison of configure, buy and build shows where the plan went wrong.
- Industry
- Legal
- Pattern
- Build vs buy
- Failure
- Cost overrun
- Lesson
- AI Architect Mindset: Requirements, ADRs, Design Reviews
- Case 3Composite
Case: Helpdesk Assistant Design Review, Trust Boundary
A design review of a B2B SaaS support assistant found tenant data crossing into a shared tool server with no boundary, and what the redrawn design changed.
- Industry
- SaaS B2B
- Pattern
- Trust boundaries
- Failure
- Data leak
- Lesson
- Diagramming AI Systems: C4, Sequence, State, ER
- Case 4Composite
Case: Billing Dispute Assistant and Retry Storms
A telecom billing-dispute assistant drew its runtime sequence and found three layers retrying one failing call, multiplying load during an outage.
- Industry
- Telecom
- Pattern
- Runtime sequences
- Failure
- Retry storm
- Lesson
- Diagramming AI Systems: C4, Sequence, State, ER
- Case 5Composite
Case: Support Chat Costs Tripled, Routing and Caching
A retail support assistant sent every message to its largest model and cost tripled after launch. A rules deflector, a router and prefix caching cut it back.
- Industry
- Retail ecommerce
- Pattern
- Model routing
- Failure
- Cost overrun
- Lesson
- Models, Routing and Inference for AI Architects
- Case 6Composite
Case: In-Product Assistant Latency, Streaming and Prefill
A SaaS assistant felt slow because of a long prompt and unstreamed replies. A latency budget, streaming, a shorter prefix and a smaller model fixed the tail.
- Industry
- SaaS B2B
- Pattern
- Streaming prefill
- Failure
- Latency tail
- Lesson
- Models, Routing and Inference for AI Architects
- Case 7Composite
Case: Classifying 2 Million Documents, Batch and Cascade
A public body priced real-time model calls for 2 million documents at 40,400 units. Rules, a batch cascade and human sampling cut it to about 5,250.
- Industry
- Public sector
- Pattern
- Batch classification
- Failure
- Cost overrun
- Lesson
- Models, Routing and Inference for AI Architects
- Case 8Composite
Case: Policy Q&A and Conflicting Documents
An insurer's policy assistant cited a superseded wording and silently picked between two conflicting documents. Version metadata and conflict surfacing fix it.
- Industry
- Financial services
- Pattern
- RAG
- Failure
- Stale data
- Lesson
- Data and Retrieval Architecture for AI Systems
- Case 9Composite
Case: Multi-Tenant Retrieval Isolation
A B2B SaaS assistant filtered tenants in application code after search, and a red team found a leak. Moving the filter below the app fixed it.
- Industry
- SaaS B2B
- Pattern
- Access control retrieval
- Failure
- Data leak
- Lesson
- Data and Retrieval Architecture for AI Systems
- Case 10Composite
Case: Legal Clause Search with Hybrid Retrieval
A law firm's clause search missed exact clause numbers and defined terms with embeddings alone. Hybrid search, a reranker and abstention fixed it.
- Industry
- Legal
- Pattern
- Hybrid search
- Failure
- Hallucination
- Lesson
- Data and Retrieval Architecture for AI Systems
- Case 11Published source
Public Case: LinkedIn Knowledge-Graph RAG for Support
LinkedIn authors built a knowledge graph from past support tickets for retrieval. What the paper states: method, metrics, and a six-month deployment result.
- Industry
- SaaS B2B
- Pattern
- Knowledge graph RAG
- Failure
- Retrieval miss
- Lesson
- Data and Retrieval Architecture for AI Systems
- Case 12Composite
Case: IT Helpdesk Access Agent and Approval Gates
A bank's helpdesk agent had one admin tool and obeyed injected ticket text. Least-privilege tools, a policy engine and approval gates fixed the design.
- Industry
- Financial services
- Pattern
- Approval gate
- Failure
- Tool abuse
- Lesson
- Agents, Tools and MCP: Architecture for AI Architects
- Case 13Composite
Case: Invoice Processing Agent and ERP Integration
A distributor's invoice agent posted duplicates and wrong totals to its ERP. Strict schemas, validation before the call and idempotency fixed it.
- Industry
- Manufacturing logistics
- Pattern
- Tool integration
- Failure
- Integration fragility
- Lesson
- Agents, Tools and MCP: Architecture for AI Architects
- Case 14Composite
Case: Engineering Assistant, MCP and Context Overflow
An engineering assistant with repo and ticket tools over MCP filled its context window with tool output. Scoped tools and a session budget fixed it.
- Industry
- Developer tools
- Pattern
- MCP
- Failure
- Context overflow
- Lesson
- Agents, Tools and MCP: Architecture for AI Architects
- Case 15Published source
Case: Uber QueryGPT, Natural Language to SQL
A sourced case on Uber's QueryGPT: how its text-to-SQL service moved from simple retrieval to workspaces and agents, how it is evaluated, and what it reports.
- Industry
- Manufacturing logistics
- Pattern
- Text to SQL agent
- Failure
- Wrong answer
- Lesson
- Agents, Tools and MCP: Architecture for AI Architects
- Case 16Composite
Case: Insurance Claim Triage and Duplicate Events
An insurer's event-driven claim triage created duplicate tasks and emails after a restart. Event keys, idempotent consumers and a dead-letter topic fixed it.
- Industry
- Financial services
- Pattern
- Event driven
- Failure
- Duplicate processing
- Lesson
- Multi-Agent and Event-Driven AI Architecture
- Case 17Composite
Case: Fraud Investigation Workflow as a Saga
A bank's AI-assisted fraud investigation left cards on hold with no case when one step failed. A saga with compensating actions and an outbox fixed it.
- Industry
- Financial services
- Pattern
- Saga
- Failure
- Partial failure
- Lesson
- Multi-Agent and Event-Driven AI Architecture
- Case 18Published source
Case: Anthropic's Multi-Agent Research System
What Anthropic's engineering post says about its lead-agent and subagent research system: design, token cost, evaluation, and production limits.
- Industry
- Developer tools
- Pattern
- Orchestrator worker
- Failure
- Cost overrun
- Lesson
- Multi-Agent and Event-Driven AI Architecture
- Case 19Composite
Case File: Silent Regression After a Prompt Tweak
A retailer's shopping assistant got worse in one product category after a prompt tweak while the overall score held. A per-category gate caught it.
- Industry
- Retail ecommerce
- Pattern
- Eval gates
- Failure
- Quality regression
- Lesson
- Evaluating AI Systems: Metrics, Judges and Gates
- Case 20Composite
Case File: A RAG Assistant That Is Confident and Wrong
A hospital staff assistant answered procedure questions fluently and wrongly. Separate retrieval and generation metrics showed why, and abstention fixed it.
- Industry
- Healthcare
- Pattern
- LLM judge
- Failure
- Retrieval miss
- Lesson
- Evaluating AI Systems: Metrics, Judges and Gates
- Case 21Composite
Case File: Summarisation Quality With a Calibrated Judge
A public agency's case-note summariser had no shared meaning of good. A rubric, a calibrated judge and pairwise checks made model changes testable.
- Industry
- Public sector
- Pattern
- Eval design
- Failure
- Quality regression
- Lesson
- Evaluating AI Systems: Metrics, Judges and Gates
- Case 22Composite
Case File: Canary Release for a Prompt Change
A prompt change passed the offline gate and quietly hurt one customer segment. A version pin, shadow run, canary gate and alias rollback limit the damage.
- Industry
- SaaS B2B
- Pattern
- Canary release
- Failure
- Quality regression
- Lesson
- LLMOps and Observability for AI Systems
- Case 23Composite
Case File: Cost Attribution Across Three Teams
Three telecom teams shared one model account and nobody could explain the bill. Tags at the gateway, team budgets and a weekly report made the spend clear.
- Industry
- Telecom
- Pattern
- Cost attribution
- Failure
- Cost overrun
- Lesson
- LLMOps and Observability for AI Systems
- Case 24Published source
Case: Anthropic's Postmortem of Three Quality Bugs
What Anthropic's September 2025 postmortem says about three overlapping bugs that degraded Claude responses, why they were hard to detect, and what changed.
- Industry
- Developer tools
- Pattern
- Tracing postmortem
- Failure
- Quality regression
- Lesson
- LLMOps and Observability for AI Systems
- Case 25Composite
Case: Provider Slowdown at Peak and Graceful Degradation
A retailer's assistant shared one provider pool across features, so a peak-hour slowdown froze order status. Bulkheads, a queue and a degraded mode fixed it.
- Industry
- Retail ecommerce
- Pattern
- Graceful degradation
- Failure
- Outage
- Lesson
- Reliability and Scaling for AI Systems
- Case 26Composite
Case: Retry Storm After a Rate Limit, Fixed With Jitter
A telecom's call-summary workers hit a tokens-per-minute limit, retried in lockstep and starved live traffic. A token bucket and jittered backoff fixed it.
- Industry
- Telecom
- Pattern
- Backoff jitter
- Failure
- Retry storm
- Lesson
- Reliability and Scaling for AI Systems
- Case 27Composite
Case: Slow Tail Latency and Hedged Retrieval Requests
A support-software assistant fanned each question out to 8 index shards, so one slow shard made many answers late. Hedged reads and a timeout budget fixed it.
- Industry
- SaaS B2B
- Pattern
- Hedged requests
- Failure
- Latency tail
- Lesson
- Reliability and Scaling for AI Systems
- Case 28Composite
Case: Pilot to Production, a 20x Cost Shock
A bank's research assistant forecast 13,816 units a month and cost 276,936 in production. Splitting cost per task showed which term to fix first.
- Industry
- Financial services
- Pattern
- Cost modeling
- Failure
- Cost overrun
- Lesson
- Cost and Unit Economics for AI Systems
- Case 29Composite
Case: Nightly Enrichment, Batch vs Real-Time Redesign
A logistics firm ran a nightly model job on the real-time path: it overran into the morning. A batch window, a cutoff and a capped fallback halved the cost.
- Industry
- Manufacturing logistics
- Pattern
- Batch vs realtime
- Failure
- Cost overrun
- Lesson
- Cost and Unit Economics for AI Systems
- Case 30Composite
Case: Legal Triage, Model Cascade Break-Even
A legal intake team sent every contract to its largest model. A cascade with a lawyer-calibrated threshold cut cost 55%, and a break-even check kept it honest.
- Industry
- Legal
- Pattern
- Model cascade
- Failure
- Cost overrun
- Lesson
- Cost and Unit Economics for AI Systems
- Case 31Published source
Case: EchoLeak, Indirect Injection in an AI Assistant
A sourced case on EchoLeak (CVE-2025-32711): how one email could steer Microsoft 365 Copilot toward data theft, and the layered defences one paper proposes.
- Industry
- SaaS B2B
- Pattern
- Prompt injection defense
- Failure
- Injection
- Lesson
- Security and Threat Modeling for AI Systems
- Case 32Composite
Case: Email Summariser Injection, Trust Boundaries
A composite case on an email-summarising assistant: why text in a message can steer it, and how a tool-less step, output checks and approvals contain it.
- Industry
- Financial services
- Pattern
- Trust boundaries
- Failure
- Injection
- Lesson
- Security and Threat Modeling for AI Systems
- Case 33Composite
Case: Tool Abuse and Data Exfiltration, Privilege Split
A composite case on a clinic assistant with a send-email tool and a broad service account: how a confused deputy leaks data and how least privilege stops it.
- Industry
- Healthcare
- Pattern
- Privilege separation
- Failure
- Data leak
- Lesson
- Security and Threat Modeling for AI Systems
- Case 34Composite
Case: PII in Traces and Logs, Redaction Design
A composite case on a support assistant whose traces stored customers' names and numbers: where redaction runs, why recall is imperfect, and what bounds it.
- Industry
- Telecom
- Pattern
- Log redaction
- Failure
- Data leak
- Lesson
- Security and Threat Modeling for AI Systems
- Case 35Composite
Case File: Screening Assistant and High-Risk Rules
A recruiting assistant ranked applicants and silently declined most of them. Reading Annex III first led to oversight, logging and a measured miss rate.
- Industry
- SaaS B2B
- Pattern
- Risk classification
- Failure
- Compliance gap
- Lesson
- Governance and Responsible AI in Practice
- Case 36Composite
Case File: Credit Decisions and the Audit Trail
A lender could not reconstruct how an AI-assisted credit decision was reached. An append-only decision record with versioned inputs made each case replayable.
- Industry
- Financial services
- Pattern
- Audit trail
- Failure
- Compliance gap
- Lesson
- Governance and Responsible AI in Practice
- Case 37Composite
Case File: A Retail Chatbot and the AI Disclosure
A retailer's order chatbot posed as a person named Mia. Article 50 disclosure at the first message, a logged event and a build test kept it simple and provable.
- Industry
- Retail ecommerce
- Pattern
- Transparency obligations
- Failure
- Compliance gap
- Lesson
- Governance and Responsible AI in Practice
- Case 38Composite
Case File: Automation Bias in a Procedure Assistant
Ward staff acted on a hospital assistant's wrong answers without opening the source. Source-first screens, drills, a report button and a stop switch fixed it.
- Industry
- Healthcare
- Pattern
- Human oversight
- Failure
- Automation bias
- Lesson
- Governance and Responsible AI in Practice
- Case 39Composite
Case: Keeping Model Processing in One Region, With Proof
A public agency's drafting assistant must process records in one named region. Region pinning, in-region logs and a daily audit check provide the evidence.
- Industry
- Public sector
- Pattern
- Data residency
- Failure
- Compliance gap
- Lesson
- Cloud Platforms for AI: Networking, Identity, Keys, Logs
- Case 40Composite
Case: Moving a Feature to Another Model Provider Safely
A retailer moves a product-question assistant to another model provider with a gateway adapter, versioned prompts, an evaluation gate and a canary release.
- Industry
- Retail ecommerce
- Pattern
- Portability
- Failure
- Lock in
- Lesson
- Cloud Platforms for AI: Networking, Identity, Keys, Logs
- Case 41Composite
Case: Calling a Managed Model Privately in a Clinic Network
A clinic network drafts visit summaries with a managed model and no public path. Private endpoint, DNS, identity, egress deny and redacted logs close the leaks.
- Industry
- Healthcare
- Pattern
- Private networking
- Failure
- Data leak
- Lesson
- Cloud Platforms for AI: Networking, Identity, Keys, Logs