What you'll learn: a seven-step method for going from a business problem to a defensible AI design, what each step hands over, and how all seven look on one project, the Lumen Order and Policy Assistant, including the alternatives you throw away and why.
At a glance
Figure 1. The seven steps and what each one produces
outcome line"] --> S2["2 Require
requirements and risks"] S2 --> S3["3 Choose patterns
shortlist"] S3 --> S4["4 Draw
architecture"] S4 --> S5["5 Evaluate
evaluation plan"] S5 --> S6["6 Threat-model
threats and controls"] S6 --> S7["7 Plan
MVP and ADRs"] S5 -.-> S3 S6 -.-> S4
How to read this: each box is one step and its handover; dotted arrows are the loops to expect, because evaluation and security work often change earlier choices. MVP is the minimum viable product (the first release); an ADR is an architecture decision record (a short note on one decision).
- Seven steps, in this order: frame, require, choose patterns, draw, evaluate, threat-model, plan.
- Every step ends with something you can hand over, so a reviewer sees what you decided.
- Step 1 includes asking whether a model is needed at all.
Scenario
Composite scenario, not a real company. Lumen Industrial Supply, a fictional distributor of industrial equipment, has 24 customer-operations agents in two regions. They handle about 9,600 contacts a month about orders and policies, searching three places: the order management system (OMS), a library of 62 policy documents and the ticketing system. An old return policy still gets quoted because the old version is searchable.
The Head of Customer Operations wants faster, consistent answers. All numbers on this page are illustrative. You must turn "use AI to help the agents" into a design the sponsor, the IT security lead and the policy owner can each challenge.
What you will be able to do
- Name the seven design steps in order and the artifact each hands over.
- Decide for each part of a job whether it needs a model, a rule or a lookup.
- Write the seven-step map for a project in seven lines.
- Record the alternatives you rejected, with the reason.
Prerequisites
No earlier FDE page is required. Read AI Architect Mindset for requirements and the ADR format; this page does not repeat them.
Core ideas
The seven steps and what each hands over
| # | Step | The question | You hand over | Goes deep in |
|---|---|---|---|---|
| 1 | Frame | What outcome, measured how, and is AI needed? | Outcome line | Vague to measurable |
| 2 | Require | Who, which data, which limits, which risks? | Requirements and risks | Discovery |
| 3 | Choose patterns | Which proven shapes fit, at what cost? | Pattern shortlist | Patterns |
| 4 | Draw | Which parts, flows, failure paths, trust boundaries? | Diagrams | Briefing |
| 5 | Evaluate | How will we know it works, at what cost? | Evaluation plan | Golden set |
| 6 | Threat-model | What can go wrong, what stops it? | Threats and controls | Security |
| 7 | Plan | What ships first, and what did we decide? | MVP scope, ADRs | MVP scope |
Step 1 first: does this part need a model?
Predict first. An agent asks, "Where is order 48213?" Does answering it need a model? Decide before opening the answer.
Answer
No. The status is a field in the OMS, and an API call made with the agent's identity returns it exactly. A model would add cost, delay and a chance of misstating it.
| Part of the job | Needs a model? | Why |
|---|---|---|
| Fetch an order and its status | No | An API call returns the facts |
| Find the current return policy | Maybe not | Keyword search can find it |
| Explain the policy in plain words, with a citation | Yes | Rewording text is what a model does well |
| Decide whether a 12 percent discount is allowed | No | P-DSC-03 is a fixed rule: code applies it |
In the interview: "Walk me through designing an AI assistant for a support team." A strong answer follows the seven steps aloud, starts with the measurable outcome, separates what needs a model from what does not, and names an alternative it rejected.
Worked example
Assumptions: all numbers are illustrative: 24 agents, 9,600 contacts a month, a baseline median of 9 minutes (24 timings by 6 agents, a small sample) and a pilot target of 6 minutes or less.
Figure 2. The design the seven steps produce
How to read this: solid arrows are requests and the dashed arrow is a log written on the side; nothing is written to the order system, because the first release is read-only. The LLM (large language model) provider hosts the model.
All seven steps on the project, one row each:
| Step | The project's answer | Artifact |
|---|---|---|
| 1 Frame | Median handling time from 9 to 6 minutes or less; every policy answer cited; no high-severity wrong policy answers; refuse when the library cannot support an answer | Outcome line |
| 2 Require | 24 agents, 4 team leads; OMS, 62 policy documents (9 with several versions), ticketing; no new public endpoint. Top risk: quoting P-RET-01 v2 (14 days), not v3 (30 days) | Requirements, risks |
| 3 Patterns | Order lookup by plain API call; policy answers by retrieval with citations; keyword search first; no agent | Shortlist |
| 4 Draw | Figure 2, plus a sequence for a combined question and a refusal path | Diagrams |
| 5 Evaluate | Golden set (fixed test questions with known right answers) with the v2 trap and a credit-hold order; track correct version, citation present, correct refusal; budget about 3,200 input and 350 output tokens a request (illustrative) | Evaluation plan |
| 6 Threat-model | Wrong customer's data shown, personal data in logs, instructions hidden in a document. Controls: identity through the gateway, read-only OMS access with only the permissions needed, logs without customer personal data | Threats, controls |
| 7 Plan | Read-only MVP; pilot with a subset of agents; write actions in phase 2, proposed then approved by a human, with an idempotency key (a unique id that makes a repeated request change nothing); one ADR per decision | MVP scope, ADRs |
Record the rejected alternatives too:
| Discarded alternative | Why it lost | Step |
|---|---|---|
| A fully autonomous agent that edits orders | Changes after picked need warehouse approval (P-ORD-04) and only Finance releases a hold (P-HLD-05) |
3, 6 |
| Fine-tuning a model on the policies | P-RET-01 changed this quarter; trained-in text goes stale, cannot cite a version, needs retraining per change | 3 |
| One big prompt with all 62 documents | The 62 documents may fit a long context, but every request pays for all of them, superseded versions sit beside current ones, and nothing limits what may be cited | 3, 5 |
Spot the flaw. A design note reads: "Steps 1 to 4 are done: gateway, orchestrator, keyword search, hosted model. We will judge quality in the pilot by asking agents whether they like it." What is wrong?
Answer
Step 5 was skipped. "Do agents like it" is not a measurement: it cannot show whether the v2 trap is fixed or wrong answers are rare. Build a golden set with the v2 trap and a refusal case, plus a cost and latency budget, before the pilot.
Decision exercise. The Head of Customer Operations asks: "Can it also apply discounts?" Options: (A) add an order-edit tool to the MVP; (B) keep the MVP read-only, answer discount questions from P-DSC-03 and say who must approve, and plan "propose a discount for approval" for phase 2; (C) build a general agent. Choose one and justify it.
Model answer
Choose B. The first release then has no write path to secure or audit, and agents still get the P-DSC-03 limits quoted correctly. A write tool needs approval, an audit trail and an idempotency key: phase 2 work. A skips all that, and C cannot be tested against a fixed rule set.
Practice task
Write the seven-step map for the Order and Policy Assistant in seven lines, one per step: the question, your answer, the artifact. Keep each line under 30 words.
1 Frame: <outcome and measure; is a model needed?> -> Outcome line
2 Require: <users, data, limits, top risk> -> Requirements and risks
...
7 Plan: <MVP, pilot, what waits> -> MVP scope and ADRsAdd one line naming two alternatives you rejected. This is artifact 1 of your design pack.
Decide first, then reveal.
Exercise
A finance team wants to read PDF supplier invoices, extract the invoice number and total, and flag any invoice whose total differs from the sum of its lines. Write steps 1, 3 and 5 as one line each.
Acceptance criteria: step 1 states a measurable outcome and which part needs a model; step 3 names one deterministic part; step 5 names a metric computable from labelled invoices. Done when no line gives the sum check to a model and every line names an artifact.
Hint 1: which part is arithmetic
Adding the lines and comparing with the total is exact arithmetic. Which part reads messy layouts, and which needs only numbers?
Hint 2: a metric you can compute
Label a few invoices by hand. The share where extracted fields match your labels is a metric; so is the share of true mismatches caught.
Common mistakes
- Starting at step 4 because drawing boxes feels like progress; the boxes then hide guesses about steps 1 and 2.
- Putting a model on every box.
- Leaving out the rejected alternatives, so each reviewer re-proposes them.
- Listing threats without controls, or controls without an owner.
Knowledge check
1. Why does step 1 ask "is AI needed?" before any pattern is chosen?
The most reliable design uses a model only where free text must be read or written. If an exact lookup or a fixed rule does the job, a model adds cost, delay and error.
2. Why is the order status fetched without a model but the policy explained with one?
The status is a stored fact an API returns exactly, so a model could only make it worse. A policy is prose; explaining it in plain words with a citation is rewording, where a model helps and the citation can be checked.
3. Why are the discarded alternatives part of the artifact set?
A reviewer asks "why not the simpler or bolder option?" If the reasons are written down, the answer is in the pack. If not, you re-argue the same points in every meeting.
4. What breaks if a team skips step 5 and goes straight to a pilot?
Nothing measurable was defined, so the first superseded policy quoted in the pilot becomes an argument, not a test result.
5. The dotted arrows in Figure 1 go backwards. When do you follow them?
When evaluation shows a chosen pattern cannot reach the target, or threat modelling shows a flow crossing a trust boundary without a control. You change the earlier step, record why in an ADR and rerun the later steps.
You can now
- Add the seven-step map to your design pack.
- Name the seven steps in order with the artifact each hands over.
- Split a job into parts that need a model and parts that need code.
- List the alternatives you rejected, with one reason each.
Further reading
- Building effective agents, which recommends finding the simplest solution possible and only increasing complexity when needed.
- The C4 model, whose four diagram levels (context, containers, components, code) suit step 4.
- Architectural Decision Records, which defines an ADR as a document that captures a single architectural decision and its rationale (step 7).
Last verified: 2026-11-03