Run a demo that shows real cases, failures and numbers, and handle changing requirements with a change log, a trade-off conversation and a state-based process.
What you'll learn: how to run a demo that builds trust (the customer's own cases, a refusal, a known miss and a measured result), and how to run change control so a new request is estimated, decided and logged instead of absorbed.
At a glance
Figure 1. A demo that earns trust
flowchart LR
accTitle: Demo structure
accDescr: A demo restates the problem and the agreed outcome, runs a real case chosen by the customer, shows a policy case with the version trap, shows a combined order and policy case, shows a refusal and a known miss, then the measured result with its limits and the decisions needed.
A["Restate problem and outcome"] --> B["Agents' own order case"]
B --> C["Policy case with the version trap"]
C --> D["Combined order and policy case"]
D --> E["A refusal and a known miss"]
E --> F["Measured result, limits, decisions"]
How to read this: the order moves from happy path to failure paths to numbers, and ends with a request for decisions.
Demo the agents' cases, not yours, and show a refusal and a known miss as well.
Every request that changes scope goes into a change log with an estimate and a decision.
A defect against an agreed criterion is fixed; a change to a criterion is decided.
Your artifact: the demo script and change log.
Scenario
Composite scenario, not a real company. At week four of an eight-week build plan, Lumen Industrial Supply, a fictional distributor, sees the first demo of its Order and Policy Assistant. A team lead asks whether discount approval could happen inside the tool, and the policy owner wants different wording on refusals. The plan has no free days: all 68 person-days are allocated. All numbers are illustrative.
What you will be able to do
Plan a 20-minute demo that includes a refusal and a known miss.
Log a change request with an estimate, a date effect and a decision.
Answer a request with options and a recommendation.
Version trap: v3 allows 30 days, old v2 said 14; the answer cites v3
2
Address change on a picked order
Order status plus P-ORD-04: warehouse approval
3
An unsupported question, then a known miss
Shows refusal and honesty
3
Result with count, sample size, limits; decisions
"It measured so", not "it feels good"
The segments total 2 + 4 + 3 + 2 + 3 + 3 = 17 minutes; 3 more of a 20-minute slot are for questions. A miss you already know about, shown with its cause, earns more trust than a perfect run.
At the customer: a sponsor asks, "Can you just show the best examples?" Say: "I will show two of your agents' cases, one refusal, one miss and the measured score, so you see it the way your agents will."
Change control as a state machine
Figure 2. Life of a change request
stateDiagram-v2
accTitle: Life of a change request
accDescr: A request is proposed and logged. It is triaged as a defect or a change. A defect goes straight to fixing. A change is estimated and then the owner decides to approve, defer or reject it. An approved change is scheduled, done and verified against its criterion. A deferred change goes to the parking list and can return as a proposal.
[*] --> Proposed
Proposed --> Triaged: logged with ID
Triaged --> Fixing: defect against an agreed criterion
Triaged --> Estimated: change to scope or criteria
Estimated --> Approved: owner agrees, schedule updated
Estimated --> Deferred: owner defers
Estimated --> Rejected: owner declines
Approved --> Done: built
Done --> Verified: criterion rerun
Fixing --> Verified
Deferred --> Proposed: returns with new reason
Verified --> [*]
Rejected --> [*]
How to read this: nothing reaches "Approved" without an estimate and an owner, and nothing reaches "Verified" without rerunning its criterion.
CHANGE LOG
ID | Date | Requested by | What | Affects (AC IDs) | Effort (days)
| Effect on date | Options offered | Decision | Decided by | Verified on
The triage question: does the request fix a miss against a criterion the sponsor already agreed, or does it add or alter something? The first is a defect and needs no decision; the second is a change and does.
The trade-off conversation
Option
What changes
Risk
Swap
Something of equal size leaves
Low, if truly lower value
Extend
The end date moves
Medium: pilot window, sponsor patience
Defer
Parking list, with a return condition
Low, if the condition is clear
Reduce assurance
Less testing or review
High: do not recommend; record the objection
Decision exercise. A team lead asks for discount approval inside the tool (12 days; P-DSC-03 sets limits by role). Options: (A) say yes and absorb it; (B) approve it and extend the pilot by the days it needs; (C) defer it to phase 2 and show a labelled mock-up. Choose one and justify it with the numbers in the worked example.
Model answer
Choose C unless the sponsor values this above the pilot date. Option A has no free days, so the 12 days would come out of testing, which proves the pilot works. Option B moves the end by about 1.4 weeks. C protects the criteria; present B as the alternative so the sponsor chooses.
Explain it back. Tell the sponsor in three sentences why even a half-day request goes into the change log. A good explanation says small requests add up, and the log shows what each "yes" costs the date.
Worked example
Assumptions: illustrative; the plan is 68 person-days over 8 weeks (40 working days for each of 2 engineers), so capacity is 68 / 8 = 8.5 person-days per week; no free days remain.
ID
Request
Type
Effort (days)
Effect on date
Decision
CR-1
Discount approval inside the tool
Change
12
12 / 8.5 = 1.4 weeks
Defer to phase 2, mock-up only
CR-2
Reword the refusal message
Change
0.5
0.5 / 8.5 = 0.06 week
Approve, swap against 0.5 day of pilot support
DEF-1
Suppose AC-3 returns 9 of 10 refusals
Defect
1
None: a fix inside the plan, not a change
Fix, rerun AC-3
Approving both CR-1 and CR-2 without a swap adds 12.5 days, 12.5 / 8.5 = 1.5 weeks. DEF-1 is a defect because AC-3 demands all 10 declines.
The message to the sponsor for CR-1:
You asked for discount approval inside the tool (12 days).
We have no free days, so I see three options:
- Defer: this becomes phase 2, and I show a mock-up on Friday.
- Extend: it is in the pilot, and the end date moves by about 1.4 weeks.
- Swap: nothing of 12 days can leave without removing a feature the criteria test.
I recommend deferring. Please choose by Wednesday.
Practice task
Plan the week-four demo for Lumen.
Write the run sheet for the 20-minute slot from the script table.
Write the "limits" sentence for the result, including the sample size.
Log three requests with a triage label, an estimate and options: "show the tracking link", "let agents export the answer" and "a return answer quoted v2 on one of the 9 version traps".
Reply to the sponsor on the largest. The run sheet and log are your demo script and change log.
Decide first, then reveal.
Demo and change calls (interactive when JavaScript is on)
A colleague wants to rehearse the demo cases the night before so nothing fails.
Use cases the agents chose; show a refusal and a known miss
Rehearse the best cases and skip the refusal
Show only failures
Answer
Use cases the agents chose; show a refusal and a known miss. Rehearsed cases hide failures, which agents then find. A miss shown with its cause earns more trust than a perfect run.
The policy owner wants different wording on the refusal message (0.5 day). The plan has no free days.
Say yes and absorb it
Log it, approve it, swap against 0.5 day of pilot support
Reject it: only criteria changes are logged
Answer
Log it, approve it, swap against 0.5 day of pilot support. A swap counts when something of equal size leaves. Absorbed requests consume the testing time.
To fit a change, the sponsor says: "Skip the retest of AC-3."
Agree: it passed once
Extend the date and say nothing about the retest
Do not recommend it; record your objection and offer swap, extend or defer
Answer
Do not recommend it; record your objection and offer swap, extend or defer. Reducing assurance is the high-risk option, and nothing is verified until its criterion is rerun. Silence hides the cost.
Exercise
Write the trade-off message for CR-2 in the format of the CR-1 message, then triage this sentence from an agent: "The assistant refused to change the address on a picked order, but the customer says the warehouse agreed."
Acceptance criteria:
The message offers at least two options, recommends one and states a decision date.
The triage names defect or change, and why.
The log entry has an ID, an estimate and an effect on the date.
Done when the sponsor could answer with one word and you could act without another meeting.
Hint 1: what does the criterion say?
Read AC-5. If the refusal follows P-ORD-04 (after picked, only with warehouse approval), the assistant behaves as agreed, so the sentence asks for a change.
Hint 2: offer a swap that exists
A swap counts only if something of equal size leaves. Check the scope statement first.
Common mistakes
Rehearsing the cases. If the agents' own cases would fail, the demo taught nobody anything.
Absorbing small requests. Together they consume the testing time.
Saying "yes" first and estimating later. The sponsor remembers the yes.
Treating a defect as a change. A miss against an agreed criterion is yours to fix.
Knowledge check
1. Why does the demo include a refusal and a known miss?
A refusal shows the assistant does not invent policy, and a miss explained with its cause shows you measure honestly. Showing only successes leaves agents to find the failures.
2. How do you tell a defect from a change?
A defect is a miss against a criterion the sponsor already agreed, so the fix needs no decision. A change adds or alters scope or a criterion and needs an estimate and the owner's decision.
3. Why does CR-1 move the end date by 1.4 weeks?
The plan delivers 68 person-days in 8 weeks, 8.5 a week, and 12 / 8.5 is 1.4. That 8.5 already includes the 75 percent availability.
4. Why offer options instead of saying yes or no?
A yes hides the cost and a no looks obstructive. Options show what leaves, how far the date moves and what assurance is lost.
5. What does "verified" require in the change process?
The affected criterion must be rerun and its result recorded. Without that, "done" means "I think it works".
You can now
Add the demo script and change log to your design pack.
Plan a demo with the agents' cases, a refusal, a known miss and a measured result.
State the limits of a result in plain words with its sample size.
Triage a request as a defect or a change and log it.
Offer swap, extend or defer options and recommend one.
Further reading
Google SRE Workbook: Canarying Releases, which defines canarying as a partial and time-limited deployment of a change and its evaluation, used to decide whether to proceed.
Lesson 6 of 8 · AI Solution Architect: Discovery, Scope and Security
Run a demo that shows real cases, failures and numbers, and handle changing requirements with a change log, a trade-off conversation and a state-based process.
In the rest of this lesson
Worked example
Practice task
Exercise
Common mistakes
Knowledge check
You can now
Enrolling is free. It unlocks every lesson in AI Solution Architect: Discovery, Scope and Security, tracks your progress across devices and keeps your quiz scores. The newsletter is optional.
Real architecture decisions, LLMOps patterns that survive production, and engineering leadership advice — from 12+ years of building at enterprise scale. Free. No spam. Unsubscribe anytime.
Get new articles by email
Free. We email a confirmation link first, and you can unsubscribe from any email.