AI Architect Templates: ADR and Design-Review Checklist

Two reusable files from the AI Architect course: an architecture decision record (ADR) template for AI design decisions, and a design-review checklist with one question per line. Each is shown in full below and is also a plain Markdown download, so the page and the file always say the same thing. The cheat sheet is the one-page map of the rules behind them.

• Download the ADR template (adr-template.md)
• Download the design-review checklist (design-review-checklist.md)

How to use the templates

1. Write an ADR for a decision that is costly to reverse, that constrains later choices, or that someone will ask about later ("why did you do it this way?"). The mindset lesson lists the decisions worth recording: model and provider, build vs buy vs configure, retrieval strategy, where humans approve, what is logged, and which data may reach which model.
2. Send the pre-read two working days before the review. It holds the requirements, constraints, options, evidence and open questions, which are the first sections of the template.
3. Run the checklist at the design review. The 60-minute agenda and the ten review areas are in Design reviews. The checklist adds the questions from every other lesson.
4. Record the answers in the ADR. Each question gets Yes, No or N/A. Every No or N/A goes into the ADR's "Design-review record" with its reason, an owner and a date, and the outcome is one of the three the mindset lesson names: approved, approved with conditions (each with an owner and a date) or not approved (with the missing evidence listed).
5. When the context changes, write a new ADR and supersede the old one. Never edit an accepted record to change the decision. A filled example, ADR 12, is in the mindset lesson under Architecture Decision Records.

ADR template

The structure is Michael Nygard's 2011 format as the mindset lesson teaches it (a numbered title, a status, the context, the decision and the consequences), with the extra fields that the mindset lesson's ADR 12 uses for an AI choice.

The full text, identical to adr-template.md:

# ADR <number>: <short noun phrase naming the decision>

Status: <Proposed | Accepted | Deprecated | Superseded by ADR <number>> (<YYYY-MM-DD>). Supersedes: <none | ADR <number>>.

Owner: <the person who answers for this record>

How to use this template: the AI Architect course, https://buildingai.in/learning/ai-architect-templates. Number records in sequence and never reuse a number. Keep the record to one or two pages, in the repository at doc/arch/adr-NNN.md. Never edit an accepted record to change the decision: write a new ADR and mark this one superseded.

## Context

State facts in value-neutral language. Name the forces at play (technological, political, social and project-local), including the ones in tension.

Requirements this decision serves. For each: the indicator, the target, the window and conditions, and the measurement point.
- <requirement 1>
- <requirement 2>

Constraints. Mark each one fixed or negotiable and name who can change it.
- <constraint 1: fixed or negotiable, owner>

Options considered. Include configure, buy and build where they apply, and the simplest non-AI option with the reason it is not enough.
- <option 1>
- <option 2>

Evidence. Name the evaluation set and its version, the number of items, the passes, the 95% interval and the date measured. Add the latency percentile and the relative cost of each option. Say what the data do not prove.
- <option 1: passes of n, interval, latency, relative cost>
- <option 2: passes of n, interval, latency, relative cost>

Data sensitivity. Which providers and logs may see which data, and for how long.
- <rule 1>

Expected volume.
- <requests per period>

## Decision

We will <the choice, in full sentences and the active voice>. We will record <the prompt, rubric and model versions> on every request.

## Consequences

Describe the resulting context once the decision is applied, all of it: positive, negative and neutral.

- Positive: <consequence>
- Negative: <consequence>
- Neutral: <consequence>
- Review trigger: <the event or measurement that makes the team revisit this decision>

## Design-review record

- Review date: <YYYY-MM-DD>
- Outcome: <Approved | Approved with conditions | Not approved>
- Conditions, each with an owner and a date: <condition, owner, date>
- Missing evidence, if not approved: <what is missing>
- Checklist answers: <IDs answered No or N/A, each with the reason, owner and date>
- Checklist used: https://buildingai.in/downloads/ai-architect/design-review-checklist.md

Design-review checklist

Every question can be answered Yes, No or N/A, and each group ends with the page to read when the answer is No. The 170 questions come only from the decision frameworks, failure tables, checklists and action tables already in the lessons (the diagram checklist, the design-review question table and, on page 14, the guard sketch and the done-when line); none is new. Where a question touches regulation, the governance lesson applies and nothing here is legal advice.

The full text, identical to design-review-checklist.md:

# AI design-review checklist

From the AI Architect course on buildingai.in: https://buildingai.in/learning/ai-architect-templates

Every question comes from a decision framework, a failure table, a checklist or the design-review question table in the lesson named at the end of its group. It adds nothing the lessons do not teach. Open the lesson for the reasoning behind a question.

## How to use it

1. Run it at the design review: after the diagrams and the draft decision records are on the table, before the decision is recorded. The review itself is described in https://buildingai.in/learning/ai-architect-mindset#design-reviews
2. Answer every question with Yes, No or N/A. N/A needs a reason, for example "no tools in this design".
3. Every No needs an owner, a date and, if the design still goes ahead, a condition. The page to read is at the end of the group.
4. Record the answers in the ADR for the decision under review (the "Design-review record" section of the ADR template), then record the outcome: approved, approved with conditions, or not approved.
5. Run it again when the design, the model, the prompt or a constraint changes.

## 1. Requirements and design reviews

- RQ1. Does every requirement have a number and a measurement point?
- RQ2. Can the owner of each requirement state what result would make them say no?
- RQ3. Is each constraint marked fixed or negotiable, with a named owner?
- RQ4. Were options that break a fixed constraint excluded before the options were compared?
- RQ5. Was an owned capability trialled first, before buying or building?
- RQ6. Is the risk tolerance named by someone with authority and written down where the review can check it?
- RQ7. Is every decision that is costly to reverse, or that constrains later choices, written as an ADR before the build?
- RQ8. Does each ADR have an owner and a review trigger?
- RQ9. Where one wrong output is high risk and hard to detect, is there a human approval step that is tested as a requirement?
- RQ10. Where a quality bar is claimed, is an interval that straddles the bar reported as not demonstrated and not as a pass?
- RQ11. Does any change to the model or prompt re-run the release set, with a drop below the bar blocking promotion?
- RQ12. Is the worst plausible output known, and who is harmed if it reaches a user?
- RQ13. Is the behaviour defined for a provider that is slow, down or returns something malformed?
- RQ14. Is the cost per request known at expected volume and at ten times that?
- RQ15. Is it known which data goes to which component, and where it is logged?
- RQ16. Is it written down what the system can do without a human approving it, and the maximum damage of one wrong action?
- RQ17. Is there a way to detect that quality dropped after a model or prompt change?
- RQ18. Is the rollback defined, and is its duration known?
- RQ19. Is the simplest non-AI option named, with the reason it is not enough?
- RQ20. Is the decision that is hardest to reverse written down?

Read: page 2, https://buildingai.in/learning/ai-architect-mindset

## 2. Diagrams

- DG1. Does each diagram have a figure title and one sentence saying what question it answers?
- DG2. Are shapes, line styles and zone markers explained beside each diagram?
- DG3. Do shape, line style or text carry whatever colour shows?
- DG4. Is each diagram about 15 nodes at most, or split into an overview and zoom-ins?
- DG5. Does every component keep the same name in every diagram and in the text?
- DG6. Does every arrow have one direction and a label for what travels?
- DG7. Does at least one diagram show a failure path (timeout, empty result or denial)?
- DG8. Are trust boundaries visible, with the control named at each crossing?
- DG9. Is the source of each diagram text next to the code, reviewed with it?
- DG10. Where there are 3 or more participants, or any async step, is there a sequence diagram?
- DG11. Where a component can loop or end more than one way, is there a state diagram in which every state has an exit?
- DG12. Has the review walked every outbound arrow from the application zone, asking which boundary it crosses and who holds the credential?
- DG13. Does each user-facing sequence have companions for empty-result, timeout and denial?

Read: page 3, https://buildingai.in/learning/ai-architect-diagramming

## 3. Models and routing

- MR1. Where the request type is known before the call, are requests routed by rules first?
- MR2. Where a cascade is used, can the cheap answers be scored and is a slower tail acceptable?
- MR3. Is the cascade threshold set on labelled data, with costly intents always sent to the large model?
- MR4. Is the prompt laid out for prefix caching, with stable content first?
- MR5. Where an answer depends on user, tenant, time or changing data, is semantic caching avoided?
- MR6. Do cache keys include the tenant and the version, with expiry on a version change and hits logged?
- MR7. Does an alert watch the cache-read share, with one warm-up request per release?
- MR8. Is each routing decision logged with its route, reason and confidence, and is a weekly sample reviewed per intent?
- MR9. Are model versions pinned and tokens counted with the target tokenizer, with trimming by policy or a defined error?
- MR10. Where a user is waiting on a long answer, does the response stream?
- MR11. If a stream fails after the first token, is the partial text kept and marked incomplete, with no tool call from it and an explicit retry?
- MR12. Where residency, pinning or sustained load is a hard requirement, was self-hosting evaluated, with an API behind a gateway otherwise?

Read: page 4, https://buildingai.in/learning/ai-architect-models-routing

## 4. Retrieval

- RT1. Where exact search meets the latency budget, is it used with a filter?
- RT2. Where queries contain identifiers or defined terms, is there a lexical index fused with RRF?
- RT3. Where recall at 50 is good and at 5 is poor, is a reranker added?
- RT4. Is the access filter applied from verified identity inside the search and not afterwards?
- RT5. Is a change of model, chunker or index parameters built as a new index beside the old one and then switched?
- RT6. Do sources keep version, date and status, with the superseding document preferred?
- RT7. Does the Retriever refuse a query when the index and the embedding model do not match?
- RT8. When a filtered query returns too few results, does the system abstain below the minimum?
- RT9. Does a version count as indexed only when both the lexical and the vector writes succeed?
- RT10. If the reranker times out, is the fused order used and the trace marked?
- RT11. Below the evidence threshold, is the model not called?
- RT12. When a permission is revoked, is the filter metadata updated first, with permission staleness measured separately?

Read: page 5, https://buildingai.in/learning/ai-architect-data-retrieval

## 5. Agents and tools

- AT1. Where the steps are known in advance, is the design a workflow in code and not an agent loop?
- AT2. Does each irreversible tool have an idempotency key, a policy decision and approval above a limit?
- AT3. Are calls, tokens and time capped from measured runs?
- AT4. Are large tool results paginated, filtered, truncated or returned as a handle?
- AT5. Are servers you do not own allow-listed, pinned, prefixed and treated as untrusted?
- AT6. Where facts must outlive a conversation, is memory scoped, with expiring and policy-checked writes?
- AT7. Does a used-up budget end the run in BudgetStop, with partial findings and one audit event?
- AT8. Does an identical repeated call get a tool error?
- AT9. Does every write carry an idempotency key, with the target returning the first result and one layer retrying?
- AT10. Do tool results stay in tool-result blocks, with the Policy engine deciding from identity and validated arguments and approval on outbound tools?
- AT11. When the context fills mid-run, is the transcript archived and compacted or cleared, or the run stopped at a resumable checkpoint?
- AT12. Does an unanswered approval request expire and return a tool error to the model?

Read: page 6, https://buildingai.in/learning/ai-architect-agents-tools-mcp

## 6. Multi-agent and event-driven designs

- EB1. Where a task cannot be split into independent parallel subtasks, or its parts share context, is it one agent or a fixed workflow?
- EB2. Where work can wait seconds and arrives in bursts, is there an Event bus between the Orchestrator and the Workers?
- EB3. Are offsets committed after processing, with an idempotent effect keyed by event id?
- EB4. Where a state change and its event must agree, is it one local transaction through an outbox?
- EB5. For an action that spans services and has irreversible steps, is there a saga with idempotent compensations?
- EB6. Is there a retry limit with backoff, a dead-letter queue with error context, a per-key quarantine and an alarm on depth?
- EB7. Is the batch size times the worst-case time per event well inside the poll interval?
- EB8. Does a failed saga compensation get idempotent retries and then a human queue with the full state?
- EB9. For work you cannot lose, does a consumer that returns after its offset expired use none or earliest, with the replay deduplicated?
- EB10. Are there effort rules by task type, a cap on subagents and a token budget per job?
- EB11. After a Worker crash, is the offset committed after the batch, with dedupe skipping applied events?

Read: page 7, https://buildingai.in/learning/ai-architect-multi-agent-eventdriven

## 7. Evaluation

- EV1. Is there a labelled golden set, started with 20 to 50 real items and extended with every incident?
- EV2. Where a judge decides the release, has it been calibrated, with kappa and failure recall reported?
- EV3. When two versions are compared, is the judging pairwise, with a position swap, on the same items?
- EV4. Where the feature uses retrieval, are retrieval and generation metrics reported per slice?
- EV5. Does the gate use a confidence bound and state the detectable drop?
- EV6. For an agent that acts on the world, are the end state, the policy steps and pass^k measured?
- EV7. Does a malformed judge output retry once and then count as an evaluation error, with a run above the error limit invalid?
- EV8. Are judge versions stored on every score, with recalibration and a new baseline when the judge changes?
- EV9. Does every slice report n and an interval, showing not measurable below the minimum count?
- EV10. Does the judge come from another model family than the system, or is a human-confirmed sample used?
- EV11. Is an inconclusive gate blocked by default?

Read: page 8, https://buildingai.in/learning/ai-architect-evaluation

## 8. Release and operations

- RO1. Does every change to an answer-shaping part get a release ID stamped on every span?
- RO2. For a change with side-effecting tools, is shadow run with stubbed tools and the canary behind approval?
- RO3. Where the canary yields fewer judged items per day than needed, is it used for hard signals, with the share or the duration widened for quality?
- RO4. Do traces hold identifiers and redacted text with short retention?
- RO5. Where teams share a model account, are gateway tags required and reconciled with the provider each month?
- RO6. Does the message header carry traceparent across a queue, with an alert on root spans that have no downstream spans?
- RO7. If the trace store or the collector is down, are requests unaffected, with spans dropped, counted and alerted?
- RO8. Does an inconclusive canary hold at its stage, with items added or a review, and never a re-run until green?
- RO9. Does the release record store the retirement date of its rollback target, with an alert ahead of it?
- RO10. Are content attributes off by default, redaction run at the collector and tests run with seeded fake personal data?
- RO11. When untagged spend rises, are direct keys revoked and the gap alert sent to the platform owner?

Read: page 9, https://buildingai.in/learning/ai-architect-llmops-observability

## 9. Reliability

- RL1. Is there one retry owner per failure class, with SDK retries set to 0?
- RL2. Is a token bucket sized to the limit in the Model gateway?
- RL3. Do retry loops use full or decorrelated jitter and an attempt limit?
- RL4. Does each feature have its own bulkhead pool, sized with Little's law?
- RL5. Is work that can wait queued with a deadline stamp and bounded consumers?
- RL6. Are hedged requests limited to idempotent reads, sent after the p95 delay and under a cost cap?
- RL7. When the spend cap is reached, are retries skipped, a degraded path taken and the budget owner alerted?
- RL8. When a provider region is down, does the breaker open on 5xx and timeouts, do calls go to the second deployment and does a status banner show?
- RL9. Are jobs past their deadline stamp skipped and reported?
- RL10. While autoscaling, does an aggregate bucket cap total calls, with a gradual scale-out ramp?
- RL11. On a 429, does the bucket pause to retry-after, with one owner retrying with jitter?

Read: page 10, https://buildingai.in/learning/ai-architect-reliability-scaling

## 10. Cost

- CO1. Is per-task cost measured at the Model gateway before any lever is chosen?
- CO2. Where output tokens dominate, were the format and max_tokens shortened first?
- CO3. Where a stable prefix is reused above break-even, is the prompt laid out for prefix caching?
- CO4. Where nobody waits and the deadline exceeds the window, is the work batched with a cutoff and a fallback?
- CO5. Is easy traffic routed by rule, with the rest cascaded?
- CO6. Where sustained load keeps utilisation above r, was self-hosting evaluated with operating cost?
- CO7. Does a team budget degrade at a soft limit, reject at a hard limit with a defined error and queue non-urgent work?
- CO8. Does a per-task cap on calls and tokens stop a runaway agent loop, returning a partial result and logging the cap?
- CO9. Does an alert fire on the cache read share, with a warm-up request after a prompt release?
- CO10. Does an alert fire at 80% of the cascade's break-even escalation rate, with those intents moved to the large model by rule?
- CO11. Does a batch that misses its window cancel at the cutoff, with the stragglers run in real time within rate limits?
- CO12. Does the cost model read ratios from config with a review date, and recount tokens on version changes?
- CO13. Were scenarios reported and not one forecast number, with a monthly refresh and a kill switch for optional features?

Read: page 11, https://buildingai.in/learning/ai-architect-cost-economics

## 11. Security

- SE1. Is every source of untrusted text drawn as an entry point with an owner and a control?
- SE2. Where the model's output can trigger a tool, does authority come from the session and a scoped credential and never from a model-written value?
- SE3. For an irreversible or external action, is approval required, showing the exact content and restricting destinations?
- SE4. Is any classifier kept as one layer next to structural controls?
- SE5. When injected text asks for a send, does the Policy engine deny it, does the step have no send tool and is a security event logged?
- SE6. If the classifier is down or refuses, are tool calls blocked while answer-only paths continue with remote images off?
- SE7. Do retention and read limits cap the exposure of a value the redactor misses, with a weekly seeded test?
- SE8. Does the approved description of a third-party MCP tool stay pinned until it is reviewed?
- SE9. Do approvals cover only external side effects, with a sample reviewed?
- SE10. If the Policy engine is down, are calls refused and the user told?

Read: page 12, https://buildingai.in/learning/ai-architect-security-threat-modeling

## 12. Governance

- GV1. Is every AI system registered with its intended purpose before launch?
- GV2. Where the intended use is in an Annex III area, is it treated as high-risk, with counsel asked before relying on Article 6(3)?
- GV3. Where decision records hold personal data, are ids and hashes kept in the audit store and the content elsewhere?
- GV4. Where a person can override the output, are override rate, review time and sampled agreement measured?
- GV5. Where the system talks to the public, is the disclosure made at first interaction, with the version shown logged?
- GV6. Does a change of model, prompt, retrieval or purpose run change control: re-evaluate, update the card, re-classify?
- GV7. Is a production system that is missing from the registry treated as unclassified, registered by a deadline or switched off?
- GV8. If the audit store is unavailable, are the decisions that need a record not released?
- GV9. Can an impact query list the answers that cited a superseded source document, for re-review?
- GV10. Where the classification is genuinely ambiguous, is the stricter handling used and counsel asked?
- GV11. Do sampled agreement and review time raise an alert when reviewers approve almost everything, quickly?

Read: page 13, https://buildingai.in/learning/ai-architect-governance-responsible-ai

Nothing in this checklist is legal advice.

## 13. Cloud

- CL1. Where a workload holds a long-lived key, has it moved to workload identity with short-lived credentials?
- CL2. Where the model endpoint is public, is there a private endpoint with the public path closed?
- CL3. Where a rule names a region or geography, is the narrowest routing level the rule allows chosen and verified in the audit log?
- CL4. Where a feature stores prompts or answers, are they redacted before storage, with retention set?
- CL5. Where stored artifacts need your control, is there a customer-managed key, with revocation rehearsed?
- CL6. If private DNS is not reachable from another network, does the call fail closed or get denied by the network rule?
- CL7. When a role was just assigned, does the caller retry briefly and then alert?
- CL8. If the key is disabled or unavailable, do requests that need protected data fail with a clear error and an alert?
- CL9. Does an audit check find inference routed outside the allowed region and raise an incident?
- CL10. Where content logging captures personal data, do redaction, retention and access limits cap the exposure?
- CL11. When a call uses an endpoint the log does not cover, does a pipeline test fail?
- CL12. Is DNS planned for on-premises and other networks, with name resolution tested from each?
- CL13. Does the private endpoint carry a policy or network rule naming the principals and actions allowed?
- CL14. Are stored keys removed from images, environment files and CI variables?
- CL15. Do administration and inference use separate identities?
- CL16. Does each service have its own least-privilege role, so the audit log names who called?
- CL17. Before the design says it is encrypted with your key, has each provider's list of what the key covers been read?
- CL18. Are prompts in transit protected by TLS and the private path, and not assumed to be covered by the key?
- CL19. Is the content-logging policy decided in the design review and recorded?
- CL20. Is there one adapter per provider behind the Model gateway, so a migration changes an adapter and a configuration and not the Orchestrator?
- CL21. Is anything not compared across providers, such as tool schemas or caching, treated as untested until the evaluation set says otherwise?
- CL22. Does a CI check fail the build on a configuration with a stored key, a public path, a region outside the allowed set or unredacted content logging?
- CL23. Does each of the six parts (identity, private path, keys, logs, region and capacity) have a named setting and a source link?

Read: page 14, https://buildingai.in/learning/ai-architect-cloud-platforms. It points to page 10 for retries, https://buildingai.in/learning/ai-architect-reliability-scaling, to page 12 for redaction, https://buildingai.in/learning/ai-architect-security-threat-modeling, and to page 13 for audit-trail rules, https://buildingai.in/learning/ai-architect-governance-responsible-ai

## Recording the answers

Paste this block into the "Design-review record" of the ADR and fill it in.

- Checklist run on: <YYYY-MM-DD>
- Answered No or N/A: <question ID: reason, owner, date>
- Outcome: <Approved | Approved with conditions | Not approved>

Last verified: 2026-10-23