Diagramming AI Systems: C4, Sequence, State, ER

Sequence diagrams for runtime behaviour

Structure diagrams do not say what happens in what order. A sequence diagram shows participants as vertical lifelines and messages as arrows read from top to bottom. c4model.com's own supporting diagram for this, the dynamic diagram, is described as showing "how elements in the static model collaborate at runtime to implement a user story, use case, feature", and the site says its collaboration and sequence styles show the same information, so either may be used.

Mermaid's sequence syntax gives you what the design needs: participant X as Name for aliases; solid arrows with a head (->>) for requests and dotted arrows (-->>) for replies; an open-headed arrow (-)) for an asynchronous message; Note over A,B: text; blocks such as alt, else, opt, loop and par; and autonumber. The Mermaid documentation warns that the word "end" can break a diagram, that a semicolon acts as a line break, and that # needs an entity code. The house rule on this site is to avoid those characters in message text.

Rule of thumb: draw one sequence per scenario, with at most about eight participants, and put the decision in a note at the point where the system makes it.

Figure 4. Normal question, answered from the policy documents

How to read this: time runs downward, solid arrows are requests, dotted arrows are replies, the open-headed arrow is asynchronous, and the note marks a decision the Orchestrator makes.

Read against the numbers: in steps 1 and 2 the gateway authenticates the session and passes the verified identity on, so the Orchestrator never trusts an identity claim from the Client app directly. Step 3 sends the user's entitlements with the search so the Retriever can restrict results to documents this customer may see. Step 4 returns passages with ids and versions, and those are what the citations can refer to. Steps 5 to 8 are the model call through the Model gateway, which is the only participant that talks to the LLM provider. The note after step 8 is the output check from Figure 3: an answer that cites a passage the Retriever never returned is rejected. Step 9 is asynchronous, so the audit write is off the customer's critical path. Steps 10 and 11 return the answer.

The happy path is the least informative diagram you can draw. The next two show what the design does when things go wrong.

Figure 5. Failure path: the retrieval step returns nothing or times out

How to read this: the alt block shows two alternatives that end in the same customer message, and the notes mark the decisions.

The point of this diagram is a decision that is easy to skip: with no evidence, the Orchestrator does not call the model at all. A model asked a policy question with no passages can still produce a fluent answer from its training data, which is exactly what the system promised not to do. The diagram also shows a single retry-free timeout on the Retriever and an honest user-facing outcome (an offer of a ticket instead of a guess). Whether to retry the Retriever once is a design choice; if you choose to, the diagram must show it and the timeout budget must pay for it.

Figure 6. Failure path: the ticket tool times out

How to read this: the two notes explain the timeout and who owns retries, and the alt block shows the success and failure endings of the single retry.

This diagram carries three decisions. First, the Policy engine is asked before the call, not after, and a deny would end the sequence at step 2. Second, the retry reuses the same idempotency key k1: the first attempt may have reached the ticketing system even though its reply was lost, so a second call without the key could create a duplicate ticket. An idempotency key is a value the caller supplies so that repeating the same request has the effect of doing it once; somewhere on that path (the Tool server, or a ticketing system that honours keys) a store of keys must return the existing ticket id when it sees a repeat. Third, exactly one layer retries. Multi-layer retries are drawn in detail in the billing-dispute case file; here it is enough to note that the Google SRE book warns against "amplifying retries by issuing retries at multiple levels", with a worked case of three layers each making 4 attempts producing 4^3 = 64 attempts on the database for one user action.

The deployment view

The deployment view answers where each container runs and who controls the network between them. c4model.com's definition: its scope is one or more software systems in a single deployment environment (production, staging and so on), its primary elements are deployment nodes, software system instances and container instances, deployment nodes can be nested, and infrastructure nodes such as DNS services, load balancers and firewalls may be included. It also says any icons you use must be in the diagram key. This page uses no icons.

Figure 7. Deployment view of the production environment (instance counts are illustrative)

How to read this: nested boxes are deployment zones, each component box says how many instances run, and arrows follow the same directions as Figure 2.

The deployment view makes questions concrete that the container view could not: the Model gateway and the Tool server are the only containers with a path out to third parties, so egress rules attach to those two. The data zone has no inbound path from the public zone. The Eval service is left out of the figure to keep it small. A second environment (staging) needs its own diagram, because instance counts, network rules and even which third-party service is real will differ.

State diagrams for agents and workflows

When a component can be in one of several modes and the rules for moving between modes matter, draw a state diagram. An agent loop is the standard case: the request moves between retrieving, reasoning, deciding, checking policy and calling a tool, and it may loop. Mermaid's stateDiagram-v2 supports [*] for start and end, labelled transitions, composite states and a <<choice>> node for a branch, which is the construct used here.

Figure 8. State of one request in the Orchestrator's agent loop

How to read this: each rounded box is a state, each arrow is a transition with the condition that triggers it, the diamond is a choice made in one instant, and the black dot and ringed dot are start and end.

The value of the state diagram is exhaustiveness. Count the exits: seven terminal outcomes (answered, rejected, no evidence, model failure, step limit, refused, tool failure), and the rule that every one of them emits exactly one audit event. If a reviewer can name a situation that has no outgoing arrow, the design has a gap. The loop CallingTool to Reasoning is the only cycle, and the choice node is the only place that tests the step budget, so an agent that keeps asking for tools ends in StepLimit instead of running up cost indefinitely. A state diagram does not replace the sequence: it says which moves are legal, not what the messages are.

Data-model views

A data-model view is an entity-relationship diagram. Use it for the data that has to survive a request: audit records, the evaluation dataset, anything a regulator, an on-call engineer or the Eval service will query later. Mermaid's erDiagram writes a relationship as A ||--o{ B : label. Each side has two characters, the outer one the maximum and the inner one the minimum: || exactly one, o| zero or one, }o/o{ zero or more, }|/|{ one or more. A solid line is an identifying relationship (the child cannot exist without the parent) and a dashed line is non-identifying. Attributes are written as type name and may be marked PK, FK or UK.

Figure 9. Data model of the audit log and the eval dataset

How to read this: each box is a table, each line is a relationship whose end marks say one or many, and PK and FK mark keys.

Four design decisions are visible in this picture. The request carries prompt_version and model_id, so an answer can be tied to the exact configuration that produced it, which is what makes a regression traceable. Retrieved passages are stored as doc_id plus doc_version, not as copied text, so the log records what evidence was used without duplicating policy text, and a later question like "which answers cited the old returns policy?" is a query. The audit events hold identifiers and outcomes, not the conversation text; that is a choice this teaching system makes to keep personal data out of the log, and a team with a different retention rule would change it. And an eval case may point back to a real request (source_request_id is nullable, so hand-written cases fit too), which closes the loop between production traffic and the Eval service. Note the tension this creates: the request record keeps no question text, but promoting a request into an eval case copies its question into the dataset, so that step needs its own redaction and consent rule.

Trust boundaries in diagrams

A trust boundary is drawn where control changes hands. Microsoft's Threat Modeling Tool documentation says its trust boundaries show "where different entities are in control", and threat-modelling sources draw them on a data flow diagram (DFD). The OWASP Threat Modeling Cheat Sheet says DFDs "visually model a system, its data flows, and interactions with external entities", asks for a clear view of trust boundaries, data flows, data stores, processes and the external entities that interact with the system, and says threat modelling is "ideally performed early in the SDLC, such as during the design phase" and that a model "should be maintained, updated and refined alongside the system". Microsoft's example DFD uses the same elements (an outside entity, a process, a data store). Both sources use STRIDE to ask what can go wrong at each crossing: spoofing, tampering, repudiation, information disclosure, denial of service and elevation of privilege.

For an AI system the useful extra is that model output and retrieved text are data from the wrong side of a boundary, even though they arrive from components you run. The prompt contains text you did not write (the customer's message, retrieved passages), and that text can influence what the model produces. So the Orchestrator treats the model's reply as a request to be checked, not a command to be executed. That is why the Policy engine sits between the agent loop and the Tool server.

Figure 10. Trust-boundary view of the policy Q&A assistant

How to read this: thin arrows stay inside a zone, thick arrows cross a trust boundary, and every thick arrow has a label starting B1 to B5 that names the control at that crossing.

Use this view as a review checklist: each thick arrow is a place to ask what an attacker or a mistake could do. B1 is spoofing and tampering from an untrusted device. B2 is where identity becomes a verified claim instead of a request header. B3 is the crossing where access control is enforced for retrieval, so one customer's documents never rank for another. B4 is a two-way crossing: outbound text leaves your control, and the reply comes back into it as untrusted data. B5 limits the Tool server's credential to the one thing it needs. The first case file below is a review in which a crossing, between the Orchestrator and a shared Tool server, was missing from the diagram, and with it the control.

Decision trees

A decision tree is a flowchart whose nodes are questions and whose leaves are outcomes. Use one when a runtime or design choice has several conditions and people disagree about the order, or when you want a rule a reader can follow without asking the author. Keep questions answerable with yes or no, and make every leaf an action. The tree below answers one question: which diagram to draw.

Figure 11. Which diagram answers my question?

How to read this: start at the top, follow yes or no at each diamond, and stop at the first rectangle.

The questions run from broad to narrow, so a team that skips the context view is pointed to it first. A real design needs several leaves, not one: the tree picks the next diagram, not the only one. It also ends in prose, because sometimes a sentence is the better picture.

The diagram checklist

Apply this list to every diagram in a review. Several items come from c4model.com's notation guidance (title, key, labelled lines, stand-alone readability, and attention to black-and-white printing and colour blindness); the node limit and the failure-path rule are house rules for this course.

Case files

Two composite case files show these diagrams doing real review work. Both are illustrative scenarios, not real companies.

• Helpdesk assistant design review: a missing trust boundary. Problem: a B2B support assistant let the model choose the tenant id passed to a shared tool server that could read every tenant. Pattern: draw trust boundaries on the container view and ask what controls each crossing.
• Billing dispute assistant: retries in three layers. Problem: three layers each retried a failing call, which would multiply load in an outage. Pattern: a runtime sequence with the retry loops visible shows who owns retries and exposes amplification the container view hides.

Decision framework

1. If the audience includes anyone who is not an engineer, then start with the context view, because it is the only level that everybody can read; revisit when the set of external systems changes.
2. If a flow has three or more participants or any asynchronous step, then draw a sequence diagram instead of describing it in prose, because ordering and ownership of retries are invisible otherwise; revisit when a participant is added.
3. If a component can loop or can end in more than one way, then add a state diagram and check that every state has an exit, because unbounded loops are a cost and safety failure; revisit when the step budget or a terminal outcome changes.
4. If data crosses from a party you control to one you do not, or model output or retrieved text can trigger an action, then mark a trust boundary and name its control on the diagram, because the crossing is where review time is best spent; revisit when a tool, a data source or a provider is added.
5. If a diagram passes about 15 nodes, then split it into an overview and a zoom-in, because a reviewer who cannot hold the picture cannot challenge it; revisit when a split view starts to repeat the same boxes.
6. If the team will change the system faster than it updates slides, then keep diagrams as text in the repository and review them in pull requests, because a diagram not checked with the code is wrong within weeks; revisit if tooling cannot render the text format.

Comparison of the main options

Failure scenarios to design for

These are failures of the diagramming practice, each with the behaviour a team should define in advance.

1. The diagram and the system drift apart. Defined behaviour: diagrams are text files in the same repository, changed in the same pull request as the code they describe, and the design-review checklist asks whether the pull request touches a diagram.
2. The diagram grows past what a reviewer can read. Defined behaviour: when a flowchart passes about 15 nodes it is split into an overview plus zoom-in views, and the overview names which zoom-in covers which box.
3. Only the happy path is drawn. Defined behaviour: every sequence for a user-facing flow has at least one companion for the empty-result, timeout and denial paths, and the state diagram has no state without an exit.
4. Names change between diagrams. Defined behaviour: a shared name list (like the table earlier on this page) is part of the design document, and a renamed component is changed everywhere in one commit.
5. Meaning is carried by colour alone. Defined behaviour: legends describe shape and line style, and the diagram is reviewed once in greyscale before it is accepted.
6. A trust boundary exists in the system but not in the picture. Defined behaviour: the review walks every outbound arrow from the application zone and asks which boundary it crosses and who holds the credential; an arrow with no answer is a finding.

Worked example

Use Figures 4 and 6 to set the timeout budget for the longest path in the system: a customer question that ends in a ticket. A budget is the maximum time each step may take, so that the sum fits inside the deadline the customer experiences.

Assumptions: these figures are illustrative, not measurements. The customer-facing deadline is 20 s. The gateway and identity step is budgeted at 0.1 s. Retrieval has one attempt with a 1.0 s timeout. Each model call has a 6 s budget in total (including any retry the Model gateway makes), and one call precedes the tool request and one follows the tool result. The Policy engine check is budgeted at 0.1 s. The ticket call has a 3 s timeout, one retry after 0.5 s backoff and no other layer retries.

Worst-case ticket step: first attempt 3.0 s, backoff 0.5 s, second attempt 3.0 s, so 3.0 + 0.5 + 3.0 = 6.5 s.

Worst case for the whole request:

gateway and identity        0.1
retrieval (1 attempt)       1.0
model call 1 (tool request) 6.0
policy check                0.1
ticket call with 1 retry    6.5
model call 2 (final text)   6.0
total                      19.7 s   (deadline 20 s, margin 0.3 s)

The sum is 0.1 + 1.0 + 6.0 + 0.1 + 6.5 + 6.0 = 19.7 s, which leaves 0.3 s. The margin is the result: it says the design has no room for a second retry anywhere, and that a slower model timeout must be paid for by removing something else. Compare the failure of drawing retries in several layers. If the gateway, the Orchestrator and the Tool server each made up to 3 attempts on the same ticket call, one customer action could reach 3 x 3 x 3 = 27 attempts at the ticketing system. That is also why Figure 6 states in a note that the Orchestrator is the only layer that retries. The AWS Builders' Library makes the same point for a five-deep stack of calls with three retries at each layer, where independent retries increase the load on the database 243x (the article gives the figure without showing the arithmetic), and its best practice is to retry at a single point in the stack.

A diagram lint for pull requests

Diagrams are only checked if a machine checks them. The repository's own checker parses every Mermaid block, but a short script can enforce the house rules that parsing cannot: each block must have accTitle and accDescr, and a flowchart should not exceed about 15 nodes. The node count is a heuristic, because it counts ids that are declared with a shape after quoted labels are blanked out, so treat a warning as a prompt to look and not as proof.

import re, sys

MAX_NODES = 15
FENCE = "`" * 3
BLOCK = re.compile("^" + FENCE + r"mermaid\s*\n(.*?)^" + FENCE, re.S | re.M)
NODE = re.compile(r"(?<![\w-])([A-Za-z]\w*)\s*(?:\(\[|\[\(|\[\[|\[|\(|\{)")

def lint(path):
    text = open(path, encoding="utf-8").read()
    problems = []
    for n, m in enumerate(BLOCK.finditer(text), 1):
        body = m.group(1)
        first = next((l.strip() for l in body.splitlines() if l.strip()), "")
        if "accTitle:" not in body or "accDescr:" not in body:
            problems.append(f"diagram {n}: missing accTitle or accDescr")
        if first.startswith(("flowchart", "graph")):
            ids = set()
            for line in body.splitlines():
                if line.strip().startswith(("accTitle", "accDescr", "subgraph")):
                    continue
                line = re.sub(r'"[^"]*"|\|[^|]*\|', '""', line)
                ids.update(NODE.findall(line))
            if len(ids) > MAX_NODES:
                problems.append(f"diagram {n}: {len(ids)} nodes, over {MAX_NODES}")
    return problems

if __name__ == "__main__":
    bad = {p: lint(p) for p in sys.argv[1:]}
    for p, errs in bad.items():
        for e in errs:
            print(f"{p}: {e}")
    sys.exit(1 if any(bad.values()) else 0)

Run it as python lint_diagrams.py content/learning/*.mdx in CI. It skips subgraph lines so that zone titles are not counted as nodes, and it reads the first non-empty line to decide the diagram type.

Common mistakes

• Mixing levels in one picture. A box for the whole system next to a box for a single class. Each diagram should have one scope; c4model.com defines a scope for every diagram type.
• Calling a library a container. In C4 a container is something that must be running for the system to work, an application or a data store. A client library inside the Orchestrator is a component or a code element.
• Unlabelled arrows. "Uses" tells the reviewer nothing. Say what travels and, between containers, which protocol.
• A sequence diagram that is only the happy path. If every alt block is missing, the diagram shows what everyone already believes.
• Treating model output as trusted. A diagram where the arrow from the LLM provider goes straight to the Tool server hides the most important check in the design.
• Colour as the only legend. Red boxes for "risky" mean nothing in greyscale. Use a shape, a line style or a word.

Try it

Time: 30 to 40 minutes. Pick one AI feature you know (or invent a small one, such as a meeting-summary assistant that posts to a chat tool).

1. Write its one-sentence caption: who asks what, and what the system answers or does.
2. Draw its context view with at most seven boxes, and its container view with at most 15 nodes, as Mermaid flowcharts.
3. Draw one sequence diagram for the normal path, and one for a failure path: an empty retrieval or a tool timeout. Put a note at each decision.
4. Mark every trust boundary on the container view with thick arrows and a label that names the control.
5. Walk the checklist above, then run node scripts/check-diagrams.mjs <your file> if your file is in this repository, or any Mermaid parser otherwise.

Done when each diagram has a figure title, an accessible title and description, consistent component names, and you can point to the failure path and the trust boundary for the feature.

Key takeaways

• A diagram answers one question for one audience. Pick the type from the question, not from habit.
• C4 gives four zoom levels and a notation discipline (titles, keys, labelled lines, stand-alone readability). On this site its views are drawn as Mermaid flowcharts with subgraphs.
• Sequence diagrams carry the runtime decisions: what happens on empty retrieval, who owns the retry, where the idempotency key goes.
• A state diagram proves a loop terminates and that every outcome has an exit and an audit event.
• A data-model view makes the audit log and the eval dataset explicit and links production requests to the Eval service.
• Mark trust boundaries and treat model output and retrieved text as untrusted data. The Policy engine belongs between the agent loop and the tools.
• Keep diagrams as text in the repository and check them with the code. About 15 nodes is the limit for one view.

Self-check

Further reading

• Simon Brown, The C4 model: abstractions, diagram types and notation guidance.
• Mermaid documentation: sequence diagrams, state diagrams, entity relationship diagrams and accessibility.
• OWASP, Threat Modeling Cheat Sheet.
• Microsoft Learn, Threats in the Microsoft Threat Modeling Tool (the STRIDE categories) and Getting started with the tool (data flow diagrams and trust boundaries).
• Google, Site Reliability Engineering: Addressing Cascading Failures.

Last verified: 2026-10-21

Next

This is the last page of Part A. Part B starts with page 4: Models, Routing and Inference for AI Architects.