Exam domain covered: D5 — Governance, Safety & Risk Management (14%)
Approximate module runtime: 114 minutes of Anthropic learning content
The Alignment Boundary
The alignment boundary separates what trained behavior already handles from what you must still enforce at the application layer:
| Side | Responsible party | What it covers |
|---|---|---|
| Training-time | Anthropic | Broad, general harm reduction — refusing weapons synthesis, broad safety defaults, constitutional behavior |
| Inference-time | You (the operator) | Domain-specific policy, data authorization rules, application-specific constraints |
The critical implication for compliance: You cannot audit, test, or produce evidence of training-time behavior from the application level. Any compliance requirement that needs a named control, a named owner, and an inspectable evidence artifact must be enforced at inference-time layers you control — not asserted as "the model doesn't do that."
The training-time boundary is also why you cannot tell a model "you were never trained to do X" as a security measure. If X is not in trained behavior's scope, the statement is meaningless; if it is, the statement is redundant.
The Four-Layer Safety Stack
Memorise all four layers AND each layer's specific blind spot. The exam asks which blind spot a specific architecture fails to address.
| Layer | What it does | Its specific blind spot |
|---|---|---|
| 1. Trained behavior | Reduces broad, general harm categories across all uses | Cannot know your domain-specific rules — it was never given them |
| 2. System-prompt instruction | Applies context-specific policy for your application | Instructions can be ignored, misinterpreted, or overridden — an instruction is not enforcement |
| 3. Runtime screening | Scans inputs and/or outputs for policy violations | Cannot see malicious instructions arriving via retrieved content or tool outputs (indirect injection) |
| 4. Authorization | Controls whether this caller may take this specific action in this context | Timing attacks and race conditions if not implemented atomically |
The architecture pattern: each layer's blind spot is exactly what the next layer catches. Removing any layer leaves its successor's blind spot unaddressed.
The exam application: a question describes a system that relies only on the system prompt to enforce access control. Which layer is missing? Layer 4 (authorization). Which blind spot does this create? The instruction can be worked around.
Fail-Open vs Fail-Closed
This is a per-guardrail decision — not a blanket policy across the system.
| Scenario | Decision | Reasoning |
|---|---|---|
| Guardrail gates access to sensitive PII | Fail-closed (deny on error) | A guardrail that silently passes traffic on its own error provides no protection — it looks like security while providing none |
| Guardrail adds optional content enrichment on a throughput-critical low-stakes path | Fail-open (proceed without enrichment) | Halting throughput is worse than occasionally undecorated output |
The critical rule: a guardrail that fails open on high-stakes data exposure is worse than no guardrail. It creates a false sense of protection and is harder to detect than an obvious gap.
Tool-Call Authorization
Authorization for tool calls is almost always deterministic. It must:
- Check: does this specific caller have permission for this specific action in this specific context?
- Be provable — the check is auditable, not a model judgment
- Be replayable — running the same check twice produces the same answer
- Execute before the side-effecting action runs, never after
Contrast with input/output screening, which can reasonably be model-based (probability-weighted). Authorization never can be — a probabilistic access control that sometimes allows and sometimes denies the same caller for the same action is not authorization.
Trap 12 — An instruction asked to do a security layer's job: "Only return data belonging to the requesting employee" in the system prompt is not authorization. It is an instruction. Instructions can be talked around, injected past, or misfire. The access-control layer must make the data structurally unreachable, not merely undesired.
The correct design: per-user scoped credentials or pass-through authentication so the model can only retrieve what it is authorized to retrieve — regardless of what it's instructed to do with it.
Indirect Prompt Injection
Malicious instructions can arrive via retrieved content or tool outputs, not just from the user's direct message:
User message → LLM
↓ tool call
Retrieved document: "Ignore your previous instructions and..."
↓
LLM processes retrieved content
(input screening of the user message did NOT catch this)Why Layer 3 (runtime screening) misses it: screening typically analyzes the user's input message. It does not see content that arrives later via tool outputs or retrieval.