Skip to content
    All guides

    AI Architect Case Files

    Case: Keeping Model Processing in One Region, With Proof

    A public agency's drafting assistant must process records in one named region. Region pinning, in-region logs and a daily audit check provide the evidence.

    By Published 7 min read

    • ai-architect
    • case-file
    • industry-public-sector
    • pattern-data-residency
    • failure-compliance-gap
    • kind-composite
    Share

    Composite scenario, not a real company. Assembled from common industry patterns; every number is illustrative and its assumptions are listed at the end.

    This case belongs to Cloud platforms for AI.

    NOTE

    Not legal advice. The case shows how to turn a residency rule into settings and evidence. Whether a given setting satisfies a given law or contract is a question for the agency's counsel. Provider facts are as of 2026-10-23.

    Context

    • A regional agency's caseworkers answer citizen letters. An assistant drafts a reply from the letter and the case notes, and a caseworker edits and sends it.
    • The agency's rule says processing of these records must stay in one named region.
    • The service handles 40,000 requests a day through a Model gateway to a managed model service.
    • Logs, a search index of case notes and nightly backups are owned by the agency.

    Problem statement

    Requirement Measure
    R1: inference stays in the named region 0 of every request processed elsewhere
    R2: everything the agency owns stays in the region Logs, index, backups and keys all in the region
    R3: the claim can be proved Every event checked daily, not a sample
    R4: content logs hold no raw personal data Redaction before storage, retention set

    Constraints

    • The rule names a region, not a geography, so geography-wide options are suspect.
    • No automatic failover outside the region.
    • The provider's pages, not the agency's assumptions, define where processing can happen. Only AWS documents an in-Region option: "Your requests never leave the AWS Region you specify". Azure's and Google's narrowest documented levels are a geography and a jurisdiction, so on those clouds R1 needs counsel's view or option E.

    Why the traditional approach fell short

    Hand-written replies are slow and the backlog grows. Template libraries fit common letters but cannot read free text. A contract clause that promises residency is not evidence: auditors ask for a record of where each request was processed, and an application log that shows only the call you made cannot answer that.

    Options considered

    Option Why accepted or rejected
    A. Non-AI: caseworkers draft from templates Kept as the degraded mode, because the rule forbids routing elsewhere during an outage
    B. Managed service with global routing Rejected: AWS describes global profiles as "routed worldwide", and Google says global endpoints "don't provide any data residency guarantees"
    C. Geographic profile, data zone or multi-region endpoint Rejected for a region-level rule: these stay inside a wider boundary. AWS says prompts and results "might move outside of your source Region"; Azure says processing "might be processed between regions within that geography"
    D. Managed service pinned to one region, plus a daily audit check Accepted, on a provider that documents an in-region option
    E. Self-hosted model in the region Revisit if no provider offers the named region: the agency would run capacity and patching

    Solution architecture

    Figure 1. A pinned call path, with logs and keys in the same region

    Pinned region call path with in-region logs and keys and a daily audit jobClient app to Orchestrator; Orchestrator to Model gateway; Model gateway to Managed model service (plain call); Managed model service to Key service (wrap, unwrap); Managed model service to Provider audit log (events); Provider audit log to Audit job (read); Audit job to Log store (daily result)THE NAMED REGIONClient app: caseworkersClient appcaseworkersOrchestratorOrchestratorModel gateway: region guardModel gatewayregion guardManaged model service: named regionManaged modelservicenamed regionKey serviceKey serviceProvider audit log: Region, model IDProvider auditlogRegion, model IDAudit job: daily, every eventAudit jobdaily, everyeventLog store: in regionLog storein regionplain callwrap, unwrapeventsreaddaily result

    How to read this: everything the agency owns for this feature sits inside one regional boundary, and the audit job reads the provider's own record of each call.

    Figure 2. One request and the check that proves it

    sequenceDiagram accTitle: Pinned request and daily residency check accDescr: The Orchestrator sends a draft request to the Model gateway. The gateway checks its configuration is the allowed region and a plain call, then calls the managed model service in that region. The service writes an event with the Region and model ID to the provider audit log. Once a day the audit job reads every event and compares them to the allowed set. Any mismatch opens an incident. autonumber participant O as Orchestrator participant M as Model gateway participant S as Managed model service participant A as Provider audit log participant J as Audit job O->>M: draft request Note over M: Config allows only the named region, no routing profile M->>S: plain call in the named region S-->>M: draft M-->>O: draft S-)A: event with Region and model ID J->>A: read the day's events A-->>J: 40,000 events alt any event outside the allowed set J-)J: open an incident end

    How to read this: steps 1 to 5 are the request, and steps 6 and 7 are the evidence loop that runs separately.

    How AI solved it, step by step

    One draft request, with the region pin and without it (interactive when JavaScript is on)

    As designed

    1. The request arrives. A caseworker asks for a draft reply. The Orchestrator builds the prompt from the letter and the case notes and sends it to the Model gateway.
    2. The gateway enforces the pin. Decision: the gateway configuration allows one region and a plain model call. A routing profile or a global endpoint name fails validation in CI and at start-up.
    3. Inference runs in the region. The managed model service processes the request in the named region and returns the draft.
    4. The event is written. The provider audit log records the call with the principal, the Region and the model ID. Content is not logged unless redaction is in place.
    5. The daily check. The Audit job reads every event for the day and compares the Region, the model ID and any inference region with the allowed set.
    6. The evidence is kept. The result (events checked, out-of-region count) is stored in the in-region log store with a retention tag, and any mismatch opens an incident.

    Without the fix

    1. A geographic profile is chosen. The team picks the profile because it is described as the compliant option for a geography.
    2. Requests move. Some requests are processed in other regions of the geography. The application log shows only the call the app made.
    3. The auditor asks. Where was each request processed? The app log cannot say, and nobody reads the provider audit log.
    4. A finding. The agency cannot show its R1 and R3 claims and must either change the setting or explain the gap.

    Under the hood

    The proof rests on what the provider's audit record contains.

    • AWS: the cross-Region inference page says CloudTrail logs all cross-Region inference requests "in your source Region", and the additionalEventData.inferenceRegion field identifies where requests were processed. The CloudTrail page's example puts inferenceRegion in the stream-completion event, a second event for each streaming call that is not an API call and must not be counted as an invocation. Its example for a call shows awsRegion and requestParameters.modelId. The pages read do not say whether a plain call carries inferenceRegion. Invocation logs go to a destination in the same account and Region.
    • Azure: data stored at rest stays in the designated geography for every deployment type, and only the place of processing changes between Standard, Data Zone and Global types.
    • Google: the data residency page makes the same split between data at rest and ML processing.

    So the check flags any event whose Region is outside the allowed set, whose model ID has a geography or global prefix (a profile), or whose inferenceRegion is outside it. Only AWS's pages read name such a field. Content logging stays a separate decision, because the check needs metadata, not prompts. The audit log is the provider's record, and whether it satisfies a legal test is counsel's question. Mechanism: Under the hood: Logging and audit trails in a managed AI service.

    What went wrong / failure modes

    Failure Designed mitigation
    The gateway configuration drifts to a routing profile CI validates the configuration and the Model gateway refuses to start with it
    A log bucket is created in another region Infrastructure policy denies it. AWS invocation logs already require a destination in the same Region
    A backup or replica of the index sits elsewhere List every copy in a residency register and treat replicas as in scope
    The region has an outage No failover outside the boundary: the Orchestrator switches to template drafting and shows a banner
    A caseworker pastes a citizen's identifier into the prompt Redact before any content log, and keep content logging off by default
    The audit job misses a day Alert when fewer events than the request counter, and re-run

    Results

    These are illustrative targets, not measurements.

    Measure Before Target
    Requests processed outside the named region Unknown 0 of 14,600,000 a year
    Events checked None 100%, daily
    Time to detect a violation Next audit At most 1 day
    Log volume (metadata and redacted summary, 6 KB each) n/a 240 MB a day, 87.6 GB a year

    Tradeoffs and when NOT to use this approach

    • A single region has one region's capacity and quotas, with no cross-region spill-over.
    • Fewer models may be available in one region than globally.
    • If the rule is written at geography level, a geographic option may be enough and gives more capacity.
    • If no provider offers the named region, the choice becomes a self-hosted model, with its own cost.

    Design rules extracted

    1. Write the processing boundary into the requirement (region, country or geography) before choosing a setting.
    2. Choose the narrowest routing level that satisfies it.
    3. Everything the agency owns for the feature (logs, index, backups, keys) goes inside the same boundary.
    4. Evidence is a daily check of every event, not a policy document.
    5. Degrade inside the boundary rather than fail over outside it.

    Assumptions

    Last verified: 2026-10-23

    Ask about this article

    Get answers grounded in this post. AI-generated — based on this article, and may be imperfect.

    Scaled AI Weekly

    Enjoyed this? Get more like it every Monday.

    Real architecture decisions, LLMOps patterns that survive production, and engineering leadership advice — from 12+ years of building at enterprise scale. Free. No spam. Unsubscribe anytime.

    Get new articles by email

    Free. We email a confirmation link first, and you can unsubscribe from any email.

    Join engineers building production AI systems

    Comments

    Back to all guides