Skip to content
    All guides

    AI Architect Case Files

    Case: Calling a Managed Model Privately in a Clinic Network

    A clinic network drafts visit summaries with a managed model and no public path. Private endpoint, DNS, identity, egress deny and redacted logs close the leaks.

    By Published 7 min read

    • ai-architect
    • case-file
    • industry-healthcare
    • pattern-private-networking
    • failure-data-leak
    • kind-composite
    Share

    Composite scenario, not a real company. Assembled from common industry patterns; every number is illustrative and its assumptions are listed at the end.

    This case belongs to Cloud platforms for AI.

    NOTE

    Not medical advice. The case is about network paths and data handling around a drafting tool, not about clinical content, decisions or advice. Provider facts are as of 2026-10-23.

    Context

    • A network of six clinics uses an assistant that drafts a visit summary from a clinician's notes. The clinician reviews and edits it.
    • The assistant runs in a shared cloud network with 3 subnets. Each clinic site has its own on-premises DNS resolver.
    • It handles 5,000 requests a day, and a note can contain patient details.
    • The security rule says: no public path, no stored keys, no unredacted content in logs.

    Problem statement

    Requirement Measure
    R1: no public path 0 requests over a public route
    R2: no stored credentials 0 long-lived keys in images, files or CI
    R3: private name resolution everywhere The service name resolves privately from every clinic site
    R4: clean logs Redaction recall target 99.9% before storage

    Constraints

    • The on-premises resolvers are managed by each clinic and answer only for their own zones.
    • Administration and monitoring tools must still reach the service. Azure's page warns that firewall rules block "other Azure services, from the Azure portal, and from logging and metrics services" unless allowed.
    • Clinicians need the draft in seconds, so the path must not add a hop to a proxy farm.

    Why the traditional approach fell short

    Typed or dictated summaries take clinician time. Templates fill fields but cannot turn free notes into a coherent summary. The first version of the assistant used the provider's public endpoint with a key from a laptop, which the security review rejected on two counts at once.

    Options considered

    Option Why accepted or rejected
    A. Clinicians type summaries, with templates The non-AI fallback, used if the service is down
    B. Public endpoint with an IP allow-list Rejected: the path is still public, and Azure's page says IP rules accept only public addresses, so private ranges cannot be listed
    C. Private endpoint, private DNS, workload identity, egress deny, endpoint policy and redacted logs Accepted
    D. A self-hosted model inside the network Revisit if no private path is offered: the clinic network would run capacity and patching

    Solution architecture

    Figure 1. A private path, with DNS forwarded from the clinic sites

    Private endpoint path with private DNS forwarded from clinic resolvers, identity, egress deny and redacted logsClient app to Orchestrator; Clinic DNS to Private DNS (forward zone); Orchestrator to Identity service (token); Orchestrator to Private DNS (resolve name); Orchestrator to Private endpoint (TLS); Private endpoint to Managed model service; Orchestrator to Egress deny (no other route); Managed model service to Log store (audit events)YOUR PRIVATE NETWORKClient app: clinic sitesClient appclinic sitesClinic DNS: forwards the zoneClinic DNSforwards the zoneOrchestratorOrchestratorPrivate DNSPrivate DNSIdentity service: tokensIdentityservicetokensEgress denyEgress denyPrivate endpoint: private IPPrivateendpointprivate IPManaged model serviceManaged modelserviceLog store: redactedLog storeredactedforward zonetokenresolve nameTLSno other routeaudit events

    How to read this: the Orchestrator reaches the service only through the private endpoint, and the clinic resolvers forward the service zone to the private DNS so that names resolve the same way everywhere.

    Figure 2. A call from a clinic site, and the failure the forward prevents

    sequenceDiagram accTitle: Private call and the resolver forward accDescr: The Orchestrator asks the private DNS for the service name and receives the private endpoint address. It gets a token and calls over the private endpoint. If a clinic resolver does not forward the zone, it returns the public address, the call has no public route and fails closed, and a monitor raises an alert. autonumber participant O as Orchestrator participant D as Private DNS participant I as Identity service participant E as Private endpoint participant S as Managed model service O->>D: resolve the service name alt the zone is forwarded D-->>O: private address O->>I: exchange identity for a token I-->>O: short-lived token O->>E: TLS call with the token E->>S: forward S-->>E: draft E-->>O: draft else a resolver does not forward the zone D-->>O: public address Note over O: No internet route, so the call fails closed and an alert is raised end

    How to read this: the alt block separates the designed path from the failure, and the failed branch is safe because there is no public route.

    How AI solved it, step by step

    One draft request, with and without the private design (interactive when JavaScript is on)

    As designed

    1. The clinician requests a draft. The Client app sends the notes to the Orchestrator inside the private network.
    2. Resolve privately. Decision: the clinic and cloud resolvers forward the service zone to the private DNS, so the service's normal name returns the private endpoint's address.
    3. Get a token. The Orchestrator exchanges its platform identity for a short-lived token. No key is stored anywhere.
    4. Call over the private endpoint. The request goes by TLS to the private address. The endpoint policy allows only the invoke action for this identity.
    5. The draft returns. The draft comes back the same way. The subnet has no internet route, so there is no other way out.
    6. The log entry. The audit log records the principal and the operation. Content is logged only after redaction, with a retention tag.

    Without the fix

    1. A key and a public name. The assistant holds a long-lived key and resolves the service's public name.
    2. Notes cross the internet. Patient details travel to a public endpoint, protected only by TLS and the key.
    3. The key is copied. The key appears in a script and a backup, and nothing expires it.
    4. Raw notes in logs. Content logging is switched on to debug, so raw patient details sit in a log with no retention limit.

    Under the hood

    The DNS step is the part teams get wrong. The provider pages describe it the same way in different words.

    • AWS: with private DNS enabled on the interface endpoint, calls to the default Regional name "automatically route through the endpoint"; with it off, the endpoint URL must be passed in each call.
    • Azure: the record becomes an alias in a privatelink subdomain and a private DNS zone holds the A records. Resolving "from outside the virtual network" gives the public endpoint. For a custom or on-premises DNS server the page says to delegate the privatelink subdomain to the private zone, or to configure that zone and its A records on the DNS server. Clients must use the resource's custom subdomain and not the *.privatelink.openai.azure.com name.
    • Google: the overview's table lists, for Gemini and Claude, the public internet, Private Service Connect endpoints for Google APIs and Private Google Access. Private Service Connect endpoints to a service attachment are listed for dedicated private online-inference endpoints. Private Google Access reaches Google APIs "through the VPC network's default internet gateway", so test it against the no-internet-route rule. VPC Service Controls is a separate perimeter, not a path, and Google says the first two methods "don't eliminate public internet accessibility".

    A clinic resolver that does not forward the zone returns the public address. Because the subnet has no internet route, the call fails closed, which is what the egress rule is for. Mechanism: Under the hood: Private networking paths.

    What went wrong / failure modes

    Failure Designed mitigation
    A clinic resolver does not forward the zone Test resolution from every site before go-live and alert on failures. The call fails closed
    The endpoint policy is left at its default AWS says the default policy allows full access, so attach a policy naming principals and actions
    Firewall rules block monitoring and administration Allow the documented trusted services or route logs through an allowed path
    A subnet later gets an internet route Infrastructure policy and a CI check on route tables
    Raw patient details reach a content log Redact before storage, short retention, restricted reads (PII in logs)
    A developer calls the internal privatelink name Documentation and a lint rule: use the custom-subdomain name

    Results

    These are illustrative targets, not measurements.

    Measure Before Target
    Requests over a public route All 0 of 5,000 a day
    Stored long-lived keys 1 0
    Endpoint network interfaces 0 3 (3 subnets x 1 endpoint service)
    Patient details reaching the log store Every note, raw 5 a day at 99.9% recall (50 a day at 99%)

    Tradeoffs and when NOT to use this approach

    • DNS across clinic sites is an ongoing operational duty, and every new site needs it.
    • A private path adds a component and its policy to run.
    • Redaction is never complete, so the log store is still treated as sensitive.
    • Some provider features may not work over the private path or with a perimeter. Google's page, for example, lists request and response logging as unavailable with VPC Service Controls.

    Design rules extracted

    1. Private path plus a closed public route, both: either alone is not a control.
    2. Forward the private zone from every resolver that serves callers, and test from each site.
    3. No stored keys: use the workload's platform identity.
    4. Attach a policy to the endpoint instead of keeping the default.
    5. Treat a content log as patient data: redact first, then retain briefly.

    Assumptions

    Last verified: 2026-10-23

    Ask about this article

    Get answers grounded in this post. AI-generated — based on this article, and may be imperfect.

    Scaled AI Weekly

    Enjoyed this? Get more like it every Monday.

    Real architecture decisions, LLMOps patterns that survive production, and engineering leadership advice — from 12+ years of building at enterprise scale. Free. No spam. Unsubscribe anytime.

    Get new articles by email

    Free. We email a confirmation link first, and you can unsubscribe from any email.

    Join engineers building production AI systems

    Comments

    Back to all guides