Cloud Platforms for AI: Networking, Identity, Keys, Logs

Quotas and capacity

How to read this: each row is a capacity fact from a provider page, as of 2026-10-23, with its effect on design.

Retry ownership, timeouts and fallbacks are on Reliability and scaling.

The portability layer: a gateway, adapters and what does not port

Figure 5. One gateway, one adapter per provider

How to read this: the Orchestrator talks to one internal contract and each adapter hides one provider's details, so a migration changes an adapter and a configuration, not the Orchestrator.

How to read this: each row is something an adapter or a configuration value must absorb. Sources: AWS API keys, AWS logging, Azure, Google.

Retention and logging defaults also differ, as the sections above show. The pages used here do not compare tool schemas or caching across the three, so treat those as untested until your evaluation set says otherwise.

Same job, three names

How to read this: read across a row for the same job, and treat each cell as a product name from the pages read, not as a claim that the features are equivalent.

Case files

• Data Residency for a Public Agency, Region Pinning. Problem: a public-sector assistant must keep processing in one region and prove it. Pattern: region pinning, in-region logs and audit evidence. Composite.
• Provider Migration With Minimal Change. Problem: a retailer moves a feature to another model provider. Pattern: gateway adapter, versioned configuration, an evaluation set and a canary. Composite.
• Private Networking for a Managed Model. Problem: a clinic network calls a managed model without a public path. Pattern: private endpoint, private DNS, identity, egress deny and logs. Composite.

Under the hood: Private networking paths

Intuition

A private endpoint puts an address for the service inside your network, and private DNS makes the service's normal name resolve to that address from inside. The request then never needs a route to the public internet.

Mechanism

Figure 6. One request over a private endpoint

How to read this: time runs downward, steps 1 and 2 are the DNS answer that makes the path private, and the alt block is the case the egress rule exists for.

1. You create the endpoint. AWS says it creates "an endpoint network interface in each subnet that you enable"; Azure says the private endpoint "is assigned an IP address from the IP address range of your virtual network".
2. DNS resolves the name to that address. AWS: with private DNS enabled, calls to the default Regional name "automatically route through the endpoint", and with it off you must pass the endpoint URL. Azure: the record becomes an alias in a privatelink subdomain, with a private DNS zone holding the A records.
3. Resolving from outside the network still gives the public address. Azure: "When you resolve the endpoint URL from outside the virtual network with the private endpoint, it resolves to the public endpoint".
4. The client keeps the same name and authorization. Azure warns that clients must use the resource's custom subdomain and not the internal *.privatelink.openai.azure.com name.

How to read this: each enabled subnet gets its own interface for each endpoint service, so a design with more subnets or services creates more addresses to plan.

Assumptions: the subnet and service counts are invented. The one-per-subnet rule is from the AWS page, as of 2026-10-23.

Walk one request through the path, with and without the fix.

What this means for architects

Under the hood: Workload identity and short-lived credentials

Intuition

The workload proves who it is to an identity service, which hands back a credential that expires soon. The model service trusts that credential, so no long-lived secret has to sit in your code.

Mechanism

Figure 7. Token exchange, then the call

How to read this: steps 1 and 2 are the exchange, step 3 is the call, and the note at the end is why a leaked token stops working on its own.

1. The workload shows what the platform knows about it. On AWS compute, "AWS delivers the temporary credentials of an IAM role to that compute resource"; outside AWS, IAM Roles Anywhere (a certificate) or STS (a web token or SAML assertion) can issue them.
2. Azure: Azure "assigns it to an Azure resource" and "rotates its credentials for you", and the client requests "a short-lived access token". Google: Workload Identity Federation follows "the OAuth 2.0 token exchange specification": you "provide a credential from your IdP to the Security Token Service", which returns a federated token.
3. The call carries the token, and the service checks the permission granted to that identity: for example Azure's "Cognitive Services OpenAI User" role.
4. The audit log records the principal, not a shared key.

How to read this: if a credential leaks at a random moment in its life and nobody revokes it, it stays useful for half its lifetime, so a shorter life shrinks the window in proportion.

Assumptions: the 90-day rotation is invented. The 12-hour figure is AWS's documented maximum for short-term Bedrock API keys, as of 2026-10-23. The model assumes a uniform leak time and no detection.

What this means for architects

Under the hood: Customer-managed keys and envelope encryption

Intuition

Data is encrypted with a data key, and the data key is itself encrypted by a key you control. Taking away the key you control makes the stored data unreadable without rewriting the data.

Mechanism

Figure 8. Envelope encryption, and what revoking does

How to read this: steps 1 to 3 create the envelope, step 4 is the read, and the alt block is what revocation changes.

1. AWS KMS defines envelope encryption as "the practice of encrypting plaintext data with a data key, and then encrypting the data key under another key". KMS keys "never leave" the KMS hardware security modules "unencrypted", so using one means calling KMS.
2. The wrapped data key is stored beside the data. AWS notes you can "safely store the encrypted data key alongside the encrypted data".
3. Rotating the key does not rewrite the data. Azure: "Rotating the key doesn't trigger re-encryption of data in the resource."
4. Revoking changes what can be read. Azure: revoking an active key "will prevent downloading of training data and results files, fine-tuning new models, and deploying fine-tuned models", while "previously deployed fine-tuned models will continue to operate". Google, for serving-profile keys (Gemini Live and Interactions APIs): requests that need protected data "fail with a FAILED_PRECONDITION error that indicates the key is unavailable".

How to read this: re-wrapping the small keys is far less work than re-encrypting the data, which is the benefit AWS lists for envelope encryption.

Assumptions: 1,000 artifacts and 50 MB are invented. 32 bytes is a 256-bit key; the table ignores wrapping overhead. Decimal units.

What this means for architects

Under the hood: Logging and audit trails in a managed AI service

Intuition

Two questions get different logs. Who did what is a small, durable audit record. What was said is the prompt and answer, and that is personal data the moment you store it.

Mechanism

Figure 9. Three layers of logs and where each one goes

How to read this: the first two layers go straight to the store, and the content layer passes through redaction first. Redaction before storage is the design from the security page.

1. Management events are recorded without setup. AWS: CloudTrail "is enabled on your AWS account when you create the account", and inference operations such as InvokeModel and Converse are management events. Google: Admin Activity logs cannot be disabled.
2. Call metadata can need setup. Google: Data Access logs are off until you "explicitly enable them". Azure: resource logs "aren't collected and stored until you create a diagnostic setting".
3. Content logging is a separate switch. AWS: model invocation logging "is disabled by default", logs "are stored until the logging configuration is deleted", only the same account and Region are supported as destinations. Calls through the bedrock-mantle endpoint "are not currently captured".
4. Google's request-response logging (Preview) writes full_request and full_response to a BigQuery table, with a sampling fraction between 0 and 1. Its VPC Service Controls page says request and response logging "isn't available with VPC Service Controls".

How to read this: content logs are 16 times larger here, and each of those bytes is also personal data to protect.

Assumptions: the record sizes and request count are invented, in decimal units. 100,000 x 0.5 KB = 50,000 KB = 50 MB; 50 MB x 365 = 18,250 MB = 18.25 GB; 100,000 x 8 KB = 800 MB; 800 MB x 365 = 292,000 MB = 292 GB.

What this means for architects

Decision framework

Figure 10. Which platform control for which exposure

How to read this: go top to bottom and apply each control you reach, because the fixes stack and the order puts credentials and exposure first.

Comparison of the main options

Failure scenarios to design for

Worked example

Assumptions:

• A service makes 40,000 requests a day and must keep processing in one region.
• AWS-style geographic profile with 3 destination Regions, as in the AWS IAM example.
• Content logs are off, so the audit check relies on the provider audit log.

How to read this: a single-Region design allows fewer Regions, and the audit job checks every event, not a sample.

The residency claim is a count, not an opinion: 14,600,000 events a year, each checked.

A pre-deployment guard for a model endpoint

The sketch rejects a configuration with a stored key, a public path, a region outside the allowed set or unredacted content logging. Field names are invented: map them to your platform.

ALLOWED_REGIONS = {"region-a"}


def check_endpoint(cfg: dict) -> list:
    """Return a list of problems with a model-endpoint configuration."""
    problems = []
    if cfg.get("auth") != "workload_identity":
        problems.append("use workload identity, not a stored key")
    if cfg.get("static_keys"):
        problems.append("remove stored keys from the deployment")
    if not cfg.get("private_endpoint"):
        problems.append("add a private endpoint")
    if cfg.get("public_access", True):
        problems.append("disable or deny the public path")
    if cfg.get("internet_egress", True):
        problems.append("remove the internet route from the workload subnet")
    if cfg.get("routing") not in {"single_region"}:
        problems.append("routing must be single_region for this rule")
    if cfg.get("region") not in ALLOWED_REGIONS:
        problems.append("region is not in the allowed set")
    if cfg.get("content_logging") and not cfg.get("redaction"):
        problems.append("content logging needs redaction before storage")
    if cfg.get("content_logging") and cfg.get("log_region") != cfg.get("region"):
        problems.append("log store must be in the same region")
    return problems

Run it in CI and fail the build on any problem. It checks configuration, not behaviour, so keep the audit-log check as the runtime proof.

Common mistakes

• Treating a private endpoint as the whole fix. Close the public path too.
• Keeping a long-lived key "just for CI". Federate the CI identity.
• Saying "encrypted with our key" about everything. Name the stored artifacts the key covers.
• Turning on content logging without a privacy plan. Redact and set retention first.
• Assuming geographic routing equals one region. Check the audit log.
• Assuming a migration is a URL change. Authorization, addressing, model names and logging defaults differ.

Try it

Time: 30 to 40 minutes. Pick one cloud and one model feature you know, or invent a small one.

1. Draw Figure 1 for your feature with your component names.
2. For each of the six parts write the setting that controls it, from the provider's page.
3. Write the routing level your rule needs and the audit field that proves it.
4. Write one sentence on what a customer-managed key does and does not protect for your feature.

Done when each of the six parts has a named setting and source link, your region rule has audit evidence, and no stored key remains.

Key takeaways

• Six parts configure a managed model service: identity, private path, keys, logs, region and capacity.
• Use workload identity: a stored key is exposed for its lifetime, a token for its short life.
• A private endpoint is a control only when the public path is closed.
• A customer-managed key governs stored artifacts, not prompts in transit.
• Audit logs answer who acted. Content logs answer what was said, and are opt-in personal data.
• Pin inference to the narrowest level the rule allows, and verify the processing region in the audit log.
• Put one adapter per provider behind a gateway, and treat what you did not test as not portable.

Self-check

Further reading

• AWS, Amazon Bedrock: interface VPC endpoints, cross-Region inference, model invocation logging and data encryption.
• Microsoft Learn, virtual networks for Foundry Tools, Microsoft Entra ID authentication, encryption at rest and deployment types.
• Google Cloud, networking access, VPC Service Controls, CMEK, data residency and Workload Identity Federation.

Last verified: 2026-10-23

Next

Page 15 is the capstone: two full designs that apply the whole course, Capstone: Two Full AI System Designs.