Ship the AI Service into a Customer's Environment

What you'll learn: how to hand a customer's platform team a service they can run: an image, Kubernetes manifests checked without a cluster, a Terraform sketch, secrets by reference, a CI gate and a rollback. No Docker daemon, cluster or Terraform was available here, and each step says what was and was not run. Facts come from the tools' documentation, as of 2026-11-04.

At a glance

Figure 1. What crosses the line into the customer's environment

How to read this: everything left of the zone is yours and carries no secrets; secrets arrive from the customer's store when the container starts.

• Hand over four things: an image, a settings table, a health endpoint and a runbook.
• CI builds an image only after the tests, including the evaluation tests, pass.
• Plan the rollback before the first release.

Scenario

Composite scenario, not a real company. Lumen Industrial Supply's platform team will run the product discovery copilot in its own Kubernetes cluster, which the FDE cannot reach. They ask what to run, what to configure, how to tell it is alive and what to do when a release goes wrong. The FDE answers with files they can review and a one-page runbook.

What you will be able to do

• Build and explain a non-root image that holds no secret.
• Write Kubernetes manifests with probes, requests and a Secret reference, and check them offline.
• Read a Terraform sketch and the CI job that validates it.
• Describe a rolling update and a rollback.

Prerequisites

• Errors, configuration and logging and Build a golden set.
• Cloud platforms for AI: where the service runs, private networking and keys.

Core ideas

The handover pack

The capstone's defaults suit a laptop: ALLOW_DEV_TOKENS is true and DEV_SECRET has a built-in value. The image and ConfigMap turn tokens off, and the Deployment takes DEV_SECRET from a Secret. The health endpoint does not call the model provider, so a provider outage cannot make the platform restart every copy.

The container image

Real file, deploy/Dockerfile, lines 3 to 26 (not built here: no Docker daemon):

FROM python:3.12-slim

WORKDIR /srv/copilot

# Dependencies first: this layer is reused until requirements.txt changes.
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# The service reads app/ and data/ only. No .env file, no key, no token secret.
COPY app/ app/
COPY data/ data/

# Unprivileged user: a fixed numeric id, the same one runAsUser sets in the Kubernetes manifest.
USER 10001

# Safe defaults only. DEV_SECRET and ANTHROPIC_API_KEY arrive when the container starts.
ENV LLM_PROVIDER=mock ALLOW_DEV_TOKENS=false PYTHONUNBUFFERED=1
EXPOSE 8000

# Exit status 0 is healthy, 1 is unhealthy. The route does not call the model provider.
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
  CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/api/v1/health', timeout=2)"]

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

deploy/Dockerfile.dockerignore keeps .venv, .env and tests out of the build context (context). To build and run (not run here):

cd capstone/fde-product-discovery-copilot
docker build -f deploy/Dockerfile -t registry.example/lumen/copilot:0.1.0 .
export DEV_SECRET="<a long random value from your secret store>"
docker run --rm -p 8000:8000 --env DEV_SECRET registry.example/lumen/copilot:0.1.0

--env VAR with no value passes the variable from your shell (docker run), so the secret is in no command or file.

The Docker daemon was not running, so docker build and docker run were not run. Instead the image's contents (app/, data/) and start command ran from a temporary folder with the environment the image sets, on Python 3.14 (the .venv), not the image's 3.12:

health probe exit=0
{"status":"ok","provider":"mock"}
dev token status=404
search status=200
closed port probe exit=1

Lines: probe exit status, health body, token endpoint (off), a catalogue search, the probe against a closed port. It does not show that the image builds or that user 10001 can read data/.

Kubernetes manifests, checked without a cluster

deploy/k8s/ holds a ConfigMap, a Deployment and a Service. There is no Secret file: it would hold the value. The Deployment points at a Secret that the customer's store creates.

Real file, deploy/k8s/deployment.yaml, lines 36 to 70:

            - name: DEV_SECRET
              valueFrom:
                secretKeyRef:
                  name: copilot-secrets
                  key: dev-secret
            - name: ANTHROPIC_API_KEY
              valueFrom:
                secretKeyRef:
                  name: copilot-secrets
                  key: anthropic-api-key
                  optional: true
          readinessProbe:
            httpGet:
              path: /api/v1/health
              port: http
            periodSeconds: 10
            timeoutSeconds: 2
            failureThreshold: 3
          livenessProbe:
            httpGet:
              path: /api/v1/health
              port: http
            initialDelaySeconds: 20
            periodSeconds: 20
            timeoutSeconds: 2
            failureThreshold: 3
          resources:
            requests:
              cpu: 100m
              memory: 256Mi
            limits:
              memory: 512Mi
          securityContext:
            runAsNonRoot: true
            runAsUser: 10001

A ConfigMap gives no secrecy or encryption (docs), so it holds non-secret settings only.

Check it with no cluster. The usual client-side check is kubectl apply --dry-run=client --validate=false -f deploy/k8s/. With a kubeconfig pointing at a closed port, it still asked the API server which kinds exist and failed (first error only):

unable to recognize "deploy\\k8s\\configmap.yaml": Get "http://127.0.0.1:9/api?timeout=32s": dial tcp 127.0.0.1:9: connectex: No connection could be made because the target machine actively refused it.

kubectl annotate --local parses files without contacting a server (reference) and prints each object as JSON. deploy/check_manifests.py reads that JSON and checks what a schema does not: selector equals Pod labels, a version tag, both probes, requests, non-root, no literal secret, no Secret object.

kubectl annotate --local -f deploy/k8s/ checked-by=ci -o json | python deploy/check_manifests.py

Real output on the committed files, then on a copy with :latest and one Pod label changed:

manifest check: ok

PROBLEM: copilot: selector does not match the pod template labels
PROBLEM: copilot/copilot: image needs a version tag, not latest
PROBLEM: copilot: no Deployment matches the Service selector
manifest check: failed

Limit, found by trying: a misspelled field (containerPortt) still passed. Left to you: kubectl apply --dry-run=server against the customer's cluster.

Terraform: a sketch for the platform team

deploy/terraform/main.tf expresses the namespace and the Deployment with the Kubernetes provider, for teams that manage clusters as code. Manage one object with manifests or Terraform, never both.

Sketch, to be validated by CI, not run here (lines 64 to 88 of deploy/terraform/main.tf):

          port {
            name           = "http"
            container_port = 8000
          }

          # A reference to a Secret that the customer's secret store creates; no value is written here.
          env {
            name = "DEV_SECRET"
            value_from {
              secret_key_ref {
                name = "copilot-secrets"
                key  = "dev-secret"
              }
            }
          }

          readiness_probe {
            http_get {
              path = "/api/v1/health"
              port = "http"
            }
            period_seconds    = 10
            timeout_seconds   = 2
            failure_threshold = 3
          }

Block names follow the provider's Deployment resource. The provider block holds no credentials; its page says to supply a kubeconfig path, for example KUBE_CONFIG_PATH.