Traffic Management in Apigee: Quotas, Spike Arrest, Caching

Product-driven limits. Rather than hard-coding counts, the policy can read its limits from the API product the caller's key or token belongs to (UseQuotaConfigInAPIProduct). That is how tiered plans work: change the product, and every proxy using that product follows. A VerifyAPIKey or OAuthV2 VerifyAccessToken step must run before the Quota so Apigee knows which product applies. By default each operation in a product has its own counter; the product's quotaCounterScope (PRODUCT or PROXY) controls whether operations without their own quota share a counter. The products lesson covers creating products.

<Quota name="Q-PlanFromProduct" continueOnError="false" enabled="true">
  <DisplayName>Q-PlanFromProduct</DisplayName>
  <Identifier ref="verifyapikey.VK-VerifyAPIKey.client_id"/>
  <Distributed>true</Distributed>
  <Synchronous>false</Synchronous>
  <AsynchronousConfiguration>
    <SyncIntervalInSeconds>10</SyncIntervalInSeconds>
  </AsynchronousConfiguration>
  <UseQuotaConfigInAPIProduct stepName="VK-VerifyAPIKey">
    <DefaultConfig>
      <Allow>1000</Allow>
      <Interval>1</Interval>
      <TimeUnit>day</TimeUnit>
    </DefaultConfig>
  </UseQuotaConfigInAPIProduct>
</Quota>

When the product defines its own quota, the product wins and the DefaultConfig is a fallback; Allow, Interval and TimeUnit outside this element are ignored. A Quota policy attached in several flows shares one counter by policy name, which makes the quota run out faster than expected: give each flow its own policy if you want separate counters.

The policy sets flow variables such as ratelimit.{policy_name}.allowed.count, .used.count and .expiry.time. Return them as headers so clients can pace themselves:

<AssignMessage name="AM-QuotaHeaders" continueOnError="false" enabled="true">
  <DisplayName>AM-QuotaHeaders</DisplayName>
  <AssignTo createNew="false" transport="http" type="response"/>
  <Set>
    <Headers>
      <Header name="X-Quota-Limit">{ratelimit.Q-PlanFromProduct.allowed.count}</Header>
      <Header name="X-Quota-Used">{ratelimit.Q-PlanFromProduct.used.count}</Header>
      <Header name="X-Quota-Reset-UTC">{ratelimit.Q-PlanFromProduct.expiry.time}</Header>
    </Headers>
  </Set>
  <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</AssignMessage>

Quota can also count after the call. With SharedName, EnforceOnly in the request flow and CountOnly in the response flow (with a MessageWeight taken from the response), you can enforce allowances based on something known only after the backend answers, for example to budget by weight rather than by call count. (Apigee also has a dedicated LLMTokenQuota policy for token budgets; check its page.)

Friendly rejections

Both policies raise faults named SpikeArrestViolation and QuotaViolation. Match on the fault name in a fault rule (inside the ProxyEndpoint) and return a clear 429 with a Retry-After hint, which also makes the status consistent whatever the default is:

<FaultRules>
  <FaultRule name="RateLimited">
    <Step>
      <Name>AM-RateLimitResponse</Name>
    </Step>
    <Condition>fault.name Matches "SpikeArrestViolation" OR fault.name Matches "QuotaViolation"</Condition>
  </FaultRule>
</FaultRules>

<AssignMessage name="AM-RateLimitResponse" continueOnError="false" enabled="true">
  <DisplayName>AM-RateLimitResponse</DisplayName>
  <Set>
    <Headers>
      <Header name="Retry-After">1</Header>
    </Headers>
    <Payload contentType="application/json" variablePrefix="@" variableSuffix="#">{"error":{"code":"RATE_LIMITED","message":"Too many requests. Slow down and retry.","correlationId":"@messageid#"}}</Payload>
    <StatusCode>429</StatusCode>
    <ReasonPhrase>Too Many Requests</ReasonPhrase>
  </Set>
  <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
  <AssignTo createNew="false" transport="http" type="response"/>
</AssignMessage>

A constant Retry-After of one second suits SpikeArrest; for quota, derive it from ratelimit.{policy_name}.expiry.time instead. The fault handling lesson goes deeper on fault rules.

Which one do I use?

How to read it: if the answer to "need accurate counts?" is yes, the limit is a business rule, so it belongs in Quota rather than SpikeArrest.

Caching in Apigee

A cache turns repeated, identical work into a memory lookup. Apigee has two groups of cache policies.

ResponseCache caches whole backend responses. Attach it once in the request flow (the lookup) and once in the response flow (the population); the docs limit it to one attachment in each, and it cannot be used in a shared flow. Best practice from the docs is to do the lookup in the ProxyEndpoint request PreFlow with as little logic as possible before it, and to populate in the response PostFlow as late as possible, after mediation, so you cache the final shape.

PopulateCache, LookupCache and InvalidateCache are the general-purpose, short-term cache. You choose what to store (Source), where to read it (AssignTo) and when to invalidate. Use them for computed values, such as the result of a lookup you would otherwise repeat with a ServiceCallout.

How to read it: a hit skips everything after the lookup; only misses cost a backend call. The condition on population keeps errors and writes out of the cache.

Cache keys

The key decides what counts as "the same request". It is built from Prefix (or the Scope default) plus one or more KeyFragment values. By default Scope is Exclusive, which scopes the key to the organisation, environment, proxy and endpoint, so keys from different proxies cannot collide. Choose fragments deliberately:

• Include everything that changes the response. The path and query string (request.uri), a tenant identifier, the language header if responses vary by language. Set UseAcceptHeader to true if clients negotiate formats; it adds the Accept, Accept-Encoding, Accept-Language and Accept-Charset request headers to the key.
• Exclude what does not. The docs advise against including the API key in the cache key unless required, since that stores the same response once per app.
• Never share per-user data under a shared key. If the response depends on the caller, the caller's identity must be in the key, or the response must not be cached.
• Keys are limited to 2 KB and cached values to 256 KB.

TTL choices

ExpirySettings accepts TimeoutInSeconds (which wins over the date and time-of-day alternatives), and UseResponseCacheHeaders makes Apigee also consider Cache-Control (s-maxage, max-age) and Expires, using the lowest value. Pick a TTL from the data's tolerance for staleness, not from what is convenient: five minutes for a product catalogue is usually fine; five minutes for an account balance is not. The docs recommend setting appropriate expirations to avoid dirty reads. Responses with error status codes are excluded by default (ExcludeErrorResponse is true), and for write operations you should not populate the cache at all.

<ResponseCache name="RC-Catalogue" continueOnError="false" enabled="true">
  <DisplayName>RC-Catalogue</DisplayName>
  <CacheKey>
    <Prefix>catalogue</Prefix>
    <KeyFragment ref="request.uri"/>
    <KeyFragment ref="request.header.accept-language"/>
  </CacheKey>
  <ExpirySettings>
    <TimeoutInSeconds>300</TimeoutInSeconds>
  </ExpirySettings>
  <SkipCachePopulation>response.status.code != 200 or request.verb != "GET"</SkipCachePopulation>
  <UseResponseCacheHeaders>true</UseResponseCacheHeaders>
</ResponseCache>

Attach it twice, restricted to reads:

<ProxyEndpoint name="default">
  <PreFlow name="PreFlow">
    <Request>
      <Step><Name>SA-ProtectBackend</Name></Step>
      <Step>
        <Name>RC-Catalogue</Name>
        <Condition>request.verb = "GET"</Condition>
      </Step>
    </Request>
    <Response/>
  </PreFlow>
  <PostFlow name="PostFlow">
    <Request/>
    <Response>
      <Step>
        <Name>RC-Catalogue</Name>
        <Condition>request.verb = "GET"</Condition>
      </Step>
    </Response>
  </PostFlow>
  <HTTPProxyConnection>
    <BasePath>/v1/catalogue</BasePath>
  </HTTPProxyConnection>
  <RouteRule name="default">
    <TargetEndpoint>default</TargetEndpoint>
  </RouteRule>
</ProxyEndpoint>

A few behaviours to remember. If the lookup comes before authentication, cached responses go to unauthenticated callers, so place it after your authentication steps unless the data is public. The docs also warn that a CORS policy's response must not be cached. A SkipCacheLookup condition (for example a bypass header) lets clients force a refresh; use it only where callers are trusted, because it lets them bypass the cache protection.

How the cache works underneath

When a caching policy runs, entries live in two levels. L1 is a short-lived (one second) in-memory cache on each Message Processor; after one second an unused entry is kept only in L2, a persistent store shared by all processors in the environment. Entries are encrypted at rest in L2. Practical consequences: invalidating an entry removes it from the L2 store and from the processor that handled the request, but other processors can serve their L1 copy for up to another second; and values above the size limit are not cached usefully (the docs list storing data over 256 KB as an anti-pattern).

Target servers, load balancing and timeouts

A target server is a named backend (host, port, protocol, enabled flag, optional TLS settings) defined per environment. Referencing it by name in the target endpoint decouples the proxy from concrete URLs, so the same proxy bundle can point at different backends in test and production. Create one with the management API (placeholders for organisation, environment and token):

curl -X POST "https://apigee.googleapis.com/v1/organizations/$ORG/environments/$ENV/targetservers" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "inventory-a",
    "host": "inventory-a.internal.example.com",
    "protocol": "http",
    "port": 443,
    "isEnabled": true,
    "sSLInfo": { "enabled": true, "enforce": true }
  }'

Repeat for inventory-b and a disaster-recovery server inventory-dr. (The docs create HTTPS targets by setting protocol http with port 443 and sSLInfo.enabled true.) Then reference them in a load balancer. The docs describe three algorithms: RoundRobin (default; alternates in listed order), Weighted (traffic in proportion to each server's Weight) and LeastConnections (the server with the fewest open connections).

<TargetEndpoint name="default">
  <HTTPTargetConnection>
    <Properties>
      <Property name="connect.timeout.millis">2000</Property>
      <Property name="io.timeout.millis">10000</Property>
    </Properties>
    <LoadBalancer>
      <Algorithm>RoundRobin</Algorithm>
      <Server name="inventory-a"/>
      <Server name="inventory-b"/>
      <Server name="inventory-dr">
        <IsFallback>true</IsFallback>
      </Server>
      <MaxFailures>3</MaxFailures>
      <ServerUnhealthyResponse>
        <ResponseCode>502</ResponseCode>
        <ResponseCode>503</ResponseCode>
      </ServerUnhealthyResponse>
    </LoadBalancer>
    <Path>/inventory</Path>
    <HealthMonitor>
      <IsEnabled>true</IsEnabled>
      <IntervalInSec>5</IntervalInSec>
      <HTTPMonitor>
        <Request>
          <ConnectTimeoutInSec>5</ConnectTimeoutInSec>
          <SocketReadTimeoutInSec>5</SocketReadTimeoutInSec>
          <Verb>GET</Verb>
          <Path>/healthz</Path>
        </Request>
        <SuccessResponse>
          <ResponseCode>200</ResponseCode>
        </SuccessResponse>
      </HTTPMonitor>
    </HealthMonitor>
  </HTTPTargetConnection>
</TargetEndpoint>

How to read it: the load balancer alternates between the two normal servers. A server that fails MaxFailures times leaves rotation; the health monitor brings it back when it answers correctly. The fallback server receives traffic only when every other server is out.

How the failure handling works, as documented:

• MaxFailures is the number of failed requests that removes a server from rotation. It defaults to 0, meaning Apigee never removes anything. A failure means Apigee received no response (connection error or timeout). A 500 from the backend is a response and resets the counter, which is why ServerUnhealthyResponse exists: list status codes that should count as failures.
• Health monitoring polls each server over TCP or HTTP and needs MaxFailures greater than 0. It returns a recovered server to rotation automatically. It cannot be created in the Apigee UI (you deploy a proxy bundle containing it) and is configured per proxy.
• Without a monitor, a server removed by MaxFailures is rechecked by Apigee every five minutes.
• Retries. The docs describe a RetryEnabled setting, true by default, with at least two target servers required. Retries trigger on I/O errors and timeouts, and on responses matching ServerUnhealthyResponse, not on arbitrary 4xx or 5xx codes. Retried requests must be safe to repeat: do not list a status code that can follow a half-completed write.
• One server with MaxFailures above zero is listed as an anti-pattern: a single target that gets marked down leaves nothing to fail over to.

Timeouts

Three settings interact; the property values must be literals, not variables.

Apigee computes the effective io.timeout.millis as the lesser of the configured value and the time remaining under api.timeout. Set timeouts so that the caller's own timeout is longer than the gateway's, otherwise clients give up before Apigee can return a clean error. Set io.timeout.millis from the backend's real latency profile (for example slightly above its p99), not from the default. When a load balancer is configured, Apigee automatically retries a failed connection up to three times, so the total connection wait can exceed connect.timeout.millis; standard single-URL targets do not retry automatically. Keep-alive is on by default (keepalive.timeout.millis 60000); the docs list disabling persistent connections as an anti-pattern because every request then pays for a new connection.

Payload sizes, headers and streaming

By default Apigee buffers request and response bodies in memory so that policies can work on them. The docs give a default limit of 10 MB for payloads in request and response flows, raisable up to 30 MB through the request.payload.parse.limit and response.payload.parse.limit properties; a larger payload fails the request, so look up the exact fault in the runtime error catalog. The Limits page lists the maximum sizes for headers, URLs and for buffered and streamed messages, and notes which are currently enforced. These numbers change, so check the current limits for your organisation type before you design around any of them.

Two cache limits, 2 KB per key and 256 KB per value, are stated in the policy pages and are safe design inputs.

Streaming. Setting request.streaming.enabled or response.streaming.enabled to true on the endpoint stops Apigee buffering the body; it passes through as-is. The price is that policies which operate on the payload (ExtractVariables on the body, JSONToXML and similar) are bypassed. Worse, if a policy does touch the payload on a streaming proxy, Apigee falls back to buffering, which the docs list as an anti-pattern because of the extra latency and memory use. Stream large downloads and uploads that you do not need to inspect; isolate proxies that routinely carry large payloads, as the best practices suggest.

How Apigee X scales

Apigee X runs the gateway as managed infrastructure; you do not size or operate the runtime nodes. What you can see in the documentation is the shape of the offer: under Pay-as-you-go pricing, the environment type sets throughput (Base up to 50 queries per second, Intermediate up to 100, Comprehensive autoscaled), and you can set a minimum number of gateway nodes per environment so extra capacity exists before a spike or a regional failover. Counters and caches that need to be shared (Quota counters, the L2 cache) live in shared stores rather than on a single node, which is why distributed Quota and cached values work across nodes. Subscription-based organisations have their own entitlements. Read the current environment-type and pricing pages before capacity planning, because these figures and names evolve.

Performance anti-patterns

The Apigee docs keep a catalogue of anti-patterns; these are the ones that matter most for traffic and speed.

• Heavy logic in script policies. The best-practice guide says to use built-in policies first (AssignMessage, ExtractVariables) because they are hardened and optimised, avoid putting all proxy logic in JavaScript, Java or Python, and use Python only when unavoidable because it is interpreted at runtime. JavaScript is for logic that is genuinely awkward in policies.
• HTTP calls from inside script policies. Make outbound calls with the ServiceCallout policy, which handles connections properly.
• Chains of ServiceCallouts. Each callout is another network round trip inside the proxy's time budget (api.timeout covers the whole execution), so a chain of three can triple the latency. If you need several backend calls, ask whether the backend should aggregate them, and cache the results that do not change. When proxies must call each other, the docs recommend local proxy chaining, which avoids extra network hops.
• Large payload transformations. Repeatedly parsing and copying big bodies consumes CPU and memory. Transform once, stream when you need no inspection, and keep large-payload proxies in their own environment.
• Caching mistakes. Caching errors, caching writes, over-large entries, and keys that include the API key.
• Quota mistakes. A non-distributed quota under multiple processors, or one Quota policy reused across flows.
• Ignoring the backend. A slow or undersized backend raises latency for every client; caching and rate limits help, but the fix is the backend.

How to implement: load-test plan

Prove the limits before production does. Use a test environment with the same policies and a stand-in or staging backend, and talk to your platform team before generating load. This k6 script is an outline you can adapt: it offers a constant arrival rate above your SpikeArrest limit and records how many requests were served versus throttled.

import http from 'k6/http';
import { check } from 'k6';
import { Counter } from 'k6/metrics';

const throttled = new Counter('throttled_429');

export const options = {
  scenarios: {
    burst: {
      executor: 'constant-arrival-rate',
      rate: 100,            // offered requests per second, above the 50ps limit
      timeUnit: '1s',
      duration: '60s',
      preAllocatedVUs: 50,
      maxVUs: 200,
    },
  },
};

export default function () {
  const res = http.get(`${__ENV.BASE_URL}/v1/catalogue/items?page=1`, {
    headers: { 'x-apikey': __ENV.API_KEY },
  });
  if (res.status === 429) throttled.add(1);
  check(res, { 'served or cleanly throttled': (r) => r.status === 200 || r.status === 429 });
}

Run it with k6 run -e BASE_URL=https://api-test.example.com -e API_KEY=$KEY script.js. The experiment:

1. Baseline: disable the policies (or use a high limit) and record backend latency and error rate at 100 requests per second.
2. Spike arrest: enable SA-ProtectBackend at 50ps and confirm the backend sees about that rate while the excess returns 429 quickly.
3. Cache: enable RC-Catalogue and repeat the same URLs. Expect the backend request count to fall to roughly one per key per TTL, and latency to drop for hits.
4. Failover: stop inventory-a mid-run and watch for the first errors, then clean traffic on inventory-b; restore it and confirm it rejoins after the health monitor passes.
5. Quota: run with a tiny daily allowance and confirm the rejection after the limit (429 because the fault rule maps it), with the X-Quota-* headers counting up on the allowed calls.
6. Compare Apigee Analytics, Debug sessions and your backend's own metrics. Do not trust the client-side numbers alone.

Use cases

1. Protecting a fragile backend from a retry storm. When a backend slows, clients time out and retry, multiplying load exactly when capacity is lowest. A SpikeArrest sized to what the backend can sustain turns the storm into fast 429s with Retry-After; clients that back off recover the system. Short connect and io timeouts stop requests piling up waiting on a dying backend, and cached reads take load off it entirely.
2. Tiered plans (free, standard, premium). Create three API products, each with its own quota (for example 1,000 per day, 50,000 per day, 1,000,000 per month; all assumed). One Quota policy using UseQuotaConfigInAPIProduct serves all three, so changing a plan is a product edit, not a redeployment. Add a gentler per-app SpikeArrest for the free tier.
3. Caching a read-heavy catalogue endpoint. A product catalogue changes a few times a day but is read thousands of times a minute. ResponseCache with a key on the full URI and language, a five-minute TTL, and population limited to successful GETs means one backend call per variant per five minutes.
4. Active-active backends with a standby. Two regional servers share load with round robin; a health monitor removes a failing server and returns it when healthy; a fallback server in another region carries traffic only if both are out.

Trade-offs at a glance

In practice: a promotion that doubles the retry load

The situation

An online retailer exposes a catalogue and inventory API through Apigee to its web front end and mobile app. Normal traffic is about 40 requests per second (assumed), and the inventory backend is comfortable up to roughly 60 per second (assumed). A seasonal promotion is expected to triple demand. Last year the backend slowed under load, clients timed out and retried, and the retries pushed it over. The team wants the API to degrade predictably, not collapse.

Starting architecture

How to read it: nothing between the apps and the one server absorbs a spike, avoids repeated reads, or fails over.

What changes

1. Cache the reads. Catalogue and product-detail GETs get RC-Catalogue with a five-minute TTL. Inventory counts, which must be fresher, get a 15-second TTL (assumed), a deliberate trade of a little staleness for load.
2. Protect the backend. SA-ProtectBackend at 50ps (assumed, just below the tested ceiling), first in the PreFlow, with UseEffectiveCount true. The fault rule returns 429 with Retry-After.
3. Plan the allowance. The mobile and partner products get quotas via UseQuotaConfigInAPIProduct, so a runaway client exhausts its own allowance, not everyone's.
4. Add capacity and failover. A second inventory server and a fallback in another zone become target servers; the target endpoint load balances with a health monitor on /healthz and MaxFailures of 3.
5. Tighten timeouts. connect.timeout.millis 2,000 and io.timeout.millis set a little above the measured backend p99 (assumed 3 seconds, so 4,000), with client timeouts longer than the gateway's.
6. Tell the clients. Document the 429 and Retry-After contract and ask client teams to back off with jitter rather than retry immediately.
7. Rehearse. Run the k6 plan at 150 requests per second against staging, including killing one server mid-run.

How to read it: reads never reach the backend, the surge is shed at the gateway, and a failing server is routed around.

Result

During the promotion the backend sees a ceiling of roughly the configured spike rate instead of the full demand, and its latency stays flat. Catalogue pages load quickly from cache; clients that hit the limit see a clear 429 and slow down instead of hammering. When one inventory server is stopped in the rehearsal, a few requests fail at the moment of failure and then traffic shifts to the remaining server. Analytics show the 429 rate, cache hit ratio and per-server errors, which is what the team watches during the event.

Architect's notes

• Order matters. SpikeArrest first, authentication, then Quota, then cache lookup where responses are not public. Cheap rejections before expensive work.
• Size from measurement. Take limits from load-test results, not guesses, and revisit them when the backend changes.
• Cache is a data-sharing decision. Every cache key is an assertion that two callers may receive the same bytes. Review keys like you review access rules.
• Make the shed visible. Alert on rising 429 counts, on cache hit ratio falling, and on servers leaving rotation. A silent rate limit looks like an outage to the client.
• Retries need an owner. Decide whether the client, the gateway or the backend retries. Layered retries multiply load; the load balancer's retry on timeouts and unhealthy responses is one more layer to account for.
• Cost. Some of these policies are classed as Extensible and may carry licensing or utilisation implications depending on your Apigee licence; load balancing and quota synchronisation also add latency. Check the policy type and measure.
• When not to cache. Personalised, rapidly changing or write-affecting responses are poor candidates; the saving rarely beats the risk of serving the wrong data.

Common mistakes

• Using SpikeArrest for billing or exact limits. Its counts are best-effort. Use a distributed, synchronous Quota where accuracy matters.
• Leaving UseEffectiveCount at the default. The docs recommend true for Apigee organisations; smoothing can reject legitimate near-simultaneous requests.
• Non-distributed quotas. With several Message Processors each keeps its own count. Set Distributed to true.
• One Quota policy in several flows. They share one counter by policy name. Use separate policies if you need separate counters.
• Quota before authentication. Without an identity, the counter is shared by everyone, or product limits cannot be resolved.
• API key in the cache key. It stores the same response many times. Key on what changes the response.
• Caching per-user responses under a shared key. Include the identity or do not cache.
• Caching errors and writes. Use SkipCachePopulation for non-200 and non-GET traffic.
• MaxFailures left at 0. Failover never happens. Set it, add a health monitor, and list the unhealthy status codes.
• Retrying non-idempotent calls. Do not mark a status code unhealthy if it can follow a partial write.
• Default timeouts. io.timeout.millis of 55 seconds lets slow requests pile up. Set it from real latency.
• Touching payloads while streaming. It silently re-enables buffering. Keep streaming proxies payload-blind.

Key takeaways

• SpikeArrest protects the backend from bursts; Quota enforces allowances; caching removes repeated work.
• Set UseEffectiveCount to true on SpikeArrest, and use Quota (distributed, synchronous where accuracy matters) for anything counted.
• Put SpikeArrest first and Quota after authentication; read quota limits from API products for tiered plans.
• ResponseCache needs a precise key, a TTL chosen for acceptable staleness, and population restricted to successful GETs; values are limited to 256 KB.
• Load balancing supports RoundRobin, Weighted and LeastConnections; MaxFailures greater than 0 plus a health monitor gives automatic failover, and a fallback server covers total loss.
• Set connect.timeout.millis, io.timeout.millis and api.timeout deliberately, and check the current limits page for payload and header sizes.
• Streaming avoids buffering but bypasses payload policies.
• Prove the configuration with a load test that includes a backend failure.

Practice exercises

Starter (about 30 minutes): Add SA-ProtectBackend at 5ps to a test proxy in front of a mock backend. Send ten requests in quick succession with a shell loop, count the 200s and 429s, then change UseEffectiveCount between true and false and compare.

Intermediate (about 2 hours): Build a catalogue proxy with RC-Catalogue (key on request.uri, 60-second TTL, population only for 200 GETs). Prove hits using Debug and response latency, change the query string to show a different key, and add a bypass condition. Then add a Quota that reads limits from two API products (a 5-per-minute "free" and a 50-per-minute "standard") and verify each app gets its own counter.

Advanced (half a day): Create three target servers, a load-balanced target endpoint with MaxFailures, ServerUnhealthyResponse, a fallback server and an HTTP health monitor. Run the k6 outline above while stopping and restarting a backend, recording error counts, time to removal from rotation and time to recovery. Write a one-page capacity note listing the limits you would set, the timeouts and why, and which alerts you would create.

Related reading

• Securing APIs in Apigee — authentication runs before Quota, and token endpoints need spike arrest.
• API design, products and developer apps in Apigee — where product quotas are defined.
• Building Apigee API proxies — flows, target endpoints and policy order.
• Fault handling, debugging and monitoring in Apigee — shaping 429 responses and watching rejections.
• Environments, revisions and deployment — target servers and caches are defined per environment.
• Capstone: design, build and ship an Orders API — applies spike arrest, quota and caching end to end.
• System design: rate limiting — the algorithms behind these policies.
• System design: load balancing — the general theory of balancing and health checks.
• System design: caching basics — keys, TTLs and invalidation in general.
• System design: resilience patterns — timeouts, retries and backoff.

Official documentation

• Comparing rate-limiting policies
• SpikeArrest policy
• Quota policy
• ResponseCache policy
• PopulateCache policy, LookupCache policy and InvalidateCache policy
• Cache internals
• Load balancing across backend servers
• Endpoint properties reference
• Apigee limits
• Best practices for API proxy design and development
• Antipattern: Configure non-distributed quota
• Antipattern: Accessing the payload when streaming is enabled
• Antipattern: Disable HTTP persistent connections
• Pay-as-you-go environment types
• Expanding Apigee to multiple regions