Design a URL shortener end to end: requirements, estimates, API, base62 ID generation, data model, caching, 301 versus 302, expiry, analytics, abuse prevention and scaling.
What you'll learn: By the end of this lesson you will be able to design a URL shortener from requirements to scaling plan, compare counter, hash and pre-generated ID strategies and choose one, explain the 301 versus 302 decision and its effect on analytics, and discuss expiry, abuse prevention and failure modes the way a strong engineer would in an interview.
A URL shortener is a coat-check desk. You hand over a long, awkward coat (a 300-character link) and receive a small numbered ticket (sho.rt/aZ3kQ9x). Later, anyone who shows the ticket gets sent to the coat. The desk does one job, it must be quick, and it must never hand out the same ticket twice.
That simplicity is why this is the classic first case study. The product fits in a sentence, yet it touches ID generation, caching, database choice, HTTP semantics, asynchronous analytics, abuse and scaling. We follow the arc from the advanced overview: requirements, estimate, API, data model, high-level design, deep dives, bottlenecks, failures, trade-offs, cheat-sheet.
The core idea
The system is a very large key-value map from a short code to a long URL, with two traffic patterns of very different sizes:
Writes (create a short link): rare and can take a few extra milliseconds.
Reads (follow a short link): extremely frequent, must be fast, and are almost all repeats of popular links.
Everything follows from that asymmetry. Optimise the read path ruthlessly (caching, simple lookups, no joins), keep the write path correct (unique codes, validation), and move everything optional (analytics) off the critical path.
Requirements and scope
In scope: create a short URL from a long one (optionally with a custom alias and expiry); redirect when visited; delete or expire links; basic click analytics (counts with country and referrer breakdowns). Out of scope: accounts and billing details, link-in-bio pages, QR rendering, editing the destination.
Non-functional requirements:
Availability: redirects are the product. Target 99.99 percent for redirects (an assumption), a little lower for creation.
Latency: redirect decision in tens of milliseconds at the server (p99 well under 100 ms).
Durability: a created link must not be lost.
Uniqueness: no two long URLs share a code.
Unpredictability (nice to have): codes should not be trivially enumerable.
Clarifying questions worth asking: How long should links live by default? Can the same long URL get several codes? Is analytics real-time or eventually consistent? Do we need to block malicious targets?
Back-of-envelope estimate
All numbers here are illustrative assumptions.
100 million new links per month. A month is about 2.6 million seconds, so roughly 40 writes per second on average.
Read-to-write ratio 100 to 1: 10 billion redirects per month, roughly 3,900 reads per second on average. With a peak factor of 3, plan for about 12,000 reads per second.
Retention five years: 100 million × 12 × 5 = 6 billion links.
Record size about 500 bytes: 6 billion × 500 B is about 3 TB. It fits a large single machine at first, but not comfortably with replication and indexes, so plan for partitioning.
Code length: with base62 (a-z, A-Z, 0-9), 62^6 is about 56.8 billion and 62^7 about 3.5 trillion. Six characters cover 6 billion links, but seven gives enormous headroom and room to mix the codes. We use 7.
Cache: if the hottest 1 percent of links (60 million) draw most traffic and a cached entry is about 200 bytes, the hot set is about 12 GB, which fits in memory across a small cache cluster.
The conclusion matters more than the digits: the data is modest, the read rate is high but very cacheable, and the write rate is tiny. No exotic technology is required.
API design
POST /v1/links
{ "url": "https://example.com/very/long/path?x=1",
"alias": "spring-sale", (optional)
"expires_at": "2027-01-01T00:00:00Z" } (optional)
201 Created
{ "code": "aZ3kQ9x", "short_url": "https://sho.rt/aZ3kQ9x" }
GET /{code}
302 Found
Location: https://example.com/very/long/path?x=1
DELETE /v1/links/{code} 204 No Content
GET /v1/links/{code}/stats 200 OK with counts
Creation accepts an idempotency key so a retried request does not create duplicates (see API design). Return 404 for unknown codes and 410 Gone for expired or deleted ones. Creation is authenticated and rate limited; the redirect endpoint is public.
Data model
One main table, accessed by primary key. Access patterns: look up by code (hot path), create with uniqueness, and list by owner (cold path, via a secondary index). There are no joins and no multi-row transactions, the signature of a key-value workload.
erDiagram
USERS ||--o{ LINKS : owns
LINKS ||--o{ CLICK_HOURLY : "aggregated into"
USERS {
string owner_id PK
string plan
}
LINKS {
string code PK
string long_url
datetime created_at
datetime expires_at
string owner_id FK
string status
}
CLICK_HOURLY {
string code FK
datetime hour_start
string country
int clicks
}
RESERVED_WORDS {
string word PK
}
expires_at is nullable and status is active, disabled for abuse, or deleted. CLICK_HOURLY lives in a separate analytics store, not beside LINKS, so reporting load never touches the redirect path. RESERVED_WORDS keeps custom aliases from claiming paths such as admin or api.
Store choice. A good candidate states both reasonable answers:
A sharded relational database such as PostgreSQL: familiar, transactional uniqueness, easy secondary indexes. Shard by a hash of code once one primary can no longer hold the data or write volume.
A distributed key-value or wide-column store (Cassandra-style or DynamoDB-style): built-in partitioning and replication, a perfect fit for single-key lookups. Uniqueness for custom aliases needs a conditional write or lightweight transaction.
At tens of writes per second, throughput is not the problem; data size, availability and operational simplicity decide. Pick what your team can run well.
High-level design
The redirect service and create service are stateless and scale separately, because their load differs by two orders of magnitude. The redirect path: look in the cache; on a miss, read the store and populate the cache; return the redirect. Click events are published asynchronously and never block the response.
sequenceDiagram
participant U as User
participant R as Redirect service
participant K as Cache
participant D as Link store
participant Q as Event queue
U->>R: GET /aZ3kQ9x
R->>K: lookup aZ3kQ9x
alt cache hit
K-->>R: long URL
else cache miss
R->>D: read by code
D-->>R: long URL
R->>K: store with TTL
end
R->>Q: click event without waiting
R-->>U: 302 with Location header
Reference topology
This is the deployment the rest of the lesson assumes: a primary region that takes writes, and a second region that serves reads from a replica and its own cache.
Writes go only to region A; region B tolerates replica lag by falling back to the primary once on a miss. The click topic decouples analytics from the redirect path entirely.
Deep dive 1: generating the short code
The code must be unique, short, and ideally hard to guess. Three strategies, each with an honest cost.
Strategy A: counter plus base62. Keep a global counter, increment per link, and encode in base62. No collisions are possible and codes are as short as they can be. Costs: one counter is a bottleneck and single point of failure, and consecutive codes reveal creation order and volume. Fix the bottleneck with range allocation: a small coordination service (a database row, or etcd or ZooKeeper) hands each create-service instance a block, say 100,000 numbers, served from memory. If an instance crashes its unused range is skipped, which is fine given the size of the space. Fix the predictability with a reversible scrambling step (a bijective permutation such as a small Feistel network with cycle-walking so the output stays below 62^7) before encoding, so adjacent numbers produce unrelated-looking codes while uniqueness is preserved.
Strategy B: hash the long URL. Compute a hash such as SHA-256, reduce part of it to a number below 62^7, encode as 7 characters. The same URL always gives the same code, which deduplicates for free, but truncating a hash means collisions: with 6 billion links in a space of 3.5 trillion, roughly 0.17 percent of new inserts would hit an existing code at full scale. You must check the store and, on collision, rehash with a salt and retry, adding a read before every write.
Strategy C: pre-generated keys. A key-generation service creates random unused codes ahead of time and hands out batches. Codes are random, there is no collision check on the hot path, and creation is fast. The cost is another component, and the pool must never give the same batch to two instances.
Strategy
Collisions
Predictable
Coordination
Best when
Counter with ranges and scrambling
Impossible
No, after scrambling
Occasional range grabs
Default choice, simplest to reason about
Hash of URL
Possible, must handle
No
Needs read-then-write
You want dedup of identical URLs
Pre-generated random keys
Prevented by pool
No
A key service
Very high create rates, many instances
flowchart TD
S["Pick a code strategy"] --> Q1{"Identical URLs must share one code?"}
Q1 -->|yes| H["Hash of URL with collision retry"]
Q1 -->|no| Q2{"Need random codes at very high create rates?"}
Q2 -->|no| C["Counter with range blocks plus scrambling"]
Q2 -->|yes| Q3{"Willing to run a key service?"}
Q3 -->|yes| K["Pre-generated random keys"]
Q3 -->|no| C2["Counter with larger ranges and several allocators"]
At this scale the counter with ranges wins unless deduplication is a hard requirement. Custom aliases bypass all of this: they need an atomic "insert if absent" and a reserved-word list. See IDs, clocks and ordering for wider ID trade-offs. The hardest write flow combines a rate-limit check, a reputation scan, code generation and an atomic insert:
sequenceDiagram
participant C as Client
participant S as Create service
participant L as Rate limiter
participant P as Reputation check
participant D as Link store
C->>S: POST link with idempotency key
S->>L: check quota for account and IP
L-->>S: allowed
S->>P: scan destination URL
P-->>S: clean
S->>S: next number from local range, scramble, encode base62
S->>D: insert if absent
alt code or alias already exists
D-->>S: conflict
S-->>C: 409 alias taken, or retry with next number
else stored
D-->>S: ok
S-->>C: 201 with short URL
end
When the in-memory range runs low the service leases the next block in the background, so range allocation never sits on the request path.
Deep dive 2: the redirect and the cache
301 versus 302
301 Moved Permanently
302 Found (temporary)
Browser behaviour
May cache the redirect and skip your server later
Asks your server each time unless told otherwise
Server load
Lower: repeat visitors do not return
Higher: every click arrives
Analytics
Undercounts, cached clicks are invisible
Accurate
Changing or disabling the target
Hard: clients may keep the old target
Easy: effective immediately
Expiry and takedown
Unreliable
Reliable
Because shorteners want analytics, expiry and takedown, 302 (or 307) is the common choice. You can keep some load benefit with Cache-Control: private, max-age=60 (a short lifetime, browser cache only) if slightly lower counts are acceptable. A shared CDN cache would hide clicks from every user, not just repeat visitors, so count at the edge if you let it cache redirects. 301 fits permanent links with no need for per-click counts.
The cache
The workload is classic cache-aside (see caching basics and advanced caching). Key: code. Value: long URL plus expiry and status. Use LRU eviction and a TTL shorter than the link's expiry.
Hot keys: a viral link can send a huge share of traffic to one cache key. Add a small in-process cache for the hottest codes and, if the CDN also reports click events, let it cache with a short TTL.
Negative caching: cache "not found" briefly so scanners guessing random codes cannot hammer the database, and clear that entry when the code is created.
Stampede protection: coalesce concurrent misses for the same key and jitter TTLs.
With a hit ratio above 95 percent (an assumption), the database sees at most about 600 reads per second at the 12,000 peak (5 percent of it), which is why a modest store is enough.
Deep dive 3: expiry, analytics and abuse
Expiry. Store expires_at and check it on every redirect (lazy expiry, always correct). A background job deletes or archives expired rows. Do not recycle expired codes immediately: hold them for a quarantine period so a printed link does not suddenly point to a stranger's destination. With 3.5 trillion codes, recycling is rarely needed.
Analytics. Never write analytics on the redirect path. Publish a small click event (code, timestamp, coarse country, user agent family, referrer) to a queue or log, as in event streaming, and let workers aggregate counts per code per hour into a column-oriented or time-series store. The pipeline is at-least-once, so aggregation should be idempotent or tolerate slight overcounting. If the queue is down, drop or locally buffer events: losing a few clicks beats failing a redirect.
Abuse prevention. Shorteners attract spammers and phishers because they hide the destination.
Validate input: only http and https, a maximum length, and no links pointing back at the shortener (loops).
Check destinations against a blocklist and a reputation service at creation, and re-scan periodically, since a harmless page can turn malicious.
Rate limit creation by account and IP (see rate limiting).
Support takedown: a disabled status the redirect path honours immediately, with an explicit cache and CDN purge.
Be careful fetching URLs server-side for previews: user-supplied URLs can be used to reach internal addresses (server-side request forgery), so use an isolated fetcher with a network allow-list.
Bottlenecks and scaling
Bottleneck
Symptom
Response
Redirect fleet CPU
High latency at peak
Stateless instances, scale out horizontally
Cache memory or a hot node
Misses, uneven load
Consistent hashing across nodes, in-process cache for hot keys
Link store size
Disk and index growth
Shard by hash of code, add replicas for reads
Counter or key service
Create latency or errors
Larger ranges, local range pre-fetch, replicate the allocator
Global latency
Slow redirects far from the primary
Read replicas and cache in several regions, writes to a primary region
Since creation is rare, multi-region is easy for reads (see multi-region and disaster recovery): replicate the store, run redirect services and caches beside each replica, route by latency, and send writes to one region. A newly created link may take a moment to appear elsewhere, so on a replica miss the redirect service can fall back to the primary once.
Failure modes
Cache cluster fails or restarts empty. All reads fall through to a store sized for a high hit ratio. Mitigate with replicated cache nodes, request coalescing, rate-limited warm-up and a circuit breaker (see resilience patterns).
A store shard is down. Replicas serve reads; creation on that shard waits for failover. The cache still serves popular links.
ID allocator unavailable. Instances keep serving from in-memory ranges; alert when remaining range is low.
Analytics pipeline fails. Redirects continue; counts are delayed, then replayed.
Replica lag. A just-created link may 404 from a lagging replica; fall back to the primary on a miss.
Trade-offs to state out loud
Short codes versus unpredictability: seven scrambled base62 characters is a good middle.
Dedup versus privacy and analytics: one code per identical URL saves space but mixes owners' statistics.
302 versus 301: control and accuracy against server load.
Consistency on the read path: a slightly stale cache is fine for redirects, but not for disabled links, so takedown purges explicitly.
Interview cheat-sheet
Topic
What to say
Key numbers (illustrative)
Requirements
Create, redirect, expire, basic analytics; redirects are the critical path
99.99 percent redirect availability
Estimate
Read-heavy, tiny writes, modest data
40 writes per second, 12,000 reads per second at peak, 3 TB over 5 years
API
POST to create with idempotency key, GET redirects, 404 versus 410
302 for analytics
ID generation
Counter with ranges plus scrambling, base62, 7 characters
62^7 about 3.5 trillion
Storage
Key-value access pattern; sharded relational or distributed KV
Cache loss, shard loss, allocator outage, replica lag
Degrade, do not collapse
In practice: operating the reference design
NOTE
A realistic scenario, not a case history. This is a composite example built from common production patterns. It does not describe any specific company, and every number in it is an illustrative assumption.
Imagine the design above run by six engineers on call in rotation, on Kubernetes with PostgreSQL, Redis, Kafka and Prometheus. The question is now how you know it is healthy and what you do when it is not.
SLOs and dashboards
SLI
SLO target (assumed)
Dashboard and alert
Redirect availability (non-5xx for valid codes)
99.99 percent per month
Success ratio panel; multi-window burn-rate alert pages the on-call
Redirect latency at the server
p99 under 100 ms
Latency histogram by region; page when budget burn is fast
Create success rate
99.9 percent
Create errors by cause (conflict, scan unavailable, range exhausted)
Takedown propagation
99 percent of disabled links stop redirecting within 60 seconds
Synthetic probe that disables and polls a test link
Click data freshness
99 percent of events visible in aggregates within 5 minutes
Consumer lag and end-to-end canary event
Cache effectiveness
Hit ratio above 95 percent (a health signal, not an SLO)
Hit ratio, evictions, memory per node
Capacity-review cadence
Weekly, skim trends for store size growth, remaining range blocks and cache hot-set size. Monthly, review peak requests per second against the measured per-pod capacity and headroom (see capacity and cost). Quarterly, run a load test at 1.5 times the forecast peak and a regional failover drill. Before any large customer campaign, ask for the expected link volume and plan a temporary scale-up.
Runbook
Symptom
Likely cause
First check
Mitigation
Redirect p99 and database CPU rising together
Cache hit ratio fell: node restart, eviction pressure or a stampede
Hit ratio and evictions panel
Enable request coalescing, scale or warm the cache, shed anonymous traffic if needed
New links return 404 in region B
Replica lag
Replication lag metric
Confirm fallback to primary on miss; throttle bulk creators
Create errors with "range exhausted"
Allocator unreachable or block size too small
Remaining-range gauge per pod
Lease ranges manually, raise block size
Click counts flat or delayed
Topic or consumer lag
Consumer lag per partition
Scale workers, replay from the topic
A disabled link still redirects
CDN or in-process cache not purged
Purge job status, CDN TTL for that path
Purge the key everywhere, shorten TTL
Burst of 404s from few IPs
Code scanning
Negative-cache hit ratio, WAF logs
Rate limit or block the sources
Decision record: 302 with a short cache lifetime
Field
Entry
Context
Product wants accurate counts and reliable takedown; infrastructure wants lower redirect load.
Decision
Return 302, with Cache-Control: private, max-age=60 so only the user's browser absorbs repeat clicks briefly; a shared CDN cache would hide clicks from origin counts, so cache redirects at the edge only if the edge also emits click events. Offer 301 only as an explicit per-link option for permanent links.
Consequences
Good: takedown within about a minute plus a purge, accurate counts for all but a user's own repeat clicks within 60 seconds. Bad: more redirect traffic than 301 would cause, and counts slightly low for rapid repeats.
Revisit when
Redirect compute becomes a large share of the bill, or customers ask for SEO-friendly permanent links.
Cost view
Cost driver
Why it matters
Levers
Redirect compute and CDN requests
Billions of tiny requests; each is cheap, the count is not
Longer CDN TTL where takedown rules allow and clicks are counted at the edge, in-process caching
Cache memory
Hot set is about 12 GB plus replicas
Size to the measured hot set, not total links
Link store and replicas
About 3 TB over five years, times replicas and indexes
Sample or aggregate at the edge, keep raw events for a short retention
Logs and metrics
Logging every redirect is enormous
Sample access logs, keep error logs complete
Egress is small because each response is tiny; request counts and the analytics pipeline shape the bill.
Architect's notes
Measure the hot set before sizing the cache. Count how few codes cover 90 percent of a day's clicks; that sets cache size and CDN settings.
Ask what a takedown costs end to end. If purging cache, CDN and replicas is not one tested command, abuse handling is slower than your policy promises.
Analytics is where the budget hides. Agree retention and sampling with the product owner early.
Decide who owns abuse. Trust and safety needs a queue, an SLA and a way to disable links without an engineer.
Document the code-space exit. Decide what happens at 70 percent of the space before anyone has to improvise.
Defer multi-region writes. Single-region writes with regional read caches cover nearly every requirement at this write rate.
Common mistakes
A random or hash code without a collision plan. Duplicates overwrite or break links. Use a scheme that cannot collide, or check and retry atomically.
Making the redirect depend on analytics. A slow queue then slows every click. Publish events fire-and-forget.
Choosing 301 and then promising accurate analytics or takedowns. Browsers cache it. Use 302, or accept the limits.
A single global counter with no range allocation. It is a bottleneck and single point of failure. Hand out blocks.
Ignoring abuse. An open shortener becomes a phishing relay and gets blocklisted. Validate, scan, rate limit and support takedown.
No negative caching. Random-code scanning bypasses the cache and loads the database.
Over-engineering the store. 3 TB at 40 writes per second needs no exotic technology.
Key takeaways
It is a read-heavy key-value problem. Optimise redirects with caching and simple lookups; keep writes correct.
Estimate first. The numbers show the data is modest and the reads are cacheable.
Seven base62 characters give about 3.5 trillion codes. Enough for decades at this scale.
Counter with range allocation and scrambling is a strong default. Hashing needs collision handling; pre-generated keys add a service.
302 keeps control and analytics; 301 saves load but loses both. State the trade-off explicitly.
Keep analytics off the critical path. Async events, eventually consistent counts.
Abuse prevention is part of the design. Validation, scanning, rate limits, takedown and SSRF care.
Plan the failures. Cache loss, shard loss and replica lag each have a specific mitigation and a runbook entry.
Practice exercises
Exercise 1 — Starter (30 min): Convert the integer 125,000,000 to base62 by hand, using the digits 0-9, a-z, A-Z in that order for values 0 to 61. Then calculate how many seven-character codes exist, and how long it would take to use them all at 1,000 new links per second.
Exercise 2 — Intermediate (2 hours): Redo the estimate for a service ten times larger (1 billion new links per month, 200 to 1 read ratio). Calculate writes per second, peak reads per second, five-year storage, cache size for the hottest 10 percent, and the number of redirect instances if one instance handles 3,000 requests per second at the target latency. State which numbers change your architecture and which do not.
Exercise 3 — Advanced (half day): Design the abuse-prevention subsystem. Draw the creation flow with validation, reputation checks, rate limits and an asynchronous re-scan. Define how a takedown propagates to the cache, CDN and replicas within a target of 60 seconds, what happens to codes of disabled links, and how you would measure false positives. Write the failure behaviour if the reputation service is unavailable (fail open or fail closed, and why).
Lesson 9 of 14 · Advanced System Design and Case Studies
Design a URL shortener end to end: requirements, estimates, API, base62 ID generation, data model, caching, 301 versus 302, expiry, analytics, abuse prevention and scaling.
In the rest of this lesson
High-level design
Deep dive 1: generating the short code
Deep dive 2: the redirect and the cache
Deep dive 3: expiry, analytics and abuse
Bottlenecks and scaling
Failure modes
Enrolling is free. It unlocks every lesson in Advanced System Design and Case Studies, tracks your progress across devices and keeps your quiz scores. The newsletter is optional.
Real architecture decisions, LLMOps patterns that survive production, and engineering leadership advice — from 12+ years of building at enterprise scale. Free. No spam. Unsubscribe anytime.
Get new articles by email
Free. We email a confirmation link first, and you can unsubscribe from any email.