System Design Case Study: Video Streaming

Reference deployment topology

Deployed, the system is three mostly independent planes: a control plane for metadata, a batch plane for transcoding, and a delivery plane for bytes.

How to read it: a failure in the batch plane delays new videos but does not interrupt playback of videos already published, and the delivery plane needs no application servers to serve segments (the player only calls the control plane to get the manifest URL).

Deep dive 1: the transcoding pipeline

Transcoding converts the original (any codec, resolution and frame rate) into a standard set of renditions, for example from a small mobile-friendly size up to high definition, each at a target bitrate. A single long video transcoded on one machine is slow and fragile, so the work is parallelised and checkpointed.

Pipeline as a DAG. A directed acyclic graph of tasks, run by an orchestrator that records state for each task:

1. Inspect and validate: probe the file, reject corrupt or unsupported input, extract metadata.
2. Split: cut at keyframe boundaries (a keyframe decodes without earlier frames, so each chunk is independently decodable), in chunks of a few to tens of seconds.
3. Transcode in parallel: one task per chunk and rendition. A ten-minute video with 6 renditions and 10-second chunks produces 360 small tasks.
4. Package: assemble each rendition's segments, write the manifests, and optionally encrypt segments.
5. Side tasks in parallel: thumbnails, audio extraction, moderation, captions, loudness normalisation.
6. Publish: write metadata, set status ready, notify the creator.

Why chunks make it robust. If a worker dies, only its chunk is retried, not the whole video. Tasks are idempotent because output keys are deterministic: chunk 17 at rendition 720p always writes the same key. Spot or preemptible machines, which are cheaper but can disappear, become practical. The orchestrator can be a workflow engine, because it needs durable state, retries, timeouts and visibility into stuck jobs (see distributed transactions for the saga mindset, and the Temporal article for workflow engines).

Prioritise. Processing a creator's first video, or a video from a partner with a scheduled release, should not wait behind a bulk catalogue re-encode. Use priority queues or separate worker pools, and cap the cost of the low-priority pool.

Quality versus cost. More compute spent on encoding produces smaller files at the same visual quality, and the saving repeats on every view. Popular videos justify an expensive second pass with a more efficient codec, while the long tail does not. A common policy is to encode cheaply first so the video is available fast, then re-encode only what turns out to be popular.

How to read it: every video gets the cheap path, and only the popular few earn the expensive pass, which pays back through lower egress on each later view.

Deep dive 2: adaptive bitrate streaming and delivery

The problem. Viewers have different screens and constantly changing bandwidth. A single fixed-quality file stalls on slow links and wastes bits on fast ones.

The solution: adaptive bitrate (ABR) over HTTP. Each rendition is cut into short segments (commonly a few seconds each). A manifest lists the renditions and their segment URLs. HLS (HTTP Live Streaming) uses playlist files, typically .m3u8, and DASH (Dynamic Adaptive Streaming over HTTP) uses an XML manifest, typically .mpd. They differ in format and ecosystem but share the idea. Many services publish both or use a common segment format so one set of media files can serve both.

The player downloads the manifest, picks a starting rendition, and fetches segments over plain HTTP. After each segment it measures throughput and buffer level and chooses the next segment's rendition: step up when the buffer is healthy and bandwidth is high, step down quickly when the buffer is draining. Switching works because every rendition's segments are aligned in time, so any segment boundary is a valid switch point.

Why plain HTTP and a CDN? Segments are small, immutable static files. Any ordinary web cache can store them, and no special streaming server is needed per viewer. Popular segments are served from edge caches near viewers. Because each segment is a distinct URL, the cache key is simple and the hit ratio is high for popular titles. A shield or mid-tier cache cuts origin load further. The first viewer after upload suffers a cold cache, so for highly anticipated releases you can pre-warm edges by requesting the first segments in advance.

Fast start. Begin at a low or medium rendition to get the first frame quickly, then ramp up. Keep the first segments short and put the manifest and initial segments in the cache of the viewer's region.

Long tail versus head. A small fraction of videos account for most views. Edge caches have finite space, so unpopular videos mostly miss the edge and are served from the origin or a regional tier. That is acceptable if the origin is sized for the tail, and it is the reason storage tiers exist.

Deep dive 3: storage tiers and cost

Cost has three big drivers: egress, storage and encoding compute.

• Egress. Usually the largest. Levers: efficient codecs, per-title ladders that avoid over-provisioned bitrates, a high CDN hit ratio, negotiated CDN pricing, and (in very large deployments) caches inside network providers' data centres. Capping the top bitrate on small screens saves bandwidth with little visible loss.
• Storage. Keep originals (so you can re-encode with future codecs) in a cooler tier, and hot renditions in the standard tier. Move rarely watched titles' high-bitrate renditions to colder storage and regenerate from the original if they become popular again, using lifecycle rules. Archival retrieval can be slow and carries fees, so check your provider's documented behaviour before relying on it for playback.
• Compute. Use interruptible machines for the bulk pool, schedule low-priority re-encodes off-peak, and skip renditions no device would pick.

Thumbnails and metadata

Generate several thumbnail candidates at evenly spaced times, then choose one by a simple heuristic (avoid black frames, prefer sharp ones) or let the creator choose. Generate a sprite sheet of tiny frames and a small index so the player can show hover previews while seeking without fetching video. These are small images served through the CDN like any other asset. Metadata, titles and counters live in the database. View counters should not be incremented per request in the main database: buffer them in a stream or in memory and aggregate periodically, accepting that counts are approximate for a while.

Bottlenecks and scaling

• CDN capacity and egress. The biggest lever is hit ratio. Monitor it per title class and per region.
• Origin storage reads on cache misses. A new viral video misses on many edges at once. Use a shield tier and request coalescing so one origin read serves many edge requests.
• Transcoding backlog. Scale workers on queue age and use priorities. Reserve capacity for fast-turnaround uploads.
• Metadata database. Read-heavy and small: cache and replicate. It is rarely the bottleneck.
• Upload bandwidth. Direct-to-storage uploads spread across the object store's capacity, not your servers'.

Failure modes

The rule "segments first, manifest last" is the video version of publishing atomically: the manifest is the commit point.

Trade-offs worth stating

Interview cheat sheet

In practice: operating the reference design

Assume a team of fourteen runs the control and batch planes on Kubernetes, with PostgreSQL for metadata, Redis for caching, a workflow engine such as Temporal for the pipeline, an S3-compatible object store with lifecycle tiers, FFmpeg-based encoders, one or two CDN vendors, and Prometheus and Grafana for observability.

SLOs and dashboards

Client-side metrics matter most here, so the player must report startup time and rebuffering. Server logs alone cannot see what a viewer experienced.

Capacity-review cadence

• Weekly: transcode queue age by priority, spot interruption rate, hit ratio on new releases, spend against budget.
• Monthly: storage by tier, share of views from the top 1% of titles, re-encode backlog and payback, CDN contracts versus actual egress.
• Quarterly: a failover drill between CDNs, a chunk-level retry test under forced worker kills, an estimate refresh against the roughly 11 PB per day egress assumption.
• Before scheduled releases: pre-warm the first segments and reserve worker capacity.

Runbook

Decision record: cheap-first encoding with upgrade for popular titles

Cost view

Egress comes first by a wide margin at these assumptions, then storage, then compute.

Architect's notes

• Instrument the player before the backend. Startup time and rebuffering are the user's truth. Backend dashboards can be green while viewers suffer.
• Estimate egress per watch hour, then track it. It is a number both finance and engineering can act on.
• Ask what happens to originals. Retention decisions are cheap on day one and impossible to reverse after deletion.
• The orchestrator is a critical dependency. Give it the same availability thinking as the database, including backups of workflow state.
• Multi-CDN is a business decision as much as a technical one. It adds routing logic, extra monitoring and contract complexity. Defer it until a single-vendor incident or price leverage justifies it.
• Retrieval fees are a hidden trap. Cold tiers save on storage, but a viral archived title can cost more than it saved. Model the worst week.

Common mistakes

• Serving video from application servers. They cannot supply the bandwidth and tie up connections. Serve segments from a CDN.
• Transcoding whole files on one worker. One failure restarts hours of work. Chunk, parallelise and checkpoint.
• One quality for everyone. Slow viewers stall and fast viewers waste bits. Use an adaptive ladder.
• Ignoring egress cost in the estimate. It is usually the largest bill. Estimate it first.
• Publishing a manifest before all segments exist. Players hit missing files. Write segments first and the manifest last.
• Deleting originals after encoding. You lose the ability to re-encode for new codecs and devices. Archive them cheaply instead.
• Counting views with a database write per view. The database becomes the bottleneck. Aggregate through a stream or buffered counters.

Key takeaways

• Video is bandwidth-bound. Egress at the CDN dominates both load and cost.
• Process once, serve many times. Spend compute at upload to save bandwidth on every view.
• Chunk the work. Parallel, idempotent chunk tasks make transcoding fast and resilient.
• Adaptive bitrate is segment-based HTTP. HLS and DASH manifests let players switch quality at segment boundaries.
• Segments are static files. Immutable, uniquely named objects are ideal for CDN caching.
• Tier your storage. Keep originals cold, hot renditions warm, and regenerate tail content on demand.
• The manifest is the commit point. Publish it only when everything it references exists.
• Cost is a first-class requirement. Track egress, storage and compute with budgets and alerts.

Practice exercises

Exercise 1 — Starter (30 min): Using the illustrative assumptions, recompute egress per day if the average bitrate drops from 3 Mbps to 2 Mbps with the same watch time, and if average watch time rises to 8 minutes. Express the result in PB per day and as peak Tbps assuming a 3x peak factor.

Exercise 2 — Intermediate (2 hours): Design the DAG for a 30-minute upload with a six-rung ladder and 10-second chunks. Count the tasks, identify which can run in parallel, define the retry and timeout policy per task type, and describe how the orchestrator decides the video is ready when only the highest rung fails repeatedly.

Exercise 3 — Advanced (half day): Propose a storage and encoding policy that saves money for a catalogue where 1% of videos get 80% of views. Define the initial encode, the trigger for a better re-encode, the lifecycle rules for each tier, and the metric you would use to prove the policy is paying for itself. Include what happens when an archived video suddenly goes viral.

Related reading

• CDN and object storage - the basics of edge caching, cache keys and multipart upload.
• System Design Case Study: Photo Sharing App - the image version of the same upload and processing pattern.
• Message queues - job dispatch, retries and dead-letter handling for the pipeline.
• Capacity planning and cost - cost drivers, headroom and efficiency levers.
• Temporal replacing cron and Spring Batch - a practical look at durable workflow orchestration.