Skip to content

    System Design

    TCP and UDP: Reliable Streams, Datagrams and What Each Costs

    Learn what TCP guarantees and what it costs, what UDP leaves to you, and watch loss, delay and duplicates hit both in a deterministic Java model.

    By Published 9 min read

    Share

    Standard lessonbeginnerJava labBackend System Design

    Before this: The Request Lifecycle

    What you'll learn: By the end of this lesson you will be able to say exactly what TCP guarantees (a reliable, in-order byte stream) and what UDP does not (delivery, order, duplicate protection), describe the handshake, acknowledgments, the window, the retransmission timer and fast retransmit, explain why one lost segment delays everything behind it, predict what an application sees on each protocol when the network loses, delays or duplicates data, and choose a transport for a given workload, using a Java model that runs eight scripted network scenarios on both.

    Think of two ways to send eight chapters of a book. The first is a registered courier service: the courier phones ahead to agree the delivery, numbers every parcel, waits for a receipt, re-sends anything that goes missing, and the reader only ever gets chapters in order. The second is a pile of postcards: you drop each one in the post box, nothing is confirmed, some arrive late, some twice, one may never arrive, and if order matters the reader must work it out from the page numbers you wrote.

    TCP is the courier and UDP is the postcards. The request lifecycle in The Request Lifecycle shows where TCP sits in a request; this lesson is the dedicated treatment of the transport itself. It is the base for TLS and HTTPS, which runs on top of TCP, and for HTTP/1.1, HTTP/2 and HTTP/3, where HTTP/3 moves to UDP.

    Verified with RFC 9293 (TCP), RFC 768 (UDP), RFC 5681 and RFC 6298, and a Java 25 toolchain lab, on 2026-11-16.

    The core idea

    TCP gives an application a reliable, in-order byte stream and pays for it with a connection set-up round trip, per-connection state and waiting; UDP gives it individual datagrams with no promise of delivery, order or duplicate protection and pays nothing. RFC 9293 puts the first half as "TCP provides a reliable, in-order, byte-stream service to applications". RFC 768 puts the second as "delivery and duplicate protection are not guaranteed".

    Neither is better: TCP copes with an unreliable network inside the operating system in one way for every application; with UDP each application decides.

    Three illustrations run through the lesson, each labelled:

    • Minimal example: the Java model and its scripted scenarios in "Build it".
    • Realistic example: ShopStream's courier location pings in "In practice".
    • Failure example: a lost segment that holds back everything behind it (head-of-line blocking), and a service that opens a new connection per call.

    What each protocol promises

    Property TCP UDP
    Unit A byte stream: no message boundaries A datagram: one send is one message
    Set-up Three-way handshake before data None
    Delivery Lost data is retransmitted A lost datagram is simply gone
    Order Data reaches the application in order Arrival order, whatever it is
    Duplicates Discarded by the receiver Delivered to the application
    Flow control Receiver's advertised window None
    State per peer Yes: numbers, timers, buffers None

    The "byte stream" row matters: two send calls can arrive as one read, so a protocol on TCP needs its own framing (a length prefix, HTTP's headers). A UDP datagram arrives whole or not at all.

    A stream hides the network; datagrams show itSender app to TCP (write bytes); TCP to Network; Network to TCP; TCP to Receiver app (in order); Sender app to Network (datagrams); Network to Receiver app (as they are)Sender appSender appTCP: numbers, ACKsTCPnumbers, ACKsNetwork: loses, delays, duplicatesNetworkloses, delays,duplicatesTCP: repairsTCPrepairsReceiver app: sees a clean streamReceiver appsees a cleanstreamSender appSender appReceiver app: sees what arrivesReceiver appsees what arriveswrite bytesin orderdatagramsas they are

    Figure 1. The same network under two transports. How to read this: the upper path passes through TCP on both sides, which repairs what the network does before the application sees it. The lower path goes straight through, so the receiving application sees loss, delay and duplication itself.

    TCP: connecting

    RFC 9293 section 3.5 calls the "three-way handshake" the procedure used to establish a connection. The client sends a SYN, the server answers with a SYN-ACK, and the client confirms. The cost to remember is time: the first byte of data cannot be sent until one full round trip has passed.

    sequenceDiagram accTitle: The TCP three-way handshake accDescr: The client sends a SYN, the server replies with a SYN-ACK, and the client sends an ACK. Data can follow once the client has the SYN-ACK, one round trip after it started. participant C as Client participant S as Server C->>S: SYN S-->>C: SYN-ACK Note over C: one round trip has passed, connected C->>S: ACK, then data

    Figure 2. Connection set-up costs one round trip. How to read this: at 50 ms each way the client waits 100 ms before it can send its first byte. The lab prints exactly this: "SYN-ACK received, connection established (one round trip)" at 100 ms.

    That 100 ms is why reusing connections matters: a connection pool or HTTP keep-alive pays it once. HTTP/1.1, HTTP/2 and HTTP/3 covers the persistent-connection side, and TLS and HTTPS adds its own round trip on top.

    TCP: staying reliable

    Four mechanisms do the work. The lab models each one.

    1. Numbering and cumulative acknowledgments. Data is numbered, and RFC 9293 says the "acknowledgment mechanism employed is cumulative": an ACK names the next piece the receiver needs, so one ACK covers everything before it. A lost ACK therefore often costs nothing, because the next ACK covers it (lab scenario 8).
    2. A window. The segment header carries a Window field, the number of octets the sender of the segment is willing to accept, which lets a slow receiver slow the sender. The lab uses a fixed sender window of 3 to 6 segments. If a receiver advertises a zero window, RFC 9293 requires the sender to support probing it and says it SHOULD increase the interval between probes exponentially.
    3. A retransmission timer. RFC 9293 requires the retransmission timeout (RTO) to be computed by the algorithm of RFC 6298. That RFC says to back off on expiry ("The host MUST set RTO <- RTO * 2") and to restart the timer when an ACK acknowledges new data. Its recommended initial RTO is 1 second; the lab uses a first timeout of 300 ms only so the output stays short, an illustrative figure and not a recommendation. Eventually TCP gives up: RFC 9293 says to close the connection when the number of transmissions of the same segment reaches a threshold R2.
    4. Fast retransmit. Waiting for a timer is slow. RFC 5681 has the receiver send "an immediate duplicate ACK when an out-of-order segment arrives", and the sender treats the arrival of three duplicate ACKs as a loss signal and retransmits at once, without waiting for the timer.
    sequenceDiagram accTitle: Fast retransmit after three duplicate ACKs accDescr: The sender sends segments 1 to 4. Segment 1 is lost. The receiver gets 2, 3 and 4 out of order and sends three duplicate ACKs asking for segment 1. After the third duplicate ACK the sender retransmits segment 1 without waiting for its timer, and the receiver then delivers 1 to 4 in order. participant S as Sender participant R as Receiver S->>R: segment 1 (lost) S->>R: segment 2 S->>R: segment 3 S->>R: segment 4 R-->>S: ACK for 1 (duplicate) R-->>S: ACK for 1 (duplicate) R-->>S: ACK for 1 (duplicate) S->>R: segment 1 again (fast retransmit) R-->>S: ACK for 5, all delivered in order

    Figure 3. Fast retransmit. How to read this: three segments behind the lost one produce three duplicate ACKs, which is the trigger. With only two segments behind it, only two duplicates come back and the sender must wait for the timer; lab scenario 3 and the traced scenario show the difference.

    The price: head-of-line blocking

    Because the application is promised an in-order stream, a lost segment holds back every later segment, even ones that have already arrived. The receiver buffers them and releases them together when the gap is filled. This is the cost you pay for reliability, and it is one reason HTTP/3 runs over QUIC, which RFC 9000 titles "A UDP-Based Multiplexed and Secure Transport"; HTTP/1.1, HTTP/2 and HTTP/3 shows how that changes what a loss stalls. In the lab, scenario 2 is the plain case: segment 2 is lost, and the application sees nothing past segment 1 until a timeout resends it.

    Closing: TIME-WAIT

    A closed connection is not forgotten at once. RFC 9293 section 3.6.1 says an endpoint closing a connection "MUST linger in the TIME-WAIT state for a time 2xMSL", and section 3.4.2 takes the Maximum Segment Lifetime to be 2 minutes, so a single closed connection can occupy state for 4 minutes. A service that opens a new connection for every call and closes it leaves many in this state; that is the failure example in "In practice".

    UDP: what is left to you

    RFC 768 defines UDP as "a datagram mode of packet-switched computer communication". The header has four fields: source port, destination port, a length (at least 8 octets, header included) and a checksum. There is no sequence number, no acknowledgment and no connection, so a sender never learns whether a datagram arrived. The receiving application sees what the network delivers: lost datagrams are missing, a late one arrives after its successors, a duplicated one arrives twice.

    Applications that use UDP add only what they need. DNS (see DNS) sends a query and retries if no answer comes; QUIC builds multiplexed streams and an integrated TLS handshake on top of it; a live video call skips old data on purpose, since a late frame is worth nothing.

    Build it: TCP-like stream versus UDP datagrams in Java

    This is the minimal example, a model you can run. It is a model of the ideas above, not an implementation of TCP: there are no sockets and no packets, and the clock is a manual event queue, so the output is the same on every run.

    Goal. Send the same eight pieces of data across the same scripted network once as a TCP-like stream and once as UDP datagrams, and see what the application receives and when.

    Setup. From the repository's examples/backend folder:

    ./gradlew :labs:tcp-udp-model:test
    ./gradlew -q :labs:tcp-udp-model:run

    The first command runs 23 tests. The second prints the table and trace below. The code is in examples/backend/labs/tcp-udp-model/src/main/java/buildingai/labs/tcpudpmodel/: TcpTransfer.java and UdpTransfer.java.

    How the model works. A Plan scripts what the network does to a given transmission (drop it, deliver it late, deliver it twice, drop an ACK) and a Link applies it plus 50 ms of latency; nothing is random. The TCP-like sender connects in one round trip, keeps at most window segments in flight and runs one timer for the oldest unacknowledged segment, doubling it on each expiry; the receiver sends cumulative ACKs, buffers early segments and hands data over only in order; with fast retransmit on, the third duplicate ACK resends at once. The UDP side sends each datagram once. Left out: real headers and byte numbering, congestion control, the advertised window, round-trip estimation, delayed and selective ACKs, teardown.

    Steps. Run the tests, then the demo. The tests include that a TCP-like transfer delivers everything exactly once and in order whatever finite loss and delay the network inflicts, that the timeout doubles after every expiry for the same segment, and that fast retransmit needs three segments behind the loss.

    Expected output, captured from a real run (Java 25 toolchain; the file examples/golden/tcp-udp-model.txt was captured on 2026-11-08 and a re-run on 2026-11-16 printed identical output):

    TCP-like stream versus UDP datagrams: 8 pieces of data, 50 ms one way (100 ms round trip), first timeout 300 ms.
    'app sees' lists what the receiving application got, in the order it got it.
    'complete at' is when the receiving application has everything it will ever get, in ms from the start.
    
    #  scenario                                                    app sees                       missing  retransmits                          complete at
    ------------------------------------------------------------------------------------------------------------------------------------------------------
    1  clean network, window 4                                TCP  [0, 1, 2, 3, 4, 5, 6, 7]       []       0                                    250 ms
                                                              UDP  [0, 1, 2, 3, 4, 5, 6, 7]       []       0                                    50 ms
    2  segment 2 lost, window 6, timer only                   TCP  [0, 1, 2, 3, 4, 5, 6, 7]       []       1 (1 timeout)                        550 ms
                                                              UDP  [0, 1, 3, 4, 5, 6, 7]          [2]      0                                    50 ms
    3  segment 2 lost, window 6, fast retransmit              TCP  [0, 1, 2, 3, 4, 5, 6, 7]       []       1 (1 fast)                           250 ms
                                                              UDP  [0, 1, 3, 4, 5, 6, 7]          [2]      0                                    50 ms
    4  segment 1 late by 120 ms, window 3, fast retransmit    TCP  [0, 1, 2, 3, 4, 5, 6, 7]       []       0                                    470 ms
                                                              UDP  [0, 2, 3, 4, 5, 6, 7, 1]       []       0                                    170 ms
    5  segment 1 late by 120 ms, window 6, fast retransmit    TCP  [0, 1, 2, 3, 4, 5, 6, 7]       []       1 (1 fast), 1 duplicate discarded    350 ms
                                                              UDP  [0, 2, 3, 4, 5, 6, 7, 1]       []       0                                    170 ms
    6  segment 1 lost twice in a row, window 4, timer only    TCP  [0, 1, 2, 3, 4, 5, 6, 7]       []       2 (2 timeout)                        1250 ms
                                                              UDP  [0, 2, 3, 4, 5, 6, 7]          [1]      0                                    50 ms
    7  segment 3 delivered twice by the network, window 4     TCP  [0, 1, 2, 3, 4, 5, 6, 7]       []       0, 1 duplicate discarded             250 ms
                                                              UDP  [0, 1, 2, 3, 3, 4, 5, 6, 7]    []       0                                    50 ms (1 duplicate)
    8  one ACK lost, window 4                                 TCP  [0, 1, 2, 3, 4, 5, 6, 7]       []       0                                    250 ms
                                                              UDP  [0, 1, 2, 3, 4, 5, 6, 7]       []       0                                    50 ms
    
    Trace of one scenario: 4 pieces of data, window 4, segment 1 lost, timer only.
    
    TCP-like stream:
        0 ms  client: SYN sent
       50 ms  server: SYN received, SYN-ACK sent
      100 ms  client: SYN-ACK received, connection established (one round trip)
      100 ms  client: DATA #0 (bytes 0-99) sent
      100 ms  client: DATA #1 (bytes 100-199) sent
      100 ms  network: DATA #1 lost
      100 ms  client: DATA #2 (bytes 200-299) sent
      100 ms  client: DATA #3 (bytes 300-399) sent
      150 ms  server: DATA #0 in order
      150 ms  server: segment #0 delivered to the application
      150 ms  server: ACK 100 sent
      150 ms  server: DATA #2 out of order, buffered (still waiting for #1)
      150 ms  server: ACK 100 sent
      150 ms  server: DATA #3 out of order, buffered (still waiting for #1)
      150 ms  server: ACK 100 sent
      200 ms  client: ACK 100 received (next expected is #1)
      200 ms  client: ACK 100 received (next expected is #1)
      200 ms  client: duplicate ACK number 1 for #1
      200 ms  client: ACK 100 received (next expected is #1)
      200 ms  client: duplicate ACK number 2 for #1
      500 ms  client: retransmission timer expired (RTO 300 ms), oldest unacknowledged is #1
      500 ms  client: DATA #1 (bytes 100-199) sent (transmission 2)
      550 ms  server: DATA #1 in order
      550 ms  server: segment #1 delivered to the application
      550 ms  server: segment #2 delivered to the application
      550 ms  server: segment #3 delivered to the application
      550 ms  server: ACK 400 sent
      600 ms  client: ACK 400 received (next expected is #4)
      600 ms  client: everything acknowledged
    
    UDP datagrams:
        0 ms  sender: datagram #0 sent
        0 ms  sender: datagram #1 sent
        0 ms  network: datagram #1 lost
        0 ms  sender: datagram #2 sent
        0 ms  sender: datagram #3 sent
       50 ms  receiver: datagram #0 arrived
       50 ms  receiver: datagram #2 arrived
       50 ms  receiver: datagram #3 arrived

    Read the table row by row. The numbers are the model's, not measurements of any real network.

    Scenario What it shows
    1 Clean network: TCP completes at 250 ms, UDP at 50 ms. TCP pays the 100 ms handshake and waits for ACKs before the second window of data.
    2 Segment 2 lost, timer only: TCP delivers everything at 550 ms after one timeout; UDP's application never gets 2.
    3 The same loss with fast retransmit: TCP completes at 250 ms, as in the clean run.
    4 and 5 Segment 1 arrives 120 ms late: UDP hands over [0, 2, 3, ..., 1], TCP always [0, 1, 2, ...]; with window 6 TCP also sends one needless retransmit.
    6 Segment 1 lost twice: two timeouts, 1250 ms for TCP; UDP never gets 1.
    7 The network delivers segment 3 twice: UDP's application sees it twice, TCP discards the copy.
    8 One ACK lost: no retransmit, because the next cumulative ACK covers it.

    The trace at the end shows one scenario in full: window 4, segment 1 lost. Only two duplicate ACKs arrive (segments 2 and 3 are behind the loss), one short of fast retransmit, so the client waits for its timer at 500 ms, resends, and segments 1, 2 and 3 are delivered together at 550 ms. That is head-of-line blocking, line by line.

    Break it. In TcpTransfer.onTimeout, comment out currentRto *= 2; and theTimeoutDoublesAfterEveryExpiryForTheSameSegment fails. In TcpTransfer.senderOnAck, change the duplicate ACK threshold from 3 to 2: tcpHidesReorderingFromTheApplication fails, because mild reordering is now mistaken for loss. Put them back afterwards.

    Stretch. Add a receiver-advertised window, or replace the fixed timeout with the smoothed round-trip estimate from RFC 6298.

    Choosing a transport: the trade-offs

    Need TCP UDP Wrong when
    Every byte must arrive, in order Built in You rebuild it You reimplement TCP badly on UDP
    Fresh data matters more than old A loss delays everything behind it Late data can be dropped Old data still matters
    Many short calls Pay the handshake each time unless connections are reused No set-up You open and close TCP connections per call
    Congestion behaviour TCP backs off under loss (RFC 5681) None built in A UDP sender floods a shared link
    flowchart TD accTitle: Choosing between TCP and UDP accDescr: A decision flow. If every byte must arrive in order, use TCP. If old data is worthless once newer data exists and the application can tolerate gaps, consider UDP with application-level handling. If the application needs reliability and avoids head-of-line blocking across streams, consider QUIC, which runs over UDP. S["A new network protocol or service"] --> Q1{"Must every byte arrive, in order?"} Q1 -->|"yes"| Q2{"Do independent streams share one connection and suffer from one loss?"} Q2 -->|"no"| T["Use TCP"] Q2 -->|"yes"| Q["Consider QUIC, which runs over UDP"] Q1 -->|"no"| Q3{"Is old data worthless once newer data exists?"} Q3 -->|"yes"| U["UDP, with sequence numbers and no retransmission"] Q3 -->|"no"| U2["UDP plus your own acknowledgments, or just use TCP"]

    Figure 4. A starting rule for choosing a transport. How to read this: it is a default to argue with. The most common correct answer for a back-end service is TCP; the cases for UDP are real-time data where freshness beats completeness, protocols such as DNS with one-shot exchanges, and QUIC.

    In practice: ShopStream's courier location pings

    This is the realistic example, an educational scenario on the course's running example ShopStream; nothing in it describes a real company. Every number other than the shared ShopStream figures is illustrative.

    This is a realistic scenario, not a case history.

    The situation

    ShopStream couriers' phones report their position so shoppers can watch a delivery on a map. A team member proposes UDP because "location data can be dropped, and TCP's retransmissions will make the map lag". The service must also keep working on weak mobile connections.

    Starting architecture

    ShopStream courier pings: phone to tracking serviceCourier phone to Edge (HTTPS over TCP); Edge to Tracking service; Tracking service to Location store (upsert latest)Courier phone: position every few secondsCourier phoneposition everyfew secondsEdge: terminates TLSEdgeterminates TLSTracking serviceTrackingserviceLocation store: latest position per courierLocation storelatest positionper courierHTTPS over TCPupsert latest

    Figure 5. The ping path today. How to read this: each ping is a small request carrying one position. The store keeps only the latest, so an old ping that arrives late must not overwrite a newer one.

    Options considered

    Option Benefit Cost Risk
    A. Keep HTTPS over TCP Works through every network; TLS and retries exist A loss delays later pings on that connection Map lag on bad networks
    B. Raw UDP No head-of-line blocking Own security, own framing, firewalls may block it Spoofed or lost data, an extra protocol to run
    C. TCP, but send only the latest position and put a timestamp on every ping Keeps TCP's benefits, avoids stale data A small change in the app Needs a timestamp and a "latest wins" rule

    The decision

    Option C. The lag is real but comes from queued old pings, not from TCP itself: when a stalled connection recovers, the stale positions go out first. The fix is in the application: send the newest position, drop older queued ones and let the store keep the newest timestamp. Option B is revisited only if measurement shows head-of-line blocking is the cause, and then HTTP/3 over QUIC is a better step than raw UDP because it keeps encryption.

    Rollout and verification

    Add the timestamp and "latest wins" rule behind a flag, then compare map age (now minus the newest position's timestamp) at p50 and p99 on weak connections before and after. Verification: p99 map age falls and no store write moves a courier backwards in time.

    The 2 a.m. failure

    The page reads: connect timeouts and "cannot assign requested address" errors from the tracking service to its location store, rising with traffic. A recent change replaced a pooled client with one that opens a new connection per upsert and closes it. Each closed connection lingers in TIME-WAIT for 2 x MSL (4 minutes under RFC 9293's 2-minute MSL), so at the service's call rate the closed-connection state piles up faster than it clears and new connections cannot be set up.

    The checks: count sockets by TCP state on the service host and compare connections opened per second with requests per second. The mitigation is rolling back to the pooled client; the lasting fix is a pool with keep-alive, a metric for connections opened per second and an alert on sockets in TIME-WAIT.

    Architect's notes

    • Default to TCP and say why you would leave it. UDP needs a written reason: freshness over completeness, one-shot exchanges, or QUIC.
    • Reuse connections. Every new TCP connection costs a round trip and later 2 x MSL of state; pools and keep-alive pay both once.
    • Timestamp, do not trust arrival order. Two connections or two retries can reorder events even over TCP.
    • Set timeouts above the transport. What the caller experiences is your retry policy (Timeouts, Retries, Exponential Backoff and Jitter), not TCP's.

    Common mistakes

    • Treating TCP as message-oriented. It is a byte stream: one write can arrive as several reads.
    • Believing "reliable" means "never fails". TCP gives up after repeated retransmissions and reports an error.
    • Opening a connection per request. It pays the handshake each time and fills the host with TIME-WAIT state.
    • Assuming UDP datagrams arrive once and in order. Duplicates and reordering are normal.

    Key takeaways

    • TCP is a reliable, in-order byte stream; UDP is unreliable datagrams with no duplicate protection.
    • TCP costs a round trip to connect, state per connection and waiting: one lost segment delays all later data.
    • Cumulative ACKs, a window, a doubling timer and fast retransmit (three duplicate ACKs) repair loss; in the lab the same loss costs 550 ms with a timer and 250 ms with fast retransmit.
    • UDP shows the application loss, delay and duplication: use it when freshness matters more than completeness, or through QUIC.
    • Reuse connections: handshakes and TIME-WAIT are the hidden bill of connection-per-request designs.

    Practice exercises

    Exercise 1 (20 min). From the lab table, for scenarios 2, 3 and 6, write when TCP completes and which mechanism, timer or fast retransmit, explains each number.

    Exercise 2 (45 min). Run the lab and the "Break it" changes, then change scenario 3 from window 6 to window 3 and explain why recovery now waits for the timer.

    Exercise 3 (1 hour). For ShopStream's checkout, courier pings and push notifications, choose a transport for each and write the cost of your choice and what you would measure to check it.

    Official documentation

    Ask about this article

    Get answers grounded in this post. AI-generated — based on this article, and may be imperfect.

    Scaled AI Weekly

    Enjoyed this? Get more like it every Monday.

    Real architecture decisions, LLMOps patterns that survive production, and engineering leadership advice — from 12+ years of building at enterprise scale. Free. No spam. Unsubscribe anytime.

    Get new articles by email

    Free. We email a confirmation link first, and you can unsubscribe from any email.

    Join engineers building production AI systems

    Comments