Skip to content

    System Design

    Virtual Threads (Project Loom): A Thread per Task Without a Pool

    Learn what Java virtual threads change, why you create one per task and do not pool them, how a Semaphore limits load, what pinning is, and test it in a lab.

    By Published 15 min read

    Share

    Standard lessonintermediateJava labBackend System Design

    Before this: Thread Pools, Executors and CompletableFuture

    What you'll learn: By the end of this lesson you will be able to explain what a virtual thread is and how it differs from a platform thread, say why the advice is one virtual thread per task and never a pool of them, limit load on a scarce resource with a Semaphore, describe what pinning is and what the Java 25 documentation lists as causing it, name what virtual threads do not fix (CPU-bound work, downstream capacity), and use a Java lab that shows these points with counters and yes or no answers instead of timings.

    Picture two waiters. The first takes an order to the kitchen hatch and stands there until the dish is ready; while she waits she can serve nobody else, so a restaurant with ten waiters can have ten orders in progress at most. The second leaves the order at the hatch and goes back to serving tables; when the dish is ready, any free waiter carries it out. The second restaurant gets far more orders in progress from the same staff, but it does not cook any faster, and if the kitchen can only prepare three dishes at once, more waiters do not help.

    Platform threads behave like the first waiter: a thread that blocks keeps its operating-system thread. Virtual threads behave like the second. The previous lesson showed how pools bound the cost of threads that block. This lesson is about a thread so cheap that you stop pooling it.

    Verified with the Java SE 25 Core Libraries guide on virtual threads, the Thread, Executors and ExecutorService API documentation, and a lab run with Java 25, on 2026-11-19. The JDK Enhancement Proposal pages returned an access error when fetched, so nothing here is cited to them.

    The core idea

    A virtual thread is a java.lang.Thread that the Java runtime, not the operating system, schedules onto a small set of platform threads called carriers. When it blocks, it can step off its carrier, so thousands of blocked tasks do not need thousands of operating-system threads. The Core Libraries guide puts it this way: "a virtual thread isn't tied to a specific OS thread", and "we can easily have a great many active virtual threads, even millions". The same guide is clear about the limit: virtual threads "are not faster threads; they do not run code any faster than platform threads. They exist to provide scale (higher throughput), not speed (lower latency)."

    Three illustrations run through the lesson, each labelled:

    • Minimal example: 100 tasks that all block, run on a pool of four platform threads and on one virtual thread per task, in the Java lab.
    • Realistic example: ShopStream's order-status page calling several services per request, in "In practice".
    • Failure example: unlimited virtual threads all waiting on a downstream system that can serve only a few at a time.

    What a virtual thread is

    The Thread API documentation describes virtual threads as "typically user-mode threads scheduled by the Java runtime rather than the operating system", which "will typically require few resources", so that a single JVM "may support millions". They suit "tasks that spend most of the time blocked, often waiting for I/O operations to complete" and "are not intended for long running CPU intensive operations". A virtual thread is still a Thread: you create one with Thread.ofVirtual().start(runnable) or Thread.startVirtualThread, and Thread.currentThread() inside it "will always return the Thread object for the virtual thread", not the carrier.

    Property Platform thread Virtual thread (Thread API)
    Scheduled by The operating system The Java runtime, onto carrier threads
    Name by default An ordinary thread name None: getName() returns the empty string
    Daemon Can be set Always a daemon; cannot be changed
    Priority Settable Always NORM_PRIORITY; setPriority is ignored
    Thread group An ordinary thread group A special group for virtual threads
    Meant for Anything, including CPU-bound work Tasks that mostly wait

    The lab's Part 1 prints these facts from the running JVM next to the main thread, so you can see them rather than take them on trust.

    Mounting and unmounting

    When the runtime runs a virtual thread it mounts it on a carrier. The Core Libraries guide says it "can unmount from its carrier. This usually happens when the virtual thread performs a blocking I/O operation. After a virtual thread unmounts from its carrier, the carrier is free, which means that the Java runtime scheduler can mount a different virtual thread on it."

    sequenceDiagram accTitle: A virtual thread unmounts while it waits accDescr: Virtual thread A runs on the only carrier and blocks on a slow call, so it unmounts. The carrier is free and runs virtual thread B. When the slow call returns, A is mounted on a carrier again and continues. participant A as Virtual thread A participant K as Carrier thread participant B as Virtual thread B participant S as Slow call A->>K: mounted, runs A->>S: blocks on the call Note over A,K: A unmounts, the carrier is free B->>K: mounted, runs S-->>A: reply is ready A->>K: mounted again, continues

    Figure 1. Blocking releases the carrier. How to read this: the carrier is the scarce thing. While A waits for the slow call it holds no carrier, so B runs on it. The code in A does not change; it still reads as a blocking call.

    The scheduler is configurable in the JDK reference implementation. The Thread documentation names two system properties: jdk.virtualThreadScheduler.parallelism, "the number of platform threads available for scheduling virtual threads", which "defaults to the number of available processors", and jdk.virtualThreadScheduler.maxPoolSize, which "defaults to 256". They are read when the JVM starts, which is why the lab sets them in its build file.

    One virtual thread per task, never a pool

    A platform-thread pool exists because platform threads are expensive and must be shared. The guide's advice follows from the opposite fact: "virtual threads are plentiful, and so each should represent not some shared, pooled, resource but a task." It adds that "the number of virtual threads is always equal to the number of concurrent tasks in your application", and gives a rule of thumb: "if your application never has 10,000 virtual threads or more, it is unlikely to benefit from virtual threads."

    The idiom is Executors.newVirtualThreadPerTaskExecutor(), which "starts a new virtual Thread for each task" and whose thread count is "unbounded". It is an ExecutorService, so the code that submits tasks does not change, and since Java 19 ExecutorService.close() waits for the submitted tasks, so a try-with-resources block ends only when they are done. The guide says the executor "doesn't employ a thread pool".

    That unbounded count is the catch. A pool's size used to double as a limit on concurrency: 16 threads meant at most 16 calls in flight to a downstream. With one thread per task, nothing limits the calls. The guide says what to use instead: "if you want to limit the concurrency of accessing some service, you should use a construct designed specifically for that purpose: the Semaphore class." It adds that "database connection pools themselves serve as a semaphore". The limit moves from the number of threads to the resource that is actually scarce.

    Many virtual threads, few carriers, one limit at the scarce resourceVirtual threads to Semaphore (acquire); Semaphore to Downstream (at most N); Carriers to Virtual threads (run them)Virtual threads: one per taskVirtualthreadsone per taskSemaphore: N permitsSemaphoreN permitsDownstream: serves a few at a timeDownstreamserves a few at atimeCarriers: few platform threadsCarriersfew platformthreadsacquireat most Nrun them

    Figure 2. The limit moves to the resource. How to read this: thousands of tasks can exist at once, and a few carriers run whichever of them is ready. The semaphore, not the thread count, keeps the downstream from being flooded.

    Pinning: when blocking keeps the carrier

    A virtual thread that cannot unmount while it blocks is pinned to its carrier. The Java 25 Core Libraries guide says: "A virtual thread cannot be unmounted during blocking operations when it is pinned to its carrier. A virtual thread is pinned when it runs a native method or a foreign function. Pinning does not make an application incorrect, but it might hinder its scalability." It documents a Java Flight Recorder event for finding it: jdk.VirtualThreadPinned "indicates that a virtual thread was pinned (and its carrier thread wasn't freed) for longer than the threshold duration", enabled by default with a threshold of 20 ms.

    Notice what that guide lists: native methods and foreign functions. Some material written for earlier JDK releases also names blocking inside a synchronized block as a cause of pinning, and recommends ReentrantLock instead. The documents this lesson could read for Java 25 do not list it, so the lab asks the question directly on the JDK it runs. It limits the scheduler to one carrier, blocks one virtual thread in three different ways, and tries to run a second virtual thread. A second thread can only run if the first released the only carrier.

    Pinning matters where code calls native methods or foreign functions that block. It is a scalability problem, not a correctness one, and the JFR event is how to see it in a running service.

    What virtual threads do not fix

    • CPU-bound work. Both the guide and the Thread documentation say virtual threads are not intended for long CPU-intensive operations. Cores are the limit; use a bounded pool near the core count.
    • Downstream capacity. More concurrent tasks means more concurrent calls. If the database accepts a fixed number of connections, thousands of virtual threads simply wait for them.
    • Latency of a single call. It is the same call.
    • Thread-local caching. The guide says caching reusable objects in thread-locals "is fundamentally at odds with virtual threads", because "virtual threads are never pooled and never reused by unrelated tasks": each task would build its own copy. Share an immutable object instead.
    • Observability habits. jcmd <PID> Thread.dump_to_file -format=text <file> lists "all threads, including both platform and virtual threads"; a plain stack dump of platform threads will not show the virtual ones.

    Build it: counting blocked tasks, limiting with a Semaphore, probing pinning

    This is the minimal example. Timing comparisons of threads are easy to get wrong, so the lab prints only counters and yes or no answers.

    Goal. See how many blocked tasks each executor can hold at once, that a Semaphore bounds concurrency without bounding threads, and whether a blocked virtual thread frees the carrier in three cases.

    Setup. From the repository's examples/backend folder:

    ./gradlew :labs:virtual-threads-loom:check --console=plain
    ./gradlew -q :labs:virtual-threads-loom:run

    The first command runs 14 tests; the second prints the four parts below. Run it through Gradle: the build file passes the two scheduler properties that give the JVM a single carrier thread. The code is in examples/backend/labs/virtual-threads-loom/src/main/java/buildingai/labs/virtualthreads/.

    How it works. BlockedTasks submits 100 tasks that each count themselves and wait on one latch, a stand-in for a slow call, and opens the latch only after the number of blocked tasks has reached the most the executor can structurally hold: 4 for a fixed pool of 4, 100 for one virtual thread per task. ConcurrencyLimit guards a section with Semaphore(3). MonitorProbe blocks one virtual thread inside synchronized, inside a ReentrantLock and while waiting to enter a monitor a platform thread holds, then asks whether a second virtual thread can run on the single carrier. The creation of the virtual-thread executor in the lab is the one-liner:

    ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();

    Expected output, captured from a real run (Java 25):

    Virtual threads lab. Every number below is a counter or a yes/no answer; no timings are printed.
    Scheduler: jdk.virtualThreadScheduler.parallelism=1, jdk.virtualThreadScheduler.maxPoolSize=1 (set by the build file: one carrier thread).
    
    Part 1. What the Thread API says about a virtual thread (compared with the main thread).
    thread           isVirtual  daemon  priority thread group
    ----------------------------------------------------------------
    main (platform)      false   false         5 main
    virtual thread        true    true         5 VirtualThreads
    A virtual thread has no name unless you give one: getName() returns "".
    
    Part 2. 100 tasks, each blocks on one gate (a stand-in for a slow call). How many are blocked at the same moment?
    executor                   submitted blocked at once waiting in queue  completed scheduler threads
    ---------------------------------------------------------------------------------------------------
    platform pool of 4               100               4               96        100                 -
    virtual thread per task          100             100                0        100                 1
    'scheduler threads' counts live platform threads named ForkJoinPool-N-worker-N (the carriers; a JDK implementation detail); '-' means not measured.
    
    Part 3. 20 virtual threads, a Semaphore with 3 permits guards the scarce resource.
    tasks=20 permits=3 peak inside the guarded section=3 completed=20
    
    Part 4. With ONE carrier thread, can a second virtual thread run while the first is blocked?
    'yes' means the blocked virtual thread released the carrier.
    PARKED_INSIDE_SYNCHRONIZED       -> yes
    PARKED_INSIDE_REENTRANT_LOCK     -> yes
    WAITING_TO_ENTER_MONITOR         -> yes

    What the output shows. Part 1 matches the table above: virtual, daemon, priority 5, a special thread group and no name. Part 2 is the lesson in two rows: the platform pool of four holds four blocked tasks and leaves 96 waiting, while one virtual thread per task holds all 100 blocked at once, on a scheduler that has a single carrier thread. Part 3 shows 20 tasks, 3 permits and a peak of 3: the limit is the semaphore. Part 4 shows that on the JDK that ran the lab, all three blocking cases released the single carrier. The scheduler-thread count of 1 is a JDK implementation detail the lab reads by thread name, not an API.

    Break it. In BlockedTasks.onPlatformPool, change the pool size from 4 to 100: the platform pool also reaches 100 blocked at once, because each task now owns an operating-system thread. The point is the cost per blocked task, not that platform threads cannot do it. Put the 4 back and run the tests again.

    Stretch. Add a task that calls a native or foreign function and blocks, and look for jdk.VirtualThreadPinned in a Flight Recorder file. Run Part 4 on another JDK release and compare; this lab did not.

    What the model leaves out. Real I/O, timing and memory per thread, and any JDK other than the one that ran the lab.

    Choosing: virtual threads or a pool

    flowchart TD accTitle: Choosing between virtual threads and a bounded pool accDescr: A decision tree. If the task is CPU-bound, use a bounded pool near the core count. If it mostly waits, use one virtual thread per task and put a Semaphore or the connection pool in front of each scarce downstream. If it calls native code that blocks, check for pinning first. A{What does the task mostly do?} A -->|computes| B[Bounded pool near the core count] A -->|waits on I/O| C[One virtual thread per task] C --> D[Limit each scarce downstream with a Semaphore or its pool] C --> E{Does it call blocking native code?} E -->|yes| F[Watch jdk.VirtualThreadPinned, consider a bounded pool for that call] E -->|no| G[Keep the code in the blocking style]

    Figure 3. A first-pass choice. How to read this: the first question is what the task spends its time on. Waiting work gets a virtual thread per task, and every scarce downstream still gets its own limit.

    Option Gain Cost Wrong when
    Bounded platform pool Natural limit on concurrency; fine for CPU work A blocked task keeps its thread, so waiting work needs many threads Most of the time is waiting on I/O
    Virtual thread per task Blocking-style code with many tasks in flight; no pool to size No built-in concurrency limit The task is CPU-bound, or the downstream is not limited
    Virtual threads plus a Semaphore Limit set at the real bottleneck A permit to acquire and release correctly Nothing is scarce downstream
    Asynchronous pipelines (CompletableFuture) Control over stages and executors Code is harder to read and debug A simple blocking flow would do

    A separate planned lesson compares this style with reactive frameworks; it has no page yet.

    In practice: ShopStream's order-status page

    This is the realistic example, an educational scenario on the course's running example ShopStream; nothing in it describes a real company. Every figure is illustrative.

    This is a realistic scenario, not a case history.

    The situation

    The order-status page shows the order, the courier's position, the payment state and the store's note. Each page view calls four internal services in turn. The service uses a bounded pool of 64 platform threads, and at the evening peak requests queue behind it even though the CPU is mostly idle: the threads are waiting for the four calls. Someone proposes adding more threads or rewriting the calls as asynchronous pipelines.

    Starting architecture

    Order-status service: a bounded pool in front of four blocking callsAPI gateway to Request pool (requests); Request pool to Order service; Request pool to Payment service; Request pool to Courier serviceAPI gatewayAPI gatewayRequest pool: 64 platform threadsRequest pool64 platformthreadsOrder serviceOrder servicePayment servicePaymentserviceCourier serviceCourierservicerequests

    Figure 4. Threads wait while the CPU idles. How to read this: every request holds one of 64 threads while it waits for the services on the right. The pool's size is the cap on concurrent requests, and it is set by waiting, not by work.

    Options considered

    Option Behaviour Risk
    A. More platform threads A bigger pool Each blocked request still costs an operating-system thread
    B. Asynchronous rewrite Fewer threads in use Large change; harder code to read and debug
    C. Virtual thread per request, a Semaphore per downstream Same blocking code, many requests in flight, limit set where capacity is Without the limits, concurrency is unbounded

    The decision

    Option C, in that order: first write down the limit each downstream can take (the payment service accepts a fixed number of concurrent calls, the database has a fixed connection pool), then switch the request executor to one virtual thread per task, then add a Semaphore in front of each limited call with a timeout on acquiring it. The change to the request handlers is small because they keep their blocking style.

    Rollout and verification

    The team uses the lab as the test model: assert counters, not times. A test blocks N tasks on a latch and checks that all N are in flight; another checks that the peak inside a guarded call never exceeds the permits. In production they watch the number of in-flight requests, permit waits per downstream, the downstream's own saturation, and the jdk.VirtualThreadPinned events. A thread dump with jcmd ... Thread.dump_to_file is the way to inspect virtual threads during an incident.

    The 2 a.m. failure

    The page reads: order-status errors rise and the payment service is slow for everyone. A deploy moved the payment call into a new client that was not behind a semaphore. With platform threads, the pool of 64 had capped the calls in flight; with virtual threads, thousands of requests reached payment at once, and the pool of connections it owns filled, so every call waited. The checks: in-flight requests, the payment service's saturation, and a thread dump to see where the virtual threads wait. The mitigation is a semaphore at the payment client, sized to what payment accepts, with a timeout and a fast failure when no permit is free. The lasting fixes are a rule that every client of a scarce resource passes through a limit, a load test that exceeds the limit on purpose, and an alert on permit wait time.

    Architect's notes

    • Move the limit, do not remove it. Name the scarce resource and give it a Semaphore or its own pool.
    • Create threads per task, not per pool. Do not keep virtual threads in a pool or reuse them across tasks.
    • Keep CPU work on a bounded pool. Virtual threads give scale for waiting, not speed for computing.
    • Look at pinning with the JFR event (jdk.VirtualThreadPinned), especially around native or foreign calls.
    • Check your own JDK. Older material on pinning can differ from the Java 25 documentation; test the cases that matter on the release you run.
    • Share immutable objects instead of caching per-thread copies in thread-locals.

    Common mistakes

    • Pooling virtual threads to "control" them; use a semaphore for the limit.
    • Removing the only concurrency limit by switching executors and not adding one at the downstream.
    • Using virtual threads for CPU-bound work and expecting a speed-up.
    • Caching expensive objects in ThreadLocal and creating one per task.
    • Assuming pinning is only about locks. The guide lists native methods and foreign functions.
    • Measuring with a stopwatch in a test; assert counters instead.

    Key takeaways

    • A virtual thread is a Thread scheduled by the Java runtime onto carrier threads; blocking usually unmounts it.
    • Use one virtual thread per task (newVirtualThreadPerTaskExecutor); do not pool them.
    • The pool size used to limit concurrency; now a Semaphore or the downstream's own pool must.
    • Virtual threads are always daemon, have priority NORM_PRIORITY and no default name.
    • Pinning keeps the carrier; the Java 25 guide lists native methods and foreign functions, and jdk.VirtualThreadPinned reports it.
    • They give scale, not speed, and do not help CPU-bound work.

    Practice exercises

    Exercise 1 (20 min). For a service with 50 concurrent database connections and tasks that each use one, say where you put the limit if tasks run on virtual threads.

    Exercise 2 (45 min). Run the lab and explain Part 2 in terms of carriers, not threads. Then change the pool size in "Break it" and explain the new row.

    Exercise 3 (1 hour). Remove the acquire and release calls in ConcurrencyLimit as the lab's README describes, predict the range of the peak, run it, and put the code back.

    Exercise 4 (2 hours). For ShopStream's checkout service, list its downstream calls, give each a limit and a timeout, and say which ones would need a bounded platform pool instead.

    Official documentation

    Ask about this article

    Get answers grounded in this post. AI-generated — based on this article, and may be imperfect.

    Scaled AI Weekly

    Enjoyed this? Get more like it every Monday.

    Real architecture decisions, LLMOps patterns that survive production, and engineering leadership advice — from 12+ years of building at enterprise scale. Free. No spam. Unsubscribe anytime.

    Get new articles by email

    Free. We email a confirmation link first, and you can unsubscribe from any email.

    Join engineers building production AI systems

    Comments