Virtual Threads in Spring Boot: What Changes and How to Use Them

@RestController
public class OrderController {

    private final OrderService orders;

    public OrderController(OrderService orders) {
        this.orders = orders;
    }

    // Blocking code — but now each request runs on a virtual thread,
    // so thousands of concurrent slow calls no longer exhaust the pool.
    @GetMapping("/orders/{id}")
    public Order get(@PathVariable String id) {
        return orders.fetchFromDbAndEnrich(id); // blocks on DB + an HTTP call
    }
}

Why this matters for AI apps specifically

LLM calls are the ultimate blocking I/O: a single request can wait seconds for a response. On platform threads, a few hundred concurrent LLM calls exhaust your pool and requests queue. On virtual threads, each in-flight LLM call parks cheaply, and you handle far more concurrency on the same hardware — without rewriting your Spring AI service in a reactive style. For a blocking ChatClient.call() workload, virtual threads are close to a free scalability win.

The pitfalls (read before you ship)

• Pinning on synchronized. If a virtual thread blocks inside a synchronized block/method, it "pins" its carrier thread and can't be parked — killing the benefit. Replace hot-path synchronized with ReentrantLock. (Newer JDKs reduce this, but audit your blocking sections.)
• Not a CPU-bound speedup. Virtual threads help I/O-bound concurrency. CPU-bound work is still limited by cores — they won't make computation faster.
• Don't pool them. Virtual threads are cheap to create; pooling them defeats the purpose. Use Executors.newVirtualThreadPerTaskExecutor(), not a fixed pool.
• Thread-locals. Heavy ThreadLocal usage across millions of threads can bloat memory; prefer scoped values or pass context explicitly.
• Downstream limits still apply. Your database connection pool (e.g. HikariCP) is still finite — virtual threads let you accept more concurrent work, but a 10-connection pool is still a 10-connection bottleneck. Size downstream resources accordingly.

When to use them

• ✅ I/O-bound web apps — DB queries, HTTP/microservice calls, LLM calls. This is the sweet spot.
• ✅ Apps you'd otherwise consider rewriting reactively just for throughput.
• ⚠️ CPU-bound services — little benefit; focus on parallelism/algorithms instead.
• ⚠️ Codebases with heavy synchronized on hot paths — fix pinning first.

FAQ

Do I need to rewrite my code? No — that's the whole point. Keep blocking, imperative code; flip the property. Just audit synchronized hot paths for pinning.

Virtual threads or WebFlux? For most teams, virtual threads give you reactive-like scalability with simple blocking code and easier debugging. Choose WebFlux when you need true streaming/backpressure semantics end to end.

Will this speed up a single request? No. It improves concurrency (how many requests you handle at once), not the latency of one request.

What Spring Boot / Java version do I need? Spring Boot 3.2+ for the auto-configuration property, running on Java 21+ (the LTS that finalized virtual threads).

Virtual threads are the rare "free lunch": keep your simple blocking Spring Boot code, add one property, and scale I/O-bound workloads — including slow LLM calls — dramatically further. Just mind the pinning and size your downstream pools.

More: the Spring AI overview, streaming LLM responses in Spring AI, and AI system resilience.

Scaling a Spring Boot AI service and want an architecture review? Let's talk.