Table of Contents
Post-Mortem: Socket Exhauster on Node 04
java.lang.OutOfMemoryError: Thread creation failed: Operating system resource exhausted
at java.base/java.lang.Thread.start(Thread.java:1539)
at java.base/java.lang.VirtualThread.start(VirtualThread.java:307)
at com.fintech.ledger.io.SocketWorkerPool.dispatch(SocketWorkerPool.java:88)
at com.fintech.ledger.GatewayServer.lambda$acceptConnections$3(GatewayServer.java:142)
at java.base/jdk.internal.vm.Continuation.enter(Continuation.java:340)
...
Caused by: java.io.UncheckedIOException: java.net.SocketException: Too many open files
at com.fintech.ledger.io.NativeEpollDispatcher.register(NativeEpollDispatcher.java:211)
It pinned. Hard. After three days of staring at raw kernel-bypass logs on Node 04, watching p99.9 latencies blow past four thousand milliseconds under a modest fifty thousand requests per second, the illusion evaporated. Everyone read the Project Loom papers, swallowed the marketing gloss about lightweight concurrency, and assumed they could swap out thread pools like changing socks. They cannot. When you run benchmarking java 21 workloads under realistic production pressure—combining legacy blocking JDBC drivers, bloated Netty pipelines, and default fork-join schedulers—virtual threads do not magically fix your I/O bottlenecks. They expose them.
The core thesis here is as blunt as a brick: Virtual threads are a scheduling abstraction, not a hardware upgrade. If your payload relies on synchronized blocks, native JNI calls, or blocking file channels, you are not utilizing cooperative scheduling. You are pinning carrier threads, starving the JVM’s ForkJoinPool, and creating garbage collection pauses that look less like latency spikes and more like rolling blackouts.
The Architecture of a Lie: Why Virtual Threads Pin
Let us look at what actually happens inside the JVM when a virtual thread encounters a blocking operation. Platform threads map one-to-one with OS threads. They cost roughly a megabyte of stack space per thread, forcing developers to use thread pools. Virtual threads decouple this. Millions of virtual threads multiplex onto a small number of carrier threads—typically equal to the number of available CPU cores. When a virtual thread executes blocking I/O, the JVM mounts the continuation, yields the carrier thread to another virtual thread, and unmounts the stack.
Except when it doesn’t.
Pinning occurs under two distinct conditions in Java 21:
- When code executes inside a
synchronizedblock or method. - When native methods or foreign function calls (FFM) are executed on the stack.
During pinning, the virtual thread cannot be unmounted from its carrier thread. If your payment gateway code wraps database calls in legacy Hibernate transactions adorned with synchronized monitors, your carrier thread stays nailed to that single virtual thread until the database responds. Scale that to twenty thousand concurrent requests hitting a degraded Postgres instance, and your entire carrier pool is exhausted. Your million virtual threads are sitting in the runnable queue, starving while the carrier threads sleep on a network socket.
Benchmarking Java 21: Setup, Harness, and Realistic Noise
We built a test harness to isolate this. No cloud-hosted micro-instances with noisy neighbors. We used bare-metal dual AMD EPYC 9654 nodes with 768GB of ECC RAM, running Linux kernel 6.5.0-custom, pinned to isolated NUMA nodes. The load generator ran on a separate physical switch to eliminate packet drops.
We used JMH (Java Microbenchmark Harness) combined with custom async profilers to measure latency distributions at high percentiles. Here is the configuration we used to drive the JMH harness:
@State(Scope.Benchmark)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@BenchmarkMode(Mode.Throughput)
public class VirtualThreadGatewayBenchmark {
private ExecutorService virtualExecutor;
private ExecutorService platformExecutor;
private SocketWorkerPool socketPool;
@Setup(Level.Trial)
public void setUp() {
this.virtualExecutor = Executors.newVirtualThreadPerTaskExecutor();
this.platformExecutor = Executors.newFixedThreadPool(64);
this.socketPool = new com.fintech.ledger.io.SocketWorkerPool(
8443,
Duration.ofMillis(1200)
);
}
@Benchmark
public void benchmarkVirtualWithBlockingIO(Blackhole bh) throws Exception {
virtualExecutor.submit(() -> {
byte[] payload = socketPool.readNextPacket();
bh.consume(payload);
}).get();
}
}
The results were ugly. When pure non-blocking NIO channels were used, virtual threads matched platform threads in throughput while consuming a fraction of the memory footprint. But the moment we introduced a standard JDBC connection pool wrapped in blocking wrappers—simulating a real enterprise database call—the tail latencies diverged catastrophically.
The Asynchronous I/O Trap and Native Thread Pools
Most enterprise applications do not use raw sockets. They use frameworks built on top of Netty or Jetty. These frameworks use their own event loops—native thread pools pinned to CPU cores using epoll or kqueue. When you bridge virtual threads into a framework designed around async event loops, you introduce context-switching overhead that defeats the purpose of the abstraction.
Consider the custom dispatcher implementation we had to write to prevent thread starvation in com.fintech.ledger.io.SocketWorkerPool:
package com.fintech.ledger.io;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.AsynchronousSocketChannel;
import java.time.Duration;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
public class NativeEpollDispatcher {
private final AsynchronousSocketChannel channel;
private final Duration timeout;
public NativeEpollDispatcher(String host, int port, Duration timeout) throws IOException {
this.channel = AsynchronousSocketChannel.open();
this.timeout = timeout;
this.channel.connect(new InetSocketAddress(host, port)).get();
}
public byte[] readPayloadAsync() throws Exception {
ByteBuffer buffer = ByteBuffer.allocate(4096);
CompletableFuture<Integer> future = new CompletableFuture<>();
channel.read(buffer, null, new java.nio.channels.CompletionHandler<Integer, Void>() {
@Override
public void completed(Integer result, Void attachment) {
future.complete(result);
}
@Override
public void failed(Throwable exc, Void attachment) {
future.completeExceptionally(exc);
}
});
try {
future.get(timeout.toMillis(), TimeUnit.MILLISECONDS);
buffer.flip();
byte[] data = new byte[buffer.remaining()];
buffer.get(data);
return data;
} catch (TimeoutException e) {
channel.close();
throw new IOException("Socket read timed out after " + timeout.toMillis() + "ms", e);
}
}
}
If you call future.get() inside a virtual thread, it yields correctly. But if that completion handler executes inside a synchronized block or triggers a native library call, the carrier thread locks up. Under load tests running at 50,000 RPS, even a 0.1% pinning rate results in hundreds of carrier threads locking up per second. The JVM’s internal periodic unblocker thread has to step in, scanning pinned continuations and attempting recovery. That recovery mechanism introduces CPU spikes that look like garbage collection thrashing, but are actually thread scheduling stalls.
Garbage Collection Pressure and Continuation Allocation
People forget that virtual threads are still Java objects (`java.lang.VirtualThread`). Every time you execute `Thread.start()` or submit a task to a virtual thread executor, the JVM allocates a continuation object, a stack chunk, and associated scheduler references. Under high throughput, this creates massive churn in the Young Generation.
We tracked allocation rates using GC logs with `-Xlog:gc*,safepoint=info`. When using platform thread pools with a fixed pool size of 200 threads, memory allocations were steady and predictable. Moving to unconstrained virtual threads without a custom bounding semaphore resulted in object creation rates exceeding 45 GB/sec. The G1 garbage collector spent 12% of its CPU time in mixed GC cycles just cleaning up abandoned stack chunks and continuation frames.
To fix this, we had to abandon unbounded task submission and implement a strict permit-based limiter:
package com.fintech.ledger.io;
import java.util.concurrent.Semaphore;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.RejectedExecutionException;
public class BoundedVirtualThreadPool {
private final ExecutorService delegate;
private final Semaphore limiter;
public BoundedVirtualThreadPool(int maxConcurrentTasks) {
ThreadFactory factory = Thread.ofVirtual().name("fintech-worker-", 0).factory();
this.delegate = Executors.newThreadPerTaskExecutor(factory);
this.limiter = new Semaphore(maxConcurrentTasks);
}
public void execute(Runnable task) {
if (!limiter.tryAcquire()) {
throw new RejectedExecutionException("Rate limit exceeded: carrier pools saturated.");
}
delegate.submit(() -> {
try {
task.run();
} finally {
limiter.release();
}
});
}
}
Flame Graph Realities: Where the Time Goes
When you pull up Async Profiler flame graphs of a misconfigured Java 21 payment gateway, the hot spots do not lie in business logic. They live in three specific places:
- `jdk.internal.vm.Continuation.yield` taking up 35% of CPU cycles due to excessive unmounting and remounting in tight polling loops.
- `java.util.concurrent.locks.ReentrantLock$Sync.lock` contention inside the ForkJoinPool worker queues when thousands of virtual threads wake up simultaneously upon receiving network packets.
- `sun.nio.ch.EPoll.epollWait` blocking native threads because asynchronous channel completion handlers are feeding back into synchronous thread pools.
If you want virtual threads to perform in high-throughput network applications, you have to audit every single dependency in your dependency tree. Replace synchronized with ReentrantLock or java.util.concurrent atomic variables. Strip out blocking JDBC drivers and move to reactive or true non-linear drivers like r2dbc, or isolate your blocking legacy calls into a separate, isolated platform thread pool using CompletableFuture.supplyAsync(task, legacyThreadPool).
Production Hardening Checklist
If you are deploying Java 21 virtual threads to production tomorrow, throw away the introductory blog posts and verify these constraints on your staging cluster:
- Run your benchmark tests with `-XX:+UnlockDiagnosticVMOptions -XX:+ShowHiddenFrames` to identify exact continuation pinning points in your logs.
- Never pass virtual threads to legacy thread pools expecting performance gains; if you must interface with blocking code, use explicit bulkheads.
- Monitor `jdk.internal.vm.Continuation` allocation rates in JFR (Java Flight Recorder); if allocation rates outpace your survivor spaces, you are spawning too many short-lived virtual tasks without a rate limiter.
- Tune your JVM’s target parallelism using `-Djdk.virtualThreadScheduler.parallelism=` to match your physical core count, preventing hyper-threaded core thrashing under heavy I/O loads.
Virtual threads are a powerful addition to the JVM, but they are not a substitute for architectural discipline. Treat them with the same suspicion you would treat any other low-level concurrency primitive, profile your tail latencies under peak load, and check your flame graphs before pushing to production.