Connection Pooling
Meridian maintains a pool of idle TCP connections to upstream endpoints. Instead of performing a TCP handshake for every request (~0.5-1ms), the proxy reuses existing connections.
How It Works
Request 1: pool empty → TCP connect → use → checkin to pool
Request 2: pool has 1 → checkout → use → checkin to pool
Request 3: pool has 1 → checkout → use → checkin to pool
...
The first request to an endpoint creates a new connection. Subsequent requests reuse pooled connections. The TCP handshake cost is paid once, not per-request.
Pool Configuration
[[clusters]]
name = "backend"
max_idle_connections = 8 # per endpoint, default 8
Pool Behavior
| Property | Value |
|---|---|
| Max idle per endpoint | Configurable (default 8) |
| Max connection age | 90 seconds |
| Eviction strategy | Oldest evicted when pool is full |
| Checkout order | LIFO (most recently returned = warmest) |
| Thread safety | Mutex<HashMap> (held only for push/pop, no async under lock) |
Keep-Alive Detection
After receiving the upstream response, the proxy checks the Connection header before stripping hop-by-hop headers:
Connection: close→ connection is dropped (not pooled)Connection: keep-aliveor absent (HTTP/1.1 default) → connection is returned to the pool
Per-Cluster Pools
Each cluster has its own ConnectionPool instance, shared across all connection handler tasks via Arc. This means:
- Pool slots for cluster A don’t compete with cluster B
- Circuit breaker and pool are co-located on the same
ClusterState - Pool metrics can be reported per-cluster