Load Balancing

Meridian distributes traffic across backend endpoints using one of several algorithms. The load balancer runs on every request and selects one healthy endpoint to receive the request.

Algorithms

Round-Robin

Cycles through healthy endpoints sequentially. Each request goes to the next endpoint. Simple, predictable, and fast (~0.7ns per pick).

lb_policy = "round_robin"

The round-robin counter uses AtomicU32 with Relaxed ordering. Each worker advances the counter independently — this naturally distributes load across workers without synchronization overhead.

Least Request (Power of Two Choices)

Picks two random endpoints and sends the request to the one with fewer active requests. This provides near-optimal load distribution with O(1) selection and is the best choice when backends have varying response times.

lb_policy = "least_request"

Active request counts are tracked via AtomicU32 per endpoint. The load balancer reads these with Relaxed ordering — approximate counts are sufficient for good load distribution.

Maglev (Consistent Hashing)

Maps a hash key to an endpoint with minimal disruption when endpoints are added or removed. Ideal for caching or session-sticky routing.

lb_policy = "maglev"

The Maglev table is a precomputed lookup table (65,537 entries) built once on config load. Picks are a single table lookup (~1.4ns).

Health-Aware Selection

All load balancers skip unhealthy endpoints. The Endpoint.healthy flag is an AtomicBool updated by the health checker task and read by the load balancer with no locking:

Health Checker ──[AtomicBool::store]──► Endpoint.healthy
                                              │
Load Balancer ──[AtomicBool::load]────────────┘

Performance