Poise documentation

Poise is a family of composable, runtime-independent load-balancing primitives for Rust. It is designed for engineers who need to explain why a backend was eligible, why a policy selected it, what happened during dispatch, and how the result changes future decisions.

The library does not treat load balancing as one algorithm. It models a control loop:

discovery → membership → eligibility → selection → dispatch → feedback
                               ↑                        │
                               └── health and load ─────┘

Each arrow is a contract. Keeping those contracts separate lets applications change discovery, policy, runtime, health strategy, or telemetry without replacing an opaque all-in-one balancer.

What Poise guarantees

Poise makes narrow guarantees that compose:

• a successful selection from a supported in-repository policy always identifies an eligible, in-bounds candidate;
• empty membership and non-empty but ineligible membership remain distinct;
• seeded stochastic policies replay exactly;
• keyed policies use documented deterministic hashing;
• cached membership state is rebuilt transactionally;
• Tower readiness is retained between selection and dispatch;
• completion and cancellation update load through RAII guards;
• discovery readers observe immutable, monotonically versioned snapshots;
• built-in metrics have fixed cardinality independent of backend and request diversity.

Those are behavioral contracts rather than aspirations. The test surface combines examples, generated laws, mutation testing, and exhaustive scheduler models. The live verification record links every published result to the GitHub Actions run that produced it.

What Poise does not own

Poise is not a reverse proxy, HTTP client, service mesh, DNS resolver, retry engine, or orchestrator SDK. It does not spawn a runtime from the core, own connections, select retry semantics, or create unbounded telemetry labels.

That restraint is useful. A proxy can use Poise without adopting its protocol stack. A library can expose policies without imposing Tokio. A control plane can publish snapshots without owning dispatch.

How to read this book

If you are evaluating the crate, begin with Choose a policy and Compose the system. They explain the decision surface and where each crate belongs.

If you are integrating Poise, read Failure semantics, Tower dispatch, and Operating Poise before adding retries or health automation.

If you are changing a policy, its focused contract and the testing strategy are part of the public API. Arithmetic, hashing, membership invalidation, and concurrency changes require evidence at the strongest applicable verification layer.

Maturity

Poise is pre-1.0. Its contracts are deliberately explicit and heavily tested, but new capabilities can still reshape APIs. All six workspace crates share one version and one release process. Versioned releases are available on crates.io; review the changelog before upgrading within the pre-1.0 series.

Documentation conventions

The book uses four kinds of statements:

• Invariant — behavior callers may rely on.
• Tradeoff — a cost or limitation that influences policy choice.
• Operational rule — a condition production integrations should enforce.
• Non-goal — behavior intentionally left to the application.

Examples prefer concrete failure handling over happy-path-only snippets. Complexity statements describe the current implementation, not an imagined future optimization.

Support the project

If Poise is useful to your infrastructure, support continued engineering and maintenance through TokenTip.

Getting started

Poise is split into six crates so the deterministic policy core does not pull an async runtime, Tower, tracing, or a discovery implementation into every build.

Choose only the layers you need

There is no umbrella crate. This is intentional dependency hygiene, not an unfinished convenience API.

Install from crates.io

Add only the layers your application uses. All Poise crates share one version:

[dependencies]
poise-core = "0.1"

For unreleased development work, pin a reviewed Git revision rather than a moving branch:

[dependencies]
poise-core = { git = "https://github.com/copyleftdev/poise-rs", rev = "<reviewed-40-character-commit>" }

For local workspace integration:

[dependencies]
poise-core = { path = "../poise-rs/crates/poise-core" }

Replace the Git placeholder with a full commit hash you have reviewed. Do not depend on an unpinned branch in a release build. Poise is pre-1.0, so review the changelog before upgrading.

Make a first selection

#![allow(unused)]
fn main() {
use poise_core::{Backend, Policy, policy::RoundRobin};

let backends = [Backend::new("alpha"), Backend::new("beta")];
let mut policy = RoundRobin::new();

let selection = policy.pick(&backends, &())?;
let backend = &backends[selection.index()];

assert_eq!(backend.id(), &"alpha");
Ok::<(), poise_core::PickError>(())
}

Policy::pick returns an index rather than cloning or borrowing a backend. The caller retains ownership of membership and can attach protocol-specific state outside the policy.

Express eligibility and capacity

#![allow(unused)]
fn main() {
use poise_core::{Backend, Policy, Status, Weight, policy::WeightedRandom};

let backends = [
    Backend::new("large").with_weight(Weight::new(4)?),
    Backend::new("small").with_weight(Weight::new(1)?),
    Backend::new("draining").with_status(Status::Draining),
];
let mut policy = WeightedRandom::seeded(7);

for _ in 0..100 {
    let selected = policy.pick(&backends, &())?;
    assert_ne!(backends[selected.index()].id(), &"draining");
}
Ok::<(), Box<dyn std::error::Error>>(())
}

Weights are nonzero integers. Status is explicit. Every general policy excludes draining and unavailable candidates, while topology panic behavior documents its narrower exceptions.

Handle the no-candidate cases

#![allow(unused)]
fn main() {
use poise_core::{Backend, PickError, Policy, Status, policy::RoundRobin};

let mut policy = RoundRobin::new();
let empty: [Backend<&str>; 0] = [];
assert_eq!(policy.pick(&empty, &()), Err(PickError::Empty));

let unavailable = [Backend::new("alpha").with_status(Status::Unavailable)];
assert_eq!(
    policy.pick(&unavailable, &()),
    Err(PickError::NoEligibleCandidates)
);
}

Do not collapse these errors at the policy boundary:

• Empty usually means discovery has no membership.
• NoEligibleCandidates means membership exists but health, draining, or operator state excluded it.

Those conditions often deserve different retry, fallback, and alert behavior.

Make stochastic behavior reproducible

Randomized policies offer seeded constructors and caller-provided RNGs. Production systems may seed from entropy; tests and simulations should use a recorded seed:

#![allow(unused)]
fn main() {
use poise_core::{Backend, Policy, policy::Random};

let backends = [Backend::new("a"), Backend::new("b"), Backend::new("c")];
let mut left = Random::seeded(0x5eed);
let mut right = Random::seeded(0x5eed);

for _ in 0..32 {
    assert_eq!(left.pick(&backends, &())?, right.pick(&backends, &())?);
}
Ok::<(), poise_core::PickError>(())
}

Reproducibility matters for incident reconstruction, property tests, and simulation comparisons. It does not make separate policy instances share state.

Continue from here

• Choose a policy compares selection goals and costs.
• Compose the system places health, discovery, Tower, and observation around selection.
• Failure semantics explains errors and lifecycle outcomes that should remain distinct.

Choose a policy

Policy choice begins with the behavior your traffic needs to preserve. “Evenly distributed” is not a complete requirement: distribution, affinity, load response, membership churn, topology, and state all pull in different directions.

Decision table

This table identifies a starting point. Validate the exact candidate count, membership churn, weight distribution, and request-key distribution you expect.

Start from the invariant

“Every eligible backend should take turns”

Use RoundRobin when configured weights and live load do not matter. It is easy to reason about and exposes ordering mistakes quickly.

Use SmoothWeightedRoundRobin when an endpoint with weight 5 should receive exactly five selections for every one selection received by weight 1 over a complete cycle, without sending the five selections as one burst.

“Capacity should shape traffic”

Use WeightedRandom when candidate membership changes frequently and an O(n) scan is acceptable. It builds no alias table and therefore has no rebuild lifecycle.

Weights are ratios, not request limits. A 4:1 configuration does not prevent the larger backend from being overloaded, and it does not reserve four concrete permits.

“Current work should shape traffic”

Use LeastLoaded when the load metric is meaningful across every candidate and sampling all candidates is affordable. Equal-load ties rotate to avoid permanently favoring the first slice entry.

Use PowerOfTwoChoices when comparing two samples better matches the desired control behavior. The policy needs a LoadMetric, not a particular tracker. InFlight, PeakEwma, and application-owned metrics can all participate.

Load selection is observational. If capacity must be enforced atomically, pair selection with InFlight::try_acquire or another admission boundary.

“A key should stay with its owner”

Use rendezvous hashing for direct, table-free affinity. Removing a backend can only move keys owned by that backend; adding a backend can only preserve the existing owner or move the key to the new backend.

Use ring hash when integrations require continuum behavior or tunable points. Use Maglev when a stable membership set serves enough keyed lookups to amortize table construction.

Use bounded-load rendezvous when hot keys must spill away from their affinity owner. It preserves the unconstrained owner in decision metadata so operators can distinguish ordinary ownership from capacity-driven spillover.

“Failure domains should shape traffic”

Priority and locality are scope selection, not decorations on endpoint weight. The policy first decides which priority is usable, then which locality receives traffic, then which endpoint wins inside that locality.

Do not flatten priority, locality, and endpoint capacity into one number. That loses the ability to explain failover and creates surprising behavior during partial outages.

Questions to answer before production

1. What identity remains stable across discovery updates?
2. Does candidate order remain stable, or can it be reconstructed by identity?
3. Are weights capacity ratios, commercial allocations, or emergency knobs?
4. Is the load signal comparable across processes and generations?
5. Does a request need affinity, and what disruption is acceptable on churn?
6. Which health states may panic routing revive, if any?
7. Is selection followed by an atomic capacity reservation?
8. What decision metadata must reach traces or incident logs?

If these answers are unclear, choose the simplest policy and instrument it before adding affinity or adaptive behavior.

Common mismatches

• Round robin over unstable ordering: membership reorder becomes traffic movement even when identities did not change.
• Affinity for mutable keys: a key derived from timestamps, random IDs, or noncanonical encodings defeats stickiness.
• Weights as hard limits: weights influence proportion; they do not enforce admission.
• Peak EWMA without cancellation: abandoned work appears completed and distorts the estimator.
• Panic as universal revival: draining and operator opt-out must remain excluded even during a health panic.
• Metrics as policy state: asynchronous exporter state is not a coherent hot-path load signal.

The focused contract chapters document each family’s exact arithmetic and membership behavior.

Compose the system

A production balancer is a pipeline of ownership boundaries. Poise works best when each boundary has one job and communicates through explicit state.

control plane                                      data plane

source → Directory → Snapshot ──┐
                                ├→ candidate view → Policy → Tower service
probe  → health state ──────────┤                         │
result → circuit/load/metrics ──┘←────────────────────────┘

The five-stage path

1. Publish coherent membership

poise-discovery::Directory is a single-writer reconciliation boundary. Apply a batch, obtain one new revision, then publish an immutable Snapshot. Readers keep older snapshots alive safely while the new generation becomes visible atomically.

Treat discovery as coalesced state, not an event log. A slow subscriber may move directly from revision 10 to revision 14; revision 14 must contain the complete coherent membership it needs.

2. Derive eligibility

Candidate eligibility should combine independent signals without losing their origins:

• administrative state: ready, draining, unavailable;
• active probe classification;
• passive circuit permission;
• group-relative outlier decision;
• topology scope.

HealthChecked composes health with an existing candidate. It does not mutate the underlying administrative state. Draining remains draining even if health is otherwise good.

3. Select without dispatching

A policy sees a coherent candidate slice and optional request context. It returns an index and, for advanced families, structured decision metadata.

Selection must not consume a Tower readiness permit, increment a load tracker, or start network work. Those effects belong to dispatch.

Selection may observe capacity when a policy’s contract calls for it. After a candidate is selected, the dispatcher must acquire any atomic capacity guard; that admission is the race-safe check. If acquisition loses a race, reselect or return overload according to the application’s bounded retry budget.

4. Retain readiness through dispatch

poise-tower::Balance polls endpoint readiness before selection. A ready endpoint retains its service reservation. call lets the policy choose only among currently ready candidates and consumes exactly the selected reservation.

This ordering avoids the classic Tower bug in which a balancer selects a service, drops the readiness permit, then calls a service whose capacity has already changed.

5. Feed back a classified outcome

The response future owns the selected endpoint’s load guard:

• a returned success completes it;
• a returned service error also completes it;
• dropping the pending future records cancellation.

Protocol code then maps the result into Poise’s portable Outcome classes for passive health and observation.

Reference deployment shape

                         ┌───────────────┐
DNS / xDS / config ─────▶│ Directory     │
                         └──────┬────────┘
                                │ immutable revision
                         ┌──────▼────────┐
active probes ──────────▶│ candidate view│◀── passive circuit
                         └──────┬────────┘
                                │ eligible slice
                         ┌──────▼────────┐
request key ────────────▶│ Policy        │
                         └──────┬────────┘
                                │ Selection
                         ┌──────▼────────┐
                         │ Tower Balance │
                         └──────┬────────┘
                                │ response future + load guard
             metrics / tracing ◀┴▶ outcome window / circuit

The diagram is a topology, not a requirement that every stage be a separate task. Small systems can keep discovery static and use only poise-core plus poise-tower.

Choose the ownership boundary

The most important identity boundary is the service generation. Reusing a logical key with a newly allocated backend should build a new endpoint and load tracker; otherwise late results from the old service pollute the new service.

Snapshot reconciliation

With the discovery feature, poise-tower stages endpoint builds before committing a new pool:

1. reject stale revisions and duplicate live identities;
2. retain endpoints whose key and backend allocation are unchanged;
3. build all new service generations;
4. abort without changing the live pool if any build fails;
5. atomically replace ordering and membership;
6. allow physically retired endpoints to remain owned by outstanding futures.

This is transactional application, not merely transactional publication.

Context projection

Affinity policies need a stable key, but the policy should not own or clone the request. Implement RequestContext<Request> to borrow the routing field. UseRequest is available when the entire request value is already the key.

Canonicalize the key before it reaches a keyed policy. Equivalent user IDs with different case, Unicode normalization, or serialization must not hash as different identities unless that distinction is intentional.

Retrying correctly

Poise selects one attempt. Retry orchestration remains outside the policy. Before retrying, answer:

• whether the same affinity key should return to the same backend;
• whether the failed endpoint is excluded from the next attempt;
• whether the first attempt was cancelled or completed;
• whether capacity is released before the next selection;
• whether the retry consumes a shared budget.

Retrying blindly through the same deterministic affinity policy can select the same failing owner repeatedly. Retry/hedge exclusion is therefore a roadmap item rather than an implicit behavior.

Minimal compositions

Start with the smallest composition. Add a layer when its state and failure semantics are understood.

Architecture

Load balancing is not one algorithm. It is a control loop with distinct stages:

discovery -> membership -> eligibility -> selection -> dispatch -> feedback
                              ^                           |
                              +-- health / outliers <-----+

In that drawing, shape says what a thing is: drums hold mutable state you write to, the stack is one immutable revision over older ones, plates carry borrowed candidates, the hexagonal prism is the only solid that turns a slice into an index, instruments measure, and dashed silhouettes are optional or outside the library. Color says which crate owns it. The image is generated by scripts/render-control-loop.mjs; edit the script rather than the SVG.

Poise keeps those stages separate. This prevents a policy from silently owning connections, spawning a runtime, deciding retry behavior, or inventing health semantics on behalf of its caller.

Layer boundaries

Membership

Membership turns DNS, static configuration, Kubernetes watches, xDS, or a custom source into versioned backend snapshots. Updates must be atomic from a selector’s perspective. Removed backends may enter a draining state before their resources are retired.

poise-discovery implements this boundary with a single-writer Directory and immutable snapshots. Change batches can be staged transactionally. A removal first changes the member to Draining; an adapter calls finish_drain only after outstanding work releases its shared backend handles. Snapshots are published through an atomic pointer swap, and strictly increasing revisions prevent stale state from replacing newer state.

With its optional discovery feature, poise-tower reconciles those snapshots into a live service pool. Stable key plus unchanged backend allocation retains the service, load tracker, and Tower readiness reservation. A new backend allocation for the same key creates a new service generation. Builds are staged before commit, duplicate identities and stale revisions are rejected, and pool ordering follows the coherent snapshot.

Snapshot readers also expose a runtime-neutral, multi-subscriber stream. Each subscriber owns an independent revision cursor and waker. Publications are coalesced state rather than a lossless event log, so a slow control loop moves directly to the newest coherent snapshot. Dropping the single publisher wakes subscribers and terminates their streams after the final state is observed.

Eligibility

Eligibility combines administrative state, active health, passive failure signals, circuit state, capacity, locality constraints, and draining policy. Every exclusion should carry a machine-readable reason. The policy core starts with the portable states Ready, Draining, and Unavailable.

poise-health adds a generic HealthSignal boundary and a composable HealthChecked candidate. Its passive circuit uses permits to make half-open probe limits race-safe. Consecutive failures open the circuit, elapsed cooldown moves it to half-open, successful probes restore it, and late results from an older circuit epoch cannot mutate newer state.

Active health is also executor-neutral. The library reserves at most one due probe, while the caller chooses the timer, runtime, protocol, timeout, and response classifier. Explicit healthy or unhealthy completion advances consecutive-result thresholds; cancellation merely reschedules. Generation tokens prevent results from superseded probes from changing current health. Clock-aware reservation and completion methods let adapters keep simulated or runtime-specific monotonic time in one domain.

poise-tokio supplies that adapter for Tokio. It waits for due reservations, enforces an optional timeout, and makes timeout classification explicit: unhealthy changes threshold state, while cancellation does not. The reservation is finalized even when the runner future is dropped. The adapter also exposes allocation-free futures for discovery snapshot streams and race-free waits for a minimum revision.

Selection

A selection policy sees a coherent candidate slice and returns an index. It does not clone a backend or dispatch work. This supports zero-copy callers, borrowed snapshots, custom candidate types, deterministic tests, and policies that need request context.

Policy families planned for the core include:

• cyclic: round robin, smooth weighted round robin;
• stochastic: random, weighted random, power of two choices;
• load-aware: least loaded, least requests, peak EWMA;
• affinity: rendezvous, weighted rendezvous, precomputed ring hash, Maglev, and bounded-load rendezvous spillover;
• topology-aware: weighted priority failover with overprovisioning and explicit panic behavior, followed by health-adjusted locality weighting and endpoint capacity selection;
• adaptive: choice policies driven by measured cost and capacity.

Dispatch

Adapters translate a selected index into work on a protocol or service stack. Readiness polling, connection pooling, queueing, timeouts, cancellation, and backpressure remain adapter concerns.

poise-tower implements this boundary without changing the core policy trait. Each endpoint retains its own Tower readiness reservation. poll_ready polls all eligible idle services, and call lets the policy choose only among those that are actually ready before consuming exactly one reservation. Pending services do not block healthy peers. A service that fails readiness is quarantined until explicitly reset, while an observer hook preserves isolated errors that do not fail the aggregate pool.

The response future holds an endpoint-specific load guard. Any returned result, including a service error, is a completed attempt; dropping a pending future is cancellation. This makes in-flight and peak-EWMA policies reflect real dispatch lifetime without requiring a particular executor. Request-context projectors also allow affinity policies to borrow a routing key without allocation.

Physical retirement drops the pool’s endpoint handle but cannot invalidate an already returned Tower future, which owns its service future and load guard. Stream polling and runtime-specific wakeup loops remain outside the reconciler; callers may either drive synchronization explicitly or use StreamingDiscoveryBalance, which polls discovery before service readiness. Its bounded per-poll update budget prevents a continuously changing control plane from starving the Tower task, and it supports last-known-good or fail-closed behavior when the publisher ends.

Feedback

Attempts produce structured outcomes: latency, cancellation, overload, transport failure, and application result. Trackers feed these into load estimators, passive health, outlier ejection, and observability without coupling the core policy trait to an async runtime.

The core load trackers use RAII completion guards, so cancellation and panic unwinding cannot strand an in-flight count. Load-aware policies compare sampled immutable metrics rather than imposing Ord on concurrently changing tracker handles. PeakEwma combines decaying observed latency with current concurrency while remaining independent of any async executor.

Bounded-load rendezvous treats InFlight counts as additive cluster load. Weighted rendezvous supplies affinity order, while a prospective weighted-share bound spills hot keys to the next ranked backend with room. Every load is sampled once into reusable policy scratch. The returned detailed decision keeps the unconstrained owner visible, and atomic admission limits remain separate because concurrent selectors can race after observing the same snapshot.

Attempt results are classified as success, failure, overload, or cancellation. Rolling outcome windows ignore cancellation, retain bounded history, and expose a configurable penalty metric with a minimum-sample gate to limit cold-start noise. Explicit overload can carry more weight than an ordinary failure.

Group-relative outlier analysis establishes a success-rate baseline only from hosts with enough samples. It returns deterministic, worst-first candidate indices below a configurable standard-deviation threshold, bounded by both a maximum ejection percentage and a minimum healthy group size. Detection is pure: the control plane decides how long to eject a host and which health signal to change.

poise-observe consumes these portable decisions and outcomes without changing their ownership. ObservedPolicy delegates transparently and reports the exact selection result. Attempt records explicit completion or drop-based cancellation with elapsed monotonic time. A cloneable metrics recorder uses fixed enum dimensions, a fixed latency histogram, and relaxed atomic counters; backend identity, request keys, endpoint indices, policy names, and errors are never metric labels. Optional tracing emits structured spans and events, while an optional Tower adapter counts isolated readiness failures without retaining their error values.

Planned workspace

These are architectural boundaries, not a promise to publish a crate for every row. A crate is split only when it produces a real dependency or compatibility boundary.

Compatibility principles

• Core public types avoid runtime and protocol dependencies.
• Randomized policies accept reproducible seeds and injectable RNGs.
• Keyed policies use documented, deterministic hashing by default.
• Policy errors distinguish an empty set from a non-empty but ineligible set.
• Adding a backend must not mutate caller-owned backend values.
• Policy implementations document complexity, allocation, and membership-change behavior.
• New policy APIs require simulation evidence and adversarial tests.

Explicit non-goals

Poise is not an HTTP client, reverse proxy, service mesh, DNS resolver, or orchestrator SDK. It should make those systems easier to build without forcing their protocols or runtimes into the core.

Failure semantics

Load balancers fail in more ways than “no backend.” Poise keeps failure classes separate so callers can choose retry, failover, alerting, and telemetry without parsing strings.

Selection failures

Empty and ineligible are deliberately not interchangeable. During startup an empty directory may be expected; during an incident a fully ineligible directory often means health automation or operator state excluded the fleet.

Discovery failures

Discovery state is revisioned and transactional:

• stale or duplicate revisions are rejected before endpoint construction;
• duplicate live identities are rejected;
• revision overflow does not partially apply a batch;
• a factory failure leaves the previous live pool intact;
• dropping the publisher wakes subscribers and ends the stream after the final snapshot.

Applications choose what stream termination means. StreamingDiscoveryBalance supports last-known-good and fail-closed modes because neither is universally correct.

Readiness failures

A Tower readiness error belongs to one endpoint. Balance quarantines that endpoint. If another endpoint is ready, aggregate readiness can still succeed. If the failure exhausts the usable pool, the error includes the endpoint index at the time of failure.

Indices are diagnostic and ephemeral. Membership may change afterward; durable logs should also capture stable candidate identity at the application boundary.

Calling without a retained readiness reservation is a caller error and returns a selection failure instead of calling an unready service.

Completion, failure, and cancellation

Poise distinguishes lifecycle from application outcome:

A returned error is still completed work for latency and concurrency tracking. Cancellation does not imply backend failure and is ignored by rolling outcome windows.

Circuit permit races

Passive circuits issue permits tied to an epoch. A late result from a prior epoch cannot close, reopen, or increment the current circuit. Half-open probe limits are reserved atomically.

Dropping a permit without completion is cancellation. It must release its reservation without inventing success or failure.

Active probe races

Only one due active probe can be reserved for a health state. Explicit healthy or unhealthy results advance consecutive thresholds. Cancellation reschedules without changing classification.

Forced operator status invalidates an outstanding probe generation. A late probe result cannot overwrite the forced state.

Arithmetic and saturation

Poise rejects arithmetic that would make a decision ambiguous:

• weight accumulation uses checked arithmetic;
• revision overflow aborts transactionally;
• bounded-load capacity overflow is an explicit error;
• fixed-cardinality observation counters saturate at u64::MAX rather than wrapping.

Saturation in telemetry is observable loss of further count precision, not a signal that selection state wrapped.

Panic routing

Topology panic is explicit policy behavior, not a global “ignore health” switch. Depending on PanicMode, it may broaden eligibility to unhealthy candidates inside the selected scope. Draining and operator opt-out remain excluded.

Record whether a decision used healthy, spillover, or panic mode. Without this metadata, incident traces cannot distinguish ordinary routing from emergency scope expansion.

Error-handling rules

1. Match typed variants; do not parse Display output.
2. Keep the last coherent snapshot when a transactional update fails.
3. Do not retry configuration and invariant violations as transient network errors.
4. Preserve cancellation as its own outcome.
5. Rate-limit logs for per-endpoint readiness failures; use bounded counters for aggregate health.
6. Alert on persistent “no eligible candidates,” not on one expected startup empty result.

Public API map

This map is an orientation aid, not a substitute for rustdoc. It shows where public concepts live and which feature boundaries introduce optional dependencies.

poise-core

Candidate model

Policy contract

Feedback and load

Probe observations

poise-discovery

poise-health

poise-tower

Enable the discovery feature for reconciliation APIs.

poise-tokio

Features health and discovery are independently selectable.

poise-observe

The tracing and tower features are off by default.

Documentation layers

• This book explains contracts, composition, and operations.
• Crate-level rustdoc is the authoritative signature and feature reference.
• Source tests are executable examples of edge behavior.
• Focused policy chapters define arithmetic and churn guarantees.
• The live showcase reports verification provenance, not API documentation.

Selection policies

This chapter covers the non-keyed policy surface in poise-core. Every policy implements the same contract:

#![allow(unused)]
fn main() {
pub trait Policy<C, Context: ?Sized = ()> {
    fn pick(
        &mut self,
        candidates: &[C],
        context: &Context,
    ) -> Result<Selection, PickError>;
}
}

Shared invariants

For every general policy:

• a returned index is inside the supplied slice;
• the selected candidate reports is_eligible() == true;
• an empty slice returns PickError::Empty;
• a non-empty slice with no eligible candidates returns PickError::NoEligibleCandidates;
• the policy does not clone, mutate, or dispatch the candidate;
• configuration and arithmetic failure remain explicit errors.

The returned Selection intentionally carries an index. Membership ownership, backend borrowing, service lookup, and dispatch stay with the caller.

Round robin

RoundRobin scans from a cursor and advances past the selected index. Ineligible candidates are skipped.

An arbitrary initial cursor is reduced modulo the current slice length. Applications that reconcile membership should preserve ordering when possible.

Smooth weighted round robin

SmoothWeightedRoundRobin<Key> maintains identity-keyed current weights. Across a complete cycle, each eligible candidate receives exactly its configured integer share, while high-weight selections are spread through the cycle.

State follows identity rather than slice position. Ineligible and absent identities are pruned. Duplicate eligible identities are rejected because two state entries cannot safely represent one logical backend.

Use this policy when exact long-run ratios matter more than independent random draws.

Uniform random

Random uses reservoir sampling to select uniformly among eligible candidates without allocating an intermediate list.

Reservoir sampling means sparse eligibility does not require a preliminary count or temporary vector.

Weighted random

WeightedRandom performs two scans. The first checks and sums eligible weights into u64; the second resolves one ticket.

The checked sum can return PickError::WeightOverflow. A configuration that cannot be represented is rejected instead of silently biasing the distribution.

No alias table is cached. This favors frequently changing candidate sets and keeps rebuild behavior out of the policy.

Least loaded

LeastLoaded samples every eligible candidate’s LoadMetric and chooses the smallest value. Ties rotate in slice order using an internal cursor.

#![allow(unused)]
fn main() {
use poise_core::{Backend, Policy, policy::LeastLoaded};

let candidates = [
    Backend::new("a").with_load(3_u64),
    Backend::new("b").with_load(1_u64),
];
let mut policy = LeastLoaded::new();

assert_eq!(policy.pick(&candidates, &())?.index(), 1);
Ok::<(), poise_core::PickError>(())
}

The metric is read during selection and may change immediately afterward. Atomic admission remains a separate concern.

Power of two choices

PowerOfTwoChoices reservoir-samples two eligible candidates, compares their load metrics, and chooses the smaller. A single eligible candidate wins directly; equal-load ties use the RNG.

The current implementation scans the slice to sample without allocation. Its advantage is the selection behavior—not an O(1) candidate lookup claim.

State ownership

Policy instances are mutable because RNG state, cursors, cached tables, or identity maps can advance. Decide deliberately how instances are shared:

• one instance per worker creates independent sequences;
• a mutex around one instance creates global sequencing and contention;
• deterministic sharding by worker preserves replay within each shard;
• reconstructing an instance resets its state.

Poise does not hide synchronization inside the Policy trait. The application chooses the concurrency boundary appropriate for its request path.

Custom candidates and the policy boundary

Implement Candidate for a borrowed snapshot view when copying into Backend would obscure ownership.

Downstream policy implementation is not currently a supported extension point: although Policy is public, constructing a successful Selection is reserved to poise-core. Use the in-repository policies and their public context and candidate extension traits. A future checked policy extension must preserve the same eligible, in-bounds result contract before downstream implementations are documented as supported.

Weighted rendezvous contract

WeightedRendezvous provides capacity-proportional, key-affine selection with minimal disruption. It follows the logarithmic weighted-HRW family described by Schindelhauer and Schomaker’s Weighted Distributed Hash Tables and the weighted score summarized by the IETF weighted-HRW draft.

Selection

For each eligible candidate, Poise hashes the request context and stable backend identity, maps that hash to U in (0, 1], and computes:

race = -ln(U) / weight

The smallest race wins. This is equivalent to maximizing -weight / ln(U). Positive integer weights therefore define relative expected assignment shares: weights 1, 3, 6 target 10%, 30%, and 60% of a sufficiently large independent key population.

Selection is O(n) time, O(1) additional memory, and allocation-free. The policy borrows the candidate slice and returns only a Selection index.

Empty slices return PickError::Empty. Non-empty slices without an eligible candidate return PickError::NoEligibleCandidates. Draining and unavailable candidates are excluded before hashing.

Minimal disruption

Every candidate score depends only on the request, that candidate’s identity, and that candidate’s own weight. Consequently:

• removing a backend only remaps keys previously assigned to it;
• adding a backend only moves keys that the new backend wins;
• changing one weight never moves a key directly between two unchanged backends;
• reordering a unique-identity candidate slice changes indices but not winning identities.

These guarantees assume the request hash, candidate identity, candidate weight, eligibility, and hash builder remain unchanged where stated.

Deterministic hash pipeline

The default pipeline is part of the compatibility contract:

1. FNV-1a hashes domain-separated request and identity values.
2. mix64 applies a bijective SplitMix64 avalanche finalizer. This prevents nearby structured FNV inputs from retaining correlations that the weighted logarithm would amplify.
3. The high 53 bits plus one form an exactly representable sample in 1..=2^53, which maps to U in (0, 1].
4. A fixed 13-term range-reduced series computes ln(U) using specified basic floating-point operations rather than platform libm.
5. The complete 64-bit mixed hash breaks equal transformed-score ties. This makes equal-weight selection exactly match ordinary Rendezvous.

The avalanche finalizer is shared by ordinary and weighted rendezvous. It improves distribution for structured keys while leaving the public FNV-1a byte algorithm itself unchanged.

FnvBuildHasher is stable and inexpensive, not collision-resistant. When keys or backend identities are adversarial, callers should use with_hasher and a builder appropriate to their threat model. Reproducible assignment then also depends on that builder being deterministic and identically configured across participants.

Duplicate identities

Eligible identities should be unique. Enforcing this inside every selection would require extra memory or quadratic work, so the allocation-free policy defines duplicates rather than rejecting them:

• duplicate identities receive the same random draw;
• the duplicate with greater weight wins;
• equal-weight duplicates resolve to the earlier slice entry.

Control planes that require strict identity uniqueness should validate their membership snapshot once, before it reaches the request path.

Bounded-load affinity

BoundedLoadRendezvous combines capacity-proportional rendezvous hashing with live concurrent-load bounds. Weighted rendezvous establishes the stable preference order for a request key. The first candidate in that order with spare prospective capacity wins, so idle routing is exactly WeightedRendezvous while hot affinity owners spill to deterministic peers.

This is the request-routing form of consistent hashing with bounded loads, not a stateful implementation of the paper’s complete balls-to-bins allocation algorithm. The original algorithm owns the global assignment set and rebalances it after updates. Poise observes a borrowed snapshot of concurrent work and chooses one destination; it therefore does not claim the paper’s global movement bounds.

Capacity invariant

For eligible candidate i, the policy computes:

capacity_i = ceil(
    balance_factor_percent * (total_current_load + 1) * weight_i
    / (100 * total_eligible_weight)
)

The + 1 accounts for the request being selected. A candidate is available when its sampled load is strictly less than this capacity, which means its load after one successful admission is no greater than the reported bound. With a factor of at least 100 percent and positive Weight values, at least one eligible candidate has room in every representable snapshot.

The default balance factor is 150 percent. A factor of 100 gives the tightest bound; larger factors preserve affinity more often but allow more imbalance. Envoy documents 120–200 percent as a typical operational range for its related hash-balance feature.

Weights affect both affinity distribution and load capacity. A backend with weight three receives three times the ideal request share of a unit-weight backend. The formula uses checked integer arithmetic and exact ceiling division; it does not route based on floating-point capacity comparisons.

Load contract

Candidate loads must implement LoadMetric<Metric = u64> and represent current concurrent work. InFlight is the intended built-in tracker. PeakEwma is not accepted because a latency-times-concurrency score is not an additive request count and has no meaningful cluster average for this bound.

Each eligible load is sampled exactly once per decision. The policy retains an O(n) sample buffer, so it allocates while growing to a new high-water candidate count and then reuses that memory. shrink_to_fit lets a control plane explicitly return retained scratch memory. Selection and sampling are O(n).

The snapshot invariant is not an atomic admission guarantee. Concurrent selectors can observe the same spare slot and race after selection. Use InFlight::with_limit or another atomic admission mechanism when exceeding a process-local hard limit must fail rather than briefly overshoot.

Decisions and errors

decide returns BoundedLoadDecision, containing:

• affinity: the unconstrained weighted-rendezvous owner;
• selection: the candidate selected after applying bounds;
• spilled: whether those candidates differ;
• the selected candidate’s sampled load and prospective capacity.

The ordinary Policy::pick implementation returns only selection. Systems that measure affinity spillover should call decide and record their own bounded-cardinality event; request keys and backend identities should not become metric labels.

Empty and wholly ineligible slices keep the standard PickError distinction. Eligible weight accumulation can return WeightOverflow, load accumulation can return LoadOverflow, and sampling-buffer growth can return StateCapacityExceeded. No backend is selected from a partial sample.

Churn behavior

While every candidate is below capacity, routing has the exact weighted rendezvous guarantees: changing one backend does not move a key directly between two otherwise unchanged backends. Overload deliberately relaxes sticky routing. A key moves to its highest-ranked candidate with room and returns to its affinity owner when later samples place that owner below capacity.

This ranking avoids the cascading neighbor overflow associated with linear probing on a ring. It also makes the result independent of candidate slice order for unique identities, weights, and corresponding load samples.

References

• Consistent Hashing with Bounded Loads, Mirrokni, Thorup, and Zadimoghaddam, SODA 2018.
• Google Research algorithm overview.
• Envoy ring-hash balance-factor contract.

Ring-hash contract

RingHash implements weighted consistent hashing with a cached virtual-node table. It follows the ring and successor lookup model introduced by Karger et al. and the weighted virtual-node practice described by Envoy’s ring-hash documentation.

Construction and lookup

Each eligible candidate receives:

normalized_weight × virtual_nodes_per_weight

points. normalized_weight is the configured positive weight divided by the greatest common divisor of all eligible weights. Thus [1, 3] and [100, 300] build identical rings with identical memory use.

Virtual points are hashed into the u64 space and sorted. A request hashes into that same space and selects the first point at or clockwise from its position, wrapping to the first point after the end of the ring.

For r virtual points, rebuild cost is O(r log r) time and O(r) memory. Every safe Policy::pick first validates the candidate slice in O(n), then performs an O(log r) binary search. An unchanged lookup allocates nothing and does not rebuild the table. generation() and RingUpdate expose rebuilds for tests and control-plane diagnostics.

Reconciliation

The cache identity includes every eligible candidate’s exact identity, weight, and slice index. Changes to membership, order, eligibility, or weight trigger a staged rebuild. The old table is committed only after the replacement has been fully validated, allocated, populated, and sorted.

Reconciliation rejects:

• duplicate eligible identity with PickError::DuplicateIdentity;
• point-count overflow, configured-cap violations, and allocation failure with PickError::StateCapacityExceeded.

Neither failure replaces the last valid table. A subsequent valid candidate slice can continue using or replace it normally.

Capacity and distribution

RingHashConfig specifies virtual nodes per normalized unit weight and a hard maximum point count. The default is 128 points per unit and 1,048,576 total points. A larger table generally approximates desired weight ratios more closely, at the cost of rebuild time and memory.

The cap is checked before table allocation. It is a normal selection error, not a reason to silently reduce resolution or omit a low-weight backend.

Disruption guarantees

When the eligible set’s weight greatest-common-divisor remains unchanged, adding or removing one backend leaves every other backend’s virtual points unchanged. Only keys won by the added backend or previously owned by the removed backend move.

When a membership or weight change alters that divisor, normalization can add or remove points for otherwise unchanged backends. This preserves scale invariance of relative weights but can cause more churn than the ideal adjacent-only ring update. Applications that require the strictest churn bound should use coprime/canonically scaled weights or WeightedRendezvous, whose scores do not use set-wide normalization.

Reordering a unique-identity slice rebuilds stored indices but preserves winning identities, except in the pathological case of complete hash collisions.

Hashing and collisions

Separate domains hash candidate identities, virtual-node replicas, and request keys. Each result passes through Poise’s stable mix64 avalanche finalizer. The default FNV builder is reproducible but not collision-resistant; use with_hasher for adversarial inputs.

Point ordering is total even if hashes collide, so lookup never panics. The fallback order is position, owner hash, replica number, then current slice index. Consequently complete collisions remain safe and deterministic for one slice, but reordering that slice may change the winner. A suitable hash builder makes this edge negligible; the behavior is defined rather than hidden.

Maglev

Maglev implements the lookup-table construction introduced in Google’s Maglev network load balancer. Each eligible backend receives a deterministic permutation of a fixed table. Construction visits those permutations in turns, so backend slot counts differ by at most one. A request hashes to one slot and therefore needs one array access after the live candidate slice has been validated.

The implementation follows the original paper’s unweighted algorithm. Backend weights are intentionally ignored: a weight-only update neither rebuilds the table nor changes an assignment. Applications that need weighted affinity should use WeightedRendezvous or RingHash; weighted Maglev should be added only with a separately specified and tested construction.

Configuration

MaglevConfig requires a prime table size. Prime sizing makes every skip in 1..table_size coprime to the table size, so each backend permutation covers every slot. The default is 65,537 entries and the hard maximum is 5,000,011, matching Envoy’s documented operational bounds. Reconciliation also rejects more eligible backends than table entries rather than committing a table in which some backend is unreachable.

A larger table improves balance and generally reduces disruption, at the cost of proportionally more memory and rebuild work. The table contains usize candidate indices, in addition to O(n) member and construction state.

Cache and reconciliation

reconcile compares eligible identity, slice index, and order with the committed membership. Eligibility, addition, removal, or reorder rebuilds the table. The policy stores candidate indices, so a reorder must rebuild; canonical identity ordering ensures that normal hash inputs retain the same winning identity afterward. If both independent ordering hashes collide, slice index is the documented deterministic fallback.

Construction happens in temporary allocations. Duplicate eligible identities, excess member count, and allocation failure return PickError without replacing the last committed table or advancing its generation. reset clears the cached state while retaining configuration and the hash builder.

The ordinary Policy::pick method reconciles before lookup. An unchanged pick therefore performs O(n) exact validation, followed by O(1) request lookup, with no allocation. Control planes that already know when membership changes can call reconcile explicitly and inspect MaglevUpdate, but safe picks still validate the borrowed slice so stale candidate indices cannot escape.

Churn semantics

Maglev is minimally disruptive in a statistical sense, not the strict remove-only remapping sense of rendezvous hashing. Adding or removing a backend usually preserves most assignments, while a small number of keys owned by unchanged backends can also move during table reconstruction. Use rendezvous hashing when that stronger invariant matters more than constant-time lookup.

The default FnvBuildHasher plus Poise’s stable avalanche finalizer makes table construction and request replay deterministic across processes using the same candidate identities and configuration. Caller-provided hash builders define their own compatibility behavior and should produce identical hashes on every balancer expected to share assignments.

References

• Maglev: A Fast and Reliable Software Network Load Balancer, Eisenbud et al., NSDI 2016.
• Envoy Maglev load-balancing policy.

Priority routing and panic

PriorityWeightedRandom routes across ordered failover groups. Priority 0 receives traffic first, then priority 1, and so on. A lower priority receives only traffic that higher priorities cannot cover according to their weighted availability and the configured overprovisioning factor.

The primitive deliberately performs endpoint selection itself using weighted random choice. Its name makes that behavior explicit; it does not pretend to wrap an arbitrary policy through an eligibility mask that the core Policy contract cannot express. Priority planning and endpoint sampling share one coherent snapshot, and the detailed decision reports both the selected priority and whether normal or panic eligibility was used.

Candidate metadata

Applications can implement PriorityCandidate for their own candidate type or wrap any existing Candidate in Prioritized<C>. Lower u32 priority values are preferred; gaps are allowed and carry no semantic weight.

Three sampled predicates remain distinct:

• priority membership: configured capacity used as the availability denominator;
• normal eligibility: healthy capacity used during ordinary routing;
• panic eligibility: members that use-all panic may revive.

The built-in wrapper excludes Draining candidates from membership and panic. It allows other unavailable members in panic by default. Set with_panic_eligibility(false) for a candidate whose hard capacity, policy, or administrative exclusion must never be bypassed. This is intentionally explicit: panic must not silently override a circuit breaker or admission limit merely because both happen to make is_eligible false.

Availability and spillover

For each priority, the policy computes fixed-point millionth-share values:

raw_availability = eligible_weight / configured_member_weight
effective_availability = min(
    100%,
    raw_availability * overprovisioning_factor
)

The default overprovisioning factor is 140 percent. Thus a priority remains able to carry all traffic until its weighted availability drops below roughly 71.4 percent. Factors below 100 are rejected.

When combined effective availability reaches 100 percent, priorities consume traffic capacity in ascending order. For example, effective availability of 70 percent at priority 0 leaves 30 percent for priority 1 even if priority 1 could handle more. When combined availability is below 100 percent, remaining shares are normalized rather than leaving an accidental random-selection gap.

Candidate Weight participates in all three relevant places: availability accounting, global-panic priority share, and endpoint selection within the chosen priority. Accumulation is checked and returns WeightOverflow rather than wrapping.

Panic

Panic is considered only when combined effective availability across all priorities is below 100 percent. If lower priorities provide enough capacity, an unhealthy primary spills traffic normally and does not panic. This avoids reviving unhealthy endpoints while a sound failover remains available.

Within a globally underprovisioned snapshot, a priority enters panic when its raw weighted availability is strictly below panic_threshold_percent. The default is 50 percent; zero disables panic.

PanicMode::UseAll chooses among explicitly panic-eligible members. Draining and opted-out members remain excluded. PanicMode::FailClosed returns PickError::PanicRejected when traffic lands on a panicking priority, making intentional load shedding distinguishable from an empty or ordinarily ineligible cluster. If every priority has zero effective availability, use-all panic distributes traffic across priorities in proportion to their panic-eligible weights.

Complexity and consistency

Membership and all eligibility predicates are sampled once per decision. The policy retains member and group buffers, allocating only when a new high-water candidate count exceeds their capacity. Priority aggregation sorts temporary groups, so calculation is O(n log n) time with O(n) retained memory; endpoint choice is O(n).

PriorityDecision exposes the candidate Selection, numeric priority, and PriorityMode::{Healthy, Panic}. The regular Policy::pick method returns only the selection. Seeded constructors provide deterministic replay for tests and simulations; ordinary construction seeds from the process random source.

References

• Envoy priority panic-threshold behavior.
• Envoy overprovisioning factor.
• Envoy locality and priority failover example.

Locality-weighted routing

LocalityWeightedRandom composes failover priority, locality preference, and endpoint capacity without collapsing their independent meanings. Every decision follows this hierarchy:

priority -> health-adjusted locality -> weighted endpoint

Priority is selected first using the same availability, overprovisioning, and panic contract as PriorityWeightedRandom. Only localities in that selected priority participate in the second stage. A very large weight in a failover priority therefore cannot steal traffic from a fully provisioned primary.

Candidate metadata

Applications can implement LocalityCandidate for their own coherent snapshot type, or compose the built-in wrappers:

#![allow(unused)]
fn main() {
use poise_core::{Backend, Weight, policy::{Localized, Prioritized}};

let endpoint = Localized::new(
    Prioritized::new(Backend::new("api-west-1"), 0),
    "us-west-2",
)
.with_locality_weight(Weight::new(3)?);
Ok::<(), poise_core::InvalidWeight>(())
}

There are two deliberately separate weights:

• LocalityCandidate::locality_weight expresses the control plane’s desired share between localities at one priority.
• Candidate::weight expresses endpoint capacity and divides traffic only inside the chosen locality.

All endpoints in one (priority, locality) group must advertise the same locality weight. InconsistentTopology rejects contradictory snapshots rather than making their result depend on candidate order.

Health and spillover

For locality L, Poise computes fixed-point millionth-share availability from weighted endpoint capacity:

availability(L) = min(100%, overprovisioning × eligible_weight(L) / total_weight(L))
effective_weight(L) = configured_locality_weight(L) × availability(L)
traffic(L) = effective_weight(L) / sum(effective_weight)

The default overprovisioning factor is 140 percent. A locality therefore keeps its full configured share while at least 5/7 of its weighted capacity remains eligible. Beyond that point, its shortfall spills proportionally across the other selected-priority localities.

For example, locality X with configured weight 1 and 50 percent available capacity has effective weight 70. Fully available locality Y with configured weight 2 has effective weight 200. Their resulting shares are approximately 26 and 74 percent. Once a locality is selected, endpoint weights select among only its eligible members.

Eligibility, priority membership, and panic eligibility are sampled once per decision by the shared priority engine. Locality health uses that exact sample; it never re-reads a changing health signal. In UseAll panic mode, panic-eligible endpoints form the selectable capacity. FailClosed retains the priority policy’s PanicRejected behavior. Draining endpoints are not configured capacity by default.

Millionth-share arithmetic is deterministic and integer-only. A nonempty locality receives a minimum effective availability unit if quantization would otherwise round it to zero. Weight accumulation is checked and returns WeightOverflow rather than wrapping.

Scope

This policy implements explicit control-plane locality weights. It does not infer that the caller’s zone must always be preferred, because that requires a request-origin distribution and zone-aware routing policy. Callers can express strict regional failover with priorities, proportional cross-region routing with locality weights, or combine both.

Calculation retains O(n) scratch and takes O(n log n) time for priority and locality grouping plus O(n) endpoint selection. Stable candidate high-water marks allocate nothing after warmup. seeded and seeded_with provide exact replay for the same candidate order and metadata.

The hierarchy and health-adjusted weighting model are informed by Envoy’s locality-weighted load balancing, priority routing, and the distinction between locality and endpoint weights in its endpoint API. Envoy’s separate zone-aware policy illustrates why implicit caller-local preference is a different primitive.

Load and feedback

Poise separates a load measurement from an admission reservation and from an attempt outcome. Conflating those concepts is a common source of stranded capacity and unstable feedback loops.

The LoadMetric boundary

A load-aware candidate exposes a metric whose smaller values represent less load. Built-in integer metrics, InFlight, and PeakEwma implement the contract.

A policy samples a metric during one decision. The value can change immediately afterward, so measurement alone is not a hard limit.

In-flight accounting

InFlight is a shared atomic counter. Reserving returns an RAII guard:

counter N ── reserve ──▶ N + 1
              │
              ├─ complete ──▶ N
              └─ drop/cancel ▶ N

The counter is balanced under normal completion, early return, future cancellation, and panic unwinding. try_acquire adds an atomic limit check; rejection does not increment the counter.

Selection and admission can still race:

1. two selectors observe the same available capacity;
2. both choose the endpoint;
3. only one atomic reservation succeeds.

The loser must select again or return overload according to the application’s budget. Never assume a bounded-load policy replaces atomic admission.

Peak EWMA

PeakEwma combines decaying observed latency with current concurrency. A new high latency raises the score immediately; without new observations, the latency component decays toward its configured floor.

The tracker uses Rust’s monotonic Instant internally. The current API does not provide caller-supplied clock injection.

Configuration rejects zero decay or default-latency parameters. Completion records latency and releases concurrency; cancellation releases concurrency without recording a latency sample.

Outcome classification

The portable Outcome classes are:

• success;
• failure;
• overload;
• cancellation.

Overload can carry a larger penalty than ordinary failure in an OutcomeWindow. Cancellation is not a backend observation and therefore does not enter success-rate statistics.

Rolling windows

OutcomeWindow stores bounded recent history. Its minimum-sample gate avoids penalizing a cold endpoint based on one early failure. When capacity is reached, the oldest observation is evicted.

The window exposes a penalty metric suitable for composition with load-aware policies. It does not automatically open a circuit or eject a host; the control plane chooses how to apply the signal.

Group-relative outliers

OutlierDetector compares sufficiently sampled hosts to their group baseline. It returns deterministic worst-first candidate indices below a configured standard-deviation threshold.

Two caps protect availability:

• maximum ejection percentage;
• minimum healthy group size.

Detection is pure. Scheduling ejection duration, changing health state, and deciding when to re-admit remain control-plane responsibilities.

Feedback-loop discipline

Any feedback controller can oscillate. Establish:

• a measurement window longer than individual request jitter;
• a minimum sample count;
• explicit overload weighting;
• bounded ejection;
• cooldown before half-open probes;
• monotonic time from one domain;
• dashboards that distinguish selection, admission, and outcome.

Avoid feeding exporter-delayed metrics back into the hot path. Use the shared in-process tracker that belongs to the service generation.

Generation safety

Load state belongs to the concrete endpoint generation, not merely its logical key. When discovery reuses a key for a new backend allocation, build a new tracker. Outstanding futures from the old generation retain and release the old guard without modifying the replacement.

Health and circuits

poise-health provides executor-neutral health state. It does not send network probes, sleep, spawn tasks, or decide which application result is a failure. Those choices remain with adapters and protocol code.

Compose independent signals

HealthChecked<Backend, Health> wraps an existing candidate and adds a HealthSignal. Candidate identity, configured weight, topology metadata, and load delegate to the inner backend. Eligibility requires both the inner candidate and health signal to permit selection.

This composition preserves administrative intent:

• a draining backend does not become selectable because its probe is healthy;
• an unavailable backend does not become ready because a circuit closes;
• nested health wrappers can express multiple independent gates.

Passive circuit states

Closed ── failure threshold ──▶ Open
  ▲                              │
  │                              │ cooldown elapsed
  │                              ▼
  └── success threshold ───── Half-open
                                 │
                                 └── probe failure ──▶ Open

CircuitConfig controls:

• consecutive failures required to open;
• open cooldown duration;
• successful half-open probes required to close;
• maximum concurrent half-open permits.

Configuration rejects a zero open duration. Large durations remain valid without requiring eager future-instant arithmetic.

Permit semantics

PassiveHealth::try_acquire returns a CircuitPermit or a typed rejection. A permit reserves half-open capacity when applicable. Exactly one terminal action should follow:

• complete with a classified outcome;
• cancel explicitly;
• drop, which cancels.

Every permit carries the circuit epoch from which it was issued. Late completion from an old epoch is ignored, preventing reordered responses from corrupting a newer forced or reopened state.

Active health

ActiveHealth owns classification and scheduling state, while the caller owns I/O and time. Its configuration defines:

• healthy-result threshold;
• unhealthy-result threshold;
• probe interval;
• initial unknown policy.

Only one caller can reserve a due probe. Completion with Healthy or Unhealthy advances consecutive thresholds. Cancellation preserves the current classification and schedules another attempt.

Use the clock-aware methods when the adapter supplies a simulated or runtime-specific Instant. Do not reserve using one time domain and complete using another.

Tokio adapter

poise-tokio::ActiveHealthRunner supplies timing around a caller-defined TokioProbe. Timeout policy is explicit:

• classify timeout as unhealthy; or
• cancel without changing classification.

Dropping the runner future finalizes its reservation as cancellation. Repeated probe scheduling follows Tokio’s clock, including paused test time.

Outcome windows versus circuits

These mechanisms answer different questions:

Combining them is reasonable, but define precedence. A typical order is administrative state → active health → circuit permission → outlier ejection.

Choosing thresholds

Thresholds are workload parameters, not library defaults to copy blindly. Derive them from:

• ordinary request rate per endpoint;
• expected transient failure bursts;
• probe interval and timeout;
• acceptable time to detect and time to recover;
• minimum healthy capacity;
• retry amplification.

A failure threshold of three means something very different at one request per minute and ten thousand requests per second.

Operational invariants

• Force operations take effect immediately and invalidate stale probes.
• Cancellation never counts as backend failure.
• Half-open concurrency is an atomic reservation, not a statistical target.
• Group-relative ejection never exceeds both configured availability caps.
• Health wrappers never revive administrative draining or opt-out.
• Control-plane actions should record the signal and epoch that caused them.

Discovery and Tower

Enable poise-tower’s optional discovery feature to reconcile poise-discovery snapshots into a live DiscoveryBalance. The feature is off by default, preserving the base Tower adapter’s minimal dependency graph.

[dependencies]
poise-tower = { version = "0.1", features = ["discovery"] }

Service generations

Discovery identity and backend allocation have separate meanings:

• the stable key identifies the logical member across revisions;
• the shared backend allocation identifies one configuration generation.

When both are unchanged, reconciliation retains the complete Endpoint: Tower service, readiness reservation, failure quarantine, and dispatch load. A drain or other membership-only transition therefore excludes new traffic without discarding live state.

An upsert creates a new backend allocation even if its value compares equal. The reconciler treats that as an intentional generation change and asks the EndpointFactory for a replacement service and load tracker. The old endpoint is dropped only when the new revision commits.

Transactional application

For every newer snapshot, reconciliation:

1. validates the revision and uniqueness of live and snapshot identities;
2. classifies members as retained, added, or rebuilt;
3. stages every required service generation through the factory;
4. commits endpoints in snapshot order and advances the applied revision.

If validation or any factory call fails, the live endpoint set, ordering, readiness, load, and applied revision remain unchanged. Successfully staged services are dropped. External effects performed inside a factory cannot be rolled back, so factories should avoid publishing a service elsewhere before returning it.

An equal revision is an idempotent no-op. An older revision returns ReconcileError::StaleRevision. ReconcileReport counts retained, added, rebuilt, removed, and draining members for control-plane observability.

Factory choices

An EndpointFactory returns both a Tower service and a LoadTracker. Closures that return (service, tracker) implement it directly. Wrap a service-only closure with in_flight_factory to create a fresh unbounded InFlight tracker for every service generation.

#![allow(unused)]
fn main() {
use std::convert::Infallible;
use poise_core::{Backend, policy::RoundRobin};
use poise_discovery::Directory;
use poise_tower::{DiscoveryBalance, in_flight_factory};
let mut directory = Directory::new();
directory.upsert("west", Backend::new("http://west"))?;

let factory = in_flight_factory(|_key: &&str, backend: &Backend<&str>| {
    // A real factory would construct a protocol client from this value.
    Ok::<_, Infallible>(backend.id().to_string())
});
let mut balance = DiscoveryBalance::new(RoundRobin::new(), factory);
let report = balance.apply_snapshot(&directory.snapshot())?;
assert_eq!(report.added(), 1);
Ok::<(), Box<dyn std::error::Error>>(())
}

Draining and retirement

A draining snapshot preserves the endpoint but makes its Discovered candidate ineligible immediately. Existing response futures remain independent of that eligibility transition. After the application observes no outstanding work and calls Directory::finish_drain, the next snapshot removes the endpoint from the pool. A response future that was already returned still owns its load guard and remains valid even after physical retirement.

Driving synchronization

DiscoveryBalance::sync performs one atomic load from a SnapshotReader and applies it. It does not poll a stream or spawn a task. An application may call it from configuration callbacks, a control-plane loop, or a runtime-specific watch task. Keeping this primitive synchronous makes its transaction and error semantics deterministic while leaving wakeups and retry policy to the adapter that owns the runtime.

For event-driven operation, SnapshotReader::subscribe returns a runtime-neutral SnapshotStream. Its first item is the current snapshot; SnapshotReader::changes starts after the current revision instead. Each subscriber has its own waker and cursor. If several revisions arrive between polls, only the newest snapshot is yielded—membership snapshots represent current state, not an event log. The stream ends after its publisher is dropped and its final revision has been observed.

StreamingDiscoveryBalance combines that subscription with a DiscoveryBalance. Every poll_ready first reconciles visible discovery state, then polls endpoint readiness. The same task waker is registered with both sources, so no driver task or Tokio dependency is required.

StreamingConfig bounds the number of applications per readiness poll. If the budget is exhausted, the wrapper self-wakes and yields Pending, preventing a busy control plane from starving other work. Publisher closure defaults to last-known-good service; StreamEndPolicy::FailClosed instead rejects readiness after the final snapshot. Reconciliation failures are returned as StreamingError::Reconcile while leaving the previous pool intact. Since the failed snapshot has been observed, a later poll can continue serving the last good pool or apply a newer publication.

Tower dispatch contract

poise-tower turns a collection of candidate metadata and Tower services into one policy-driven Service. It deliberately does not create a runtime, spawn a task, buffer requests, or choose retry behavior.

Endpoint model

An Endpoint<C, S, L> owns three durable values:

• candidate metadata C for identity, weight, and administrative or health eligibility;
• Tower service S for readiness and dispatch;
• load tracker L, exposed to load-aware Poise policies and held for the lifetime of each dispatched response future.

The default tracker is an unbounded InFlight. Endpoint::with_tracker accepts another LoadTracker, including PeakEwma. The endpoint’s policy load is this dispatch tracker, not the load field of C; this ensures the metric describes the service instance that will actually receive the call.

Readiness lifecycle

Idle --poll_ready(Ok)--> Ready --selected call--> Idle
  |                         |
  +--poll_ready(Err)--> Failed --explicit reset--+

Balance::poll_ready scans all administratively eligible idle endpoints. A pending endpoint remains idle and is polled again when its service wakes the caller. A ready endpoint retains its reservation and is not polled again until selected. This matters for services whose poll_ready reserves a permit.

Readiness failure quarantines one endpoint. If another endpoint is ready, the aggregate remains ready. Install with_readiness_observer to record every such error. If the failure exhausts the usable pool, poll_ready returns it with the endpoint index. Reset failed services explicitly after repairing or replacing them.

As required by Tower, callers should await readiness before each call. Calling without a retained reservation returns a selection error rather than invoking an unready inner service.

Completion and cancellation

Dispatch reserves the selected endpoint’s tracker before calling its service. The returned ResponseFuture owns both the service future and load guard:

• Ok(response) completes the load guard;
• Err(error) also completes it and preserves the dispatch-time endpoint index;
• dropping the pending future drops the guard as cancellation.

No response future is boxed by the adapter. poll_ready takes O(n) time over the endpoint set and performs no allocation. call adds the selected policy’s normal time complexity.

Affinity context

Balance::new supplies unit context for request-independent policies. Use with_context(UseRequest) when the entire request is an affinity key. For structured requests, implement RequestContext<Request> to borrow only the stable routing field; the projection does not need to allocate or clone it.

Membership changes

push, remove, and endpoints_mut support controlled membership changes. New endpoints always enter idle. Mutable access to a service invalidates its readiness reservation. An error’s endpoint index describes the ordering at dispatch or readiness time and may become stale after later membership edits; use candidate identity for durable telemetry labels.

Stream-driven reconciliation with poise-discovery is a separate adapter so the base Tower service remains runtime-neutral. Enable the optional discovery feature for the versioned static reconciler described in Discovery and Tower.

Tokio integration

poise-tokio is an optional runtime boundary. Tokio does not appear in the dependency graphs of poise-core, poise-discovery, poise-health, or the base poise-tower adapter.

The crate’s default features enable both integration families. With default features disabled, health pulls in Tokio timing and poise-health, while discovery pulls in only poise-discovery; the discovery wait futures are executor-neutral and need no Tokio dependency of their own.

Active health

ActiveHealthRunner::run_once waits until a probe is due, acquires the sole reservation, and runs one caller-provided async probe. The probe translates its protocol result into ProbeResult; the runner owns timing and state-machine finalization.

#![allow(unused)]
fn main() {
use std::{num::NonZeroU32, time::Duration};

use poise_health::{ActiveHealth, ActiveHealthConfig, ProbeResult};
use poise_tokio::{ActiveHealthRunner, ProbeRunnerConfig};

async fn example() {
let health = ActiveHealth::new(
    ActiveHealthConfig::new(
        Duration::from_secs(10),
        NonZeroU32::new(2).unwrap(),
        NonZeroU32::new(3).unwrap(),
    )
    .unwrap(),
);
let config = ProbeRunnerConfig::new(Duration::from_secs(2)).unwrap();
let mut runner = ActiveHealthRunner::new(
    health,
    || async {
        // Perform a protocol-specific request and classify its response here.
        ProbeResult::Healthy
    },
    config,
);

let report = runner.run_once().await.unwrap();
assert_eq!(report.result(), Some(ProbeResult::Healthy));
}
}

A finite timeout is unhealthy by default. ProbeTimeoutPolicy::Cancel instead preserves the existing health classification and threshold counters. Disabling the timeout is explicit with ProbeRunnerConfig::without_timeout().

The library deliberately does not spawn a background task or prescribe a shutdown channel. Applications can loop over run_once and cancel it using their existing task supervision or tokio::select!. Dropping run_once before or during a probe cancels its reservation and schedules the next interval; it never leaves the health state marked in flight.

If another owner already holds the active probe reservation, run_once returns ProbeRunnerError::ProbeAlreadyRunning instead of spinning. The caller can retry after that external owner completes or cancels its reservation.

The runner consistently converts tokio::time::Instant into the core’s clock-aware APIs. Consequently intervals, timeouts, and cancellation remain deterministic under Tokio’s paused test clock.

Discovery waits

next_snapshot returns an allocation-free future borrowing a SnapshotStream. It avoids requiring a general stream-extension dependency for the common single-item wait. wait_for_revision creates a subscription before examining its current state, closing the load-then-register race, and then waits until the publisher reaches or passes the requested revision.

#![allow(unused)]
fn main() {
use poise_discovery::{Revision, Snapshot, snapshot_channel};
use poise_tokio::wait_for_revision;

async fn example() {
let (mut publisher, reader) = snapshot_channel(Snapshot::empty());
publisher
    .publish(Snapshot::new(Revision::new(1), vec!["backend-a"]))
    .unwrap();

let snapshot = wait_for_revision(&reader, Revision::new(1))
    .await
    .expect("publisher remains open");
assert_eq!(snapshot.revision(), Revision::new(1));
}
}

Snapshot streams coalesce bursts to the latest coherent state. They are state notifications, not lossless event logs. A wait returns None when the sole publisher closes before the requested revision is available.

Observability contract

poise-observe turns portable policy decisions and attempt outcomes into telemetry without changing selection or dispatch behavior. Its default build has no tracing, metrics-facade, runtime, or Tower dependency.

Cardinality budget

The built-in Metrics recorder has no caller-defined labels. Its storage shape is fixed at compile time:

The complete recorder therefore owns 28 saturating counters, regardless of backend count, traffic shape, or input diversity.

Backend identity, endpoint index, request or affinity key, policy name, route, error text, and discovery source are never metric dimensions. This remains true even when those values have millions of distinct values. Applications may add their own bounded labels while exporting a MetricsSnapshot, but doing so is outside the library’s safety guarantee.

Counters saturate at u64::MAX. Recording uses relaxed atomic operations. A snapshot reads those atomics independently, so under concurrent updates it represents a narrow interval rather than a globally transactional instant.

DecisionKind::ALL and AttemptKind::ALL let exporters enumerate every stable dimension. ATTEMPT_LATENCY_BOUNDS and ATTEMPT_LATENCY_BUCKET_COUNT expose the complete cumulative histogram shape.

Policy decisions

ObservedPolicy implements the same Policy contract as its inner policy. It returns the original Selection or PickError unchanged and emits exactly one DecisionEvent afterward.

#![allow(unused)]
fn main() {
use poise_core::{Backend, Policy, policy::LeastLoaded};
use poise_observe::{DecisionKind, Metrics, ObservedPolicy};

let metrics = Metrics::new();
let mut policy = ObservedPolicy::new(LeastLoaded::new(), metrics.clone());
let candidates = [Backend::new("a"), Backend::new("b")];

let _selection = policy.pick(&candidates, &()).unwrap();
assert_eq!(metrics.snapshot().decisions(DecisionKind::Selected), 1);
}

Observers receive candidate count and selected slice index as diagnostic context. Metrics discards both values as labels; TracingObserver can include them as event fields.

Attempt lifetime

Attempt owns an observer and a monotonic start time. Calling success, failure, overloaded, cancel, or complete records exactly one result. Dropping an unfinished guard records cancellation, covering task cancellation, early returns, and panic unwinding.

#![allow(unused)]
fn main() {
use poise_observe::{Attempt, AttemptKind, Metrics};

let metrics = Metrics::new();
let attempt = Attempt::new(metrics.clone());
// Dispatch protocol-specific work.
attempt.success();

assert_eq!(metrics.snapshot().attempts(AttemptKind::Success), 1);
}

Application code retains responsibility for translating protocol and application results into Poise’s portable outcome classes.

Tracing

The optional tracing feature adds:

• TracingObserver, which emits decision and attempt events at DEBUG and readiness failures at WARN;
• TracedPolicy, which wraps each synchronous policy call in a poise.policy.pick span and records its fixed decision classification.

Targets are poise::decision, poise::attempt, and poise::readiness. Tracing records contain numeric indices and fixed classifications, but exclude backend IDs, request keys, policy names, and errors by default.

Use Fanout::new(metrics, TracingObserver) when the same records should reach both built-in sinks. Wrapping a TracedPolicy in an ObservedPolicy is also valid: the former supplies the timing span and the latter supplies metrics or completion events.

Tower readiness

The optional tower feature adds TowerObserver, which implements poise_tower::ObserveReadinessError. Install it through Balance::with_readiness_observer:

#![allow(unused)]
fn main() {
use poise_core::{Backend, policy::RoundRobin};
use poise_observe::{Metrics, TowerObserver};
use poise_tower::{Balance, Endpoint};
use std::{convert::Infallible, future};
use tower::service_fn;
fn echo(value: u32) -> future::Ready<Result<u32, Infallible>> {
    future::ready(Ok(value))
}
let metrics = Metrics::new();
let endpoints = vec![Endpoint::new(Backend::new("a"), service_fn(echo))];
let balance = Balance::new(endpoints, RoundRobin::new())
    .with_readiness_observer(TowerObserver::new(metrics.clone()));
let _ = balance;
}

The adapter counts the error and discards endpoint and error values for metric purposes. Applications that need full error diagnostics can install a Tower closure that forwards to the adapter and separately logs or handles the borrowed error.

Operating Poise

Operating a load balancer means observing the control loop without turning high-cardinality request data into a second failure mode.

Production-readiness checklist

Membership

• Stable backend identities survive ordinary discovery refreshes.
• Revisions increase monotonically and are logged at reconciliation boundaries.
• Duplicate identities fail the update rather than overwrite silently.
• Draining has a defined completion condition.
• Stream termination chooses last-known-good or fail-closed deliberately.

Selection

• Stochastic seeds can be recorded in simulations and incident reproductions.
• Key encoding is canonical and documented.
• Weight changes are reviewed as traffic changes, not harmless metadata edits.
• Custom policies are tested for in-bounds eligible selection.
• Affinity retry behavior is explicit.

Dispatch

• Every call follows a successful Tower readiness poll.
• Endpoint generation changes replace readiness and load state together.
• Capacity reservation failure has bounded reselection behavior.
• Cancellation is preserved when a response future is abandoned.
• Retry and hedge budgets live outside the selection policy.

Health

• Active probe timing and timeout classification are documented.
• Passive circuit thresholds are calibrated to request rate.
• Half-open limits cannot exceed safe recovery traffic.
• Outlier ejection preserves minimum healthy capacity.
• Operator draining always outranks automated recovery.

Observation

• Metric labels remain bounded.
• Stable identity appears in sampled logs or traces, not built-in metric labels.
• Healthy, spillover, and panic topology modes are distinguishable.
• Readiness failures have both a bounded counter and rate-limited diagnostics.
• Saturated counters are detectable during export.

Recommended signals

The built-in Metrics recorder exposes a deliberately fixed surface:

• decision results by DecisionKind;
• attempts by AttemptKind;
• readiness-failure count;
• cumulative attempt-latency buckets;
• attempt-latency sum.

Derive rates and ratios in the monitoring system. Avoid resetting shared counters on scrape; snapshots are cumulative.

Application-level telemetry may add bounded dimensions such as service name, cluster, or deployment environment. Never add raw request keys, endpoint error strings, or unbounded backend identities as metric labels.

Suggested service-level indicators

Interpretation depends on traffic. A high cancellation rate may reflect client deadlines rather than backend failure.

Incident runbook

No eligible candidates

1. Confirm whether membership is empty or non-empty.
2. Record current discovery revision and publisher state.
3. Separate draining, unavailable, circuit-open, and outlier-ejected members.
4. Inspect whether panic routing is disabled or intentionally fail-closed.
5. Avoid force-enabling draining members merely to restore capacity.

Uneven traffic

1. Confirm the policy family and candidate order.
2. Compare configured weights and actual eligible duration.
3. For stochastic policies, inspect a meaningful sample window.
4. For affinity, measure key distribution before endpoint distribution.
5. For load-aware policies, inspect metric generation ownership and stranded guards.

Churn causes excessive remapping

1. Verify identity stability and canonical key encoding.
2. Distinguish reorder from addition, removal, and weight change.
3. Confirm ring or Maglev cache rebuild reason.
4. Compare the measured movement to that policy’s documented guarantee.
5. Check whether retry exclusions or application fallbacks add extra movement.

Load never returns to zero

1. Find outstanding response futures.
2. Confirm cancellation paths drop their guards.
3. Check for deliberate leaks through forgotten guards in application code.
4. Verify endpoint generations are not sharing one tracker accidentally.
5. Reproduce with the Loom in-flight models before changing atomic ordering.

Rollout strategy

For a new policy:

1. replay production-shaped keys or load in simulation;
2. shadow decisions without dispatching;
3. compare eligibility and selected identity to the current balancer;
4. canary one bounded traffic slice;
5. watch exhaustion, cancellation, readiness, and topology mode;
6. retain an immediate configuration rollback.

Do not interpret a distribution benchmark as a rollout plan. Failure modes usually appear in membership, readiness, cancellation, or key skew.

Capacity planning

Configured weight is a relative routing input. Capacity planning must also account for:

• per-endpoint concurrency limits;
• request cost variance;
• retry amplification;
• health headroom;
• priority overprovisioning;
• locality failure;
• connection-pool and runtime limits.

The balancer can preserve a capacity model only if the supplied weights and load signals describe the real service generation.

Performance model

Poise documents algorithmic cost and allocation behavior, and does not publish unsupported throughput claims. Criterion baselines now exist for poise-core selection and membership-change cost, and scripts/bench.sh reproduces them. Dispatch, health, and discovery remain unmeasured.

Read the recorded numbers for shape rather than magnitude. The constants belong to one machine and one compiler; how a policy responds to candidate count and to churn is the part that transfers to a different deployment.

Selection cost

n is candidate count, p ring-point count, and m Maglev table size. Eligibility density and membership churn influence constants materially.

The two cached rows are stated carefully because their headline is easy to misread, and this table previously did. A Maglev table lookup is O(1) and a ring lookup is O(log p), but neither is what a call to pick costs: every safe pick first validates the cached table against the candidate slice, which is O(n). Measured cost therefore grows with candidate count for both, and the constant-time lookup is not the term that dominates.

This does not make them slow. Validating a candidate is far cheaper than hashing one, so at 512 members Maglev picks in about a fifth of the time plain rendezvous takes, and about a fifteenth of bounded-load rendezvous. The correction is to the asymptote, not to the ranking: a cached policy is cheap because its per-candidate work is small, not because it avoids per-candidate work.

Recorded baselines

Median pick time from benches/selection.rs, 100 samples per point, on an AMD Ryzen Threadripper PRO 5975WX with rustc 1.97.1. Reproduce with scripts/bench.sh.

Every candidate is eligible in these runs, which is the cheapest case for the policies that scan until they find an eligible member and the most expensive for those that must consider everything. Round robin is the one whose measured shape differs from its worst case for that reason: it is flat here because the next candidate is always eligible, and its O(n) row describes a slice where that is not true.

Everything else tracks candidate count, and the affinity policies separate by constant rather than by shape: rendezvous hashes every candidate, while ring hash and Maglev only validate every candidate and then consult a table. That constant is the whole practical difference between them at 512 members.

Membership-change cost

Cached affinity policies compute a membership fingerprint from the fields that affect their structure. An unchanged membership reuses the table. Relevant identity, eligibility, order, or weight changes rebuild transactionally.

Rebuild failure preserves the previous live cache. Callers should expose rebuild errors and avoid retrying them for every request without backoff or configuration repair.

Measured cost of a pick that triggers a rebuild, from benches/membership.rs, which flips one member in and out on every iteration:

Three things follow, and none of them are visible from the complexity column alone.

Ring hash and Maglev trade places. Ring rebuild is O(p) in ring points, which scale with members, so it grows with the fleet. Maglev rebuild is O(m) in a table size fixed by configuration, so it is nearly flat and its cost at eight members is almost its cost at five hundred. Ring hash is an order of magnitude cheaper on small fleets and several times more expensive on large ones; on this machine they cross somewhere between 64 and 512 members. A deployment choosing between them on churn cost should know which side of that crossing it sits on.

Rebuild dwarfs selection. A 512-member ring rebuild costs roughly three thousand times a steady-state pick at the same size. Rebuilds are amortized across the requests that follow a membership change, so this is not a per-request cost — but it is the cost of a rollout, and a deployment whose membership changes faster than it can amortize a rebuild has chosen the wrong policy, whatever its lookup complexity says.

Smooth weighted round robin has no rebuild penalty at all. Its churn numbers match its steady-state numbers, because identity-keyed state migrates incrementally rather than being rebuilt. That is the property the table calls identity-keyed state migration, and it is the reason it appears here despite caching nothing to rebuild.

Dispatch cost

poise-tower::Balance::poll_ready scans endpoint readiness in O(n) and performs no allocation. A ready service retains its reservation until selected.

call performs the policy’s normal selection cost, acquires the endpoint load guard, and returns a concrete response future without boxing it.

Snapshot reconciliation is control-plane work. Unchanged service generations retain service, readiness, and load state; new generations are built before commit.

Atomic costs

InFlight uses atomic updates for reservation and release. PeakEwma couples atomic concurrency with synchronized estimator state. Metrics uses relaxed atomic saturating counters.

Contention depends on how policy and tracker instances are shared. One global policy mutex may dominate every algorithmic difference in the table above.

Memory bounds

• Discovery snapshots own one immutable membership vector per live revision retained by readers.
• Outcome windows have configured fixed capacity.
• Metrics own 28 counters regardless of traffic diversity.
• Ring and Maglev caches are bounded by validated configuration.
• The showcase renderer is unrelated to library runtime and caps particles and device pixel ratio in the browser.

How to benchmark your deployment

Measure at least:

1. candidate counts at p50, p95, and maximum;
2. eligibility ratios during healthy and degraded operation;
3. membership update frequency;
4. affinity key skew;
5. tracker contention at real worker counts;
6. policy construction and cache rebuild separately from steady-state pick;
7. complete readiness-to-response lifetime, not selection alone.

Use fixed seeds for comparable stochastic runs. Prevent the optimizer from removing decisions. Report allocator activity and tail latency, not only mean throughput.

scripts/bench.sh runs the in-tree benchmarks and accepts criterion’s baseline arguments, so --save-baseline before a change and --baseline after it reports the regression directly.

Deciding what counts as a regression

Criterion tests each benchmark independently, and this workspace measures dozens of points. Reading those labels one at a time produces a gate nobody can trust, for two separate reasons.

scripts/bench.sh --save-baseline before
# make the change
scripts/bench.sh --baseline before
node scripts/bench-regressions.mjs

bench-regressions.mjs applies two filters and calls a regression only what survives both.

Holm-Bonferroni across every comparison in the run. Testing many hypotheses at a five percent level means five percent of each, so the chance that at least one unchanged benchmark is labelled changed grows with the suite. The correction controls the family-wise rate instead, and Holm rather than plain Bonferroni because it is uniformly more powerful and no less valid.

An effect-size floor, defaulting to five percent. Round robin picks in about five nanoseconds; an impeccably significant one percent regression there is fifty picoseconds. Significance answers whether a difference is real, not whether it matters, and only the second question should stop a merge.

The second filter turns out to do most of the work, which was not the expectation going in. Comparing two runs of identical code on this machine produced 48 comparisons; the classifier here, testing criterion’s point estimates against their standard errors, found 39 significant at the uncorrected level and 35 still significant after Holm — with a median effect of 1.69% and nothing above 5%.

Neither the classifier nor criterion is crying wolf there. A hundred samples with tight variance genuinely resolve sub-two-percent differences between runs, caused by machine state rather than by the code, and a correction designed for the case where no difference exists cannot suppress differences that are real but irrelevant. Only a magnitude threshold can.

POISE_BENCH_ALPHA and POISE_BENCH_MIN_EFFECT override the defaults. Raising the floor is the honest response to a noisy machine; lowering alpha is not, and mostly buys silence rather than confidence. Unlike the mutation and model-checking wrappers, it applies no CPU quota, memory ceiling, or nice level: those bound a job whose cost is the problem, whereas here the measurement is the product, and throttling it would not produce a conservative number but a wrong one. A benchmark is bounded by running it on an idle machine. The script warns when the load average suggests otherwise and leaves the decision to the operator.

Interpreting results

A faster isolated pick is not necessarily a better production policy. A policy that reduces backend queueing or preserves cache affinity can save far more time than it spends selecting. Conversely, an O(1) table lookup can be a poor choice when membership changes faster than rebuild cost can be amortized.

Benchmark the control objective, not only the function call.

Security model

Poise processes control-plane membership, routing keys, weights, health outcomes, and service errors. Even without unsafe Rust or direct network I/O, those inputs can affect availability, resource use, and traffic isolation.

Trust boundaries

Validate external inputs before constructing policy configuration. Typed constructors reject zero weights, invalid percentages, invalid prime/table sizes, and arithmetic overflow; they cannot determine whether a syntactically valid value is operationally safe.

Memory safety

The workspace forbids unsafe Rust through a workspace lint. This reduces one class of defects; it does not eliminate denial of service, logic error, dependency compromise, starvation, or incorrect operational configuration.

Tower validates custom-policy indices before endpoint access. Snapshot and policy cache updates are staged so a partial failure cannot expose mixed state.

Algorithmic denial of service

Candidate count, ring points, Maglev table size, and outcome-window capacity must be bounded by configuration. Do not let one request choose these values.

Affinity keys can be attacker-controlled. A malicious key distribution may concentrate traffic even when the hashing implementation is deterministic and collision-safe. Bounded-load affinity limits prospective capacity but still requires atomic admission.

Complete hash collisions have defined deterministic behavior; they do not panic. Deterministic FNV-based hashing is used for stable placement, not as a cryptographic MAC or untrusted hash-table defense.

Topology isolation

Priority panic can deliberately broaden health eligibility. Review PanicMode as a security and isolation control:

• fail-closed preserves exclusion at the cost of availability;
• broader panic can route to unhealthy capacity;
• draining and operator opt-out remain excluded.

Do not encode tenant authorization as a load-balancer preference. A policy selects among candidates it is given; the caller must construct an authorized candidate scope first.

Sensitive data

Built-in metrics exclude backend IDs, affinity keys, endpoint indices, policy names, and error strings from labels. Optional tracing can expose numeric indices and fixed classifications.

Applications adding logs or traces should consider:

• affinity keys may contain user or tenant identifiers;
• backend identity may reveal internal topology;
• error text may contain protocol or customer data;
• full discovery snapshots can expose infrastructure inventory.

Prefer stable opaque IDs, sampling, and access-controlled diagnostic sinks.

Supply-chain controls

The repository enforces:

• Cargo.lock in CI;
• cargo-deny advisories, licenses, bans, and source policy;
• an independent RustSec advisory check;
• immutable GitHub Action commit pins;
• Dependabot updates for Cargo and Actions;
• protected release environments;
• OIDC trusted publishing after the first crate bootstrap;
• an explicit release kill switch.

The initial crates.io publication is exceptional because trusted publishing cannot claim a new crate name. Its short-lived token belongs only in the protected bootstrap environment and must be deleted afterward.

Vulnerability reporting

Do not open a public issue for a suspected vulnerability. Follow the private process in the repository security policy.

A useful report includes the affected crate and revision, threat model, minimal reproduction, observed impact, and whether the issue requires attacker-controlled membership, keys, weights, timing, or application code.

Security review checklist

• Are all externally derived sizes and weights bounded?
• Can an attacker force expensive membership rebuilds per request?
• Are affinity keys canonical, opaque, and free of secrets?
• Does topology panic cross an isolation boundary?
• Are retry and hedge counts bounded?
• Can cancellation strand capacity?
• Can an old generation update current health or load?
• Are diagnostic labels and logs cardinality-bounded?
• Are release dependencies pinned and reviewed?

Testing and verification

Poise treats behavioral laws as part of its public API. Ordinary unit tests cover examples, edge cases, reference vectors, distributions, concurrency, and transactional failures. The poise-core integration suite adds generated, shrinking property tests that exercise only public APIs.

Run the standard property suite with:

cargo test -p poise-core --test property_selection
cargo test -p poise-core --test property_affinity
cargo test -p poise-core --test property_topology

Each property runs 256 generated cases by default. The suite currently checks:

• every policy either selects an eligible in-bounds endpoint or returns the precise empty/ineligible error;
• least-loaded minimality, round-robin cycle coverage, and exact smooth weighted-round-robin conservation;
• exact seeded replay for stochastic, priority, and locality policies;
• affinity identity independence from candidate order;
• rendezvous addition and removal minimal-disruption laws;
• equivalence of weighted and unweighted rendezvous at equal weights;
• priority health/panic eligibility and fully healthy failover suppression;
• locality decisions preserving selected priority and topology metadata.

For a longer local or scheduled soak, set the Poise-specific case count:

POISE_PROPTEST_CASES=10000 cargo test -p poise-core --test property_selection

Shrinking is bounded at 10,000 iterations. Minimized failure seeds are written beside each integration test in a .regressions file. Those files are intentional repository artifacts: commit newly discovered seeds so every later run replays the failure before exploring novel inputs.

The property suite uses Proptest 1.11 with only its std feature. That release declares Rust 1.85 compatibility, matching the workspace MSRV. It is an exact development-dependency pin so a test-only dependency update cannot silently raise the supported compiler version.

Distribution sensitivity

A test asserting that counts fall inside a hand-picked band is a hypothesis test whose operating point nobody wrote down. The in-module distribution tests use bands about six standard deviations wide, which is the right shape — a false alarm is then about one in a billion, so the suite does not flake — but at their sample sizes six standard deviations is also five percent of the expected count, so a sampler biased by four percent passes.

tests/distribution_power.rs keeps the threshold and fixes the other half. The sample size is derived from the deviation the test claims to detect, and each assertion first checks that its own design is powerful enough to see that deviation.

Sizing carries an explicit power target. Placing the claimed deviation exactly on the rejection boundary detects it only about half the time, because the estimate is centred on the boundary and falls short as often as it clears it, so the sample must place that deviation Z + Z_power standard deviations out:

n >= margin * (Z * sqrt(p(1-p)) + Z_power * sqrt(q(1-q)))^2 / (q - p)^2

where q = p(1 + relative) is the deviation being detected. Z is 6.8 and Z_power is 1.645, a ninety-five percent one-sided detection target, which costs about 1.5 times the samples a fifty percent target would.

Each term carries the variance belonging to its own hypothesis: the null’s under the rejection threshold, the alternative’s under the power term. Using the null’s for both is tidier, and lands below the required count.

Sizing is checked against an exact one-proportion power calculation performed outside this repository. That check is what found the tidier expression sitting 0.15% to 0.66% short — always short, never over, which is how a stated sensitivity decays into a slogan. The size now carries a two percent margin, wider than the disagreement between standard forms, and a test holds it between the requirement and a tenth above it so the margin cannot grow to cover a real error. An underpowered test fails as loudly as a biased sampler, so sensitivity cannot decay quietly as the code around it changes. Two further tests check the checker, one supplying a sample too small and one a bias just past the claimed resolution.

Randomized policies are swept over a fixed set of seeds rather than one. A single seed makes a distribution test a regression test on one draw: a sampler biased for every seed but that one passes forever. Fixing the set keeps the run reproducible, which the mutation gate requires, while sampling several points of the seed space.

These tests are #[ignore] by default and run in release:

cargo test --release -p poise-core --test distribution_power -- --ignored

The samples run to millions. That costs tens of milliseconds optimized and about three seconds unoptimized, and the mutation campaign runs the unoptimized suite once per mutant, where three seconds would become half an hour of campaign time. The split keeps the sensitivity without charging every mutant for it.

Reference equivalence

The verification bar asks for a comparison against a reference implementation. For most policies here that would be a copy. A reference for rendezvous is argmax over per-candidate hashes, which is what the shipped code already is, and a test comparing an expression to itself proves only that it was typed twice.

The comparison has teeth where the shipped implementation is optimised away from its specification. RingHash, Maglev, and BoundedLoadRendezvous retain a table or a scratch buffer and rebuild it transactionally when membership changes. Their specification is that a decision depends on the candidates and the context and on nothing else, so the naive implementation of that specification is a policy with no history at all: reference_equivalence.rs drives one policy through twelve generations of churn and requires it to agree, across a key sweep, with a freshly constructed one.

That targets what caching can break and nothing else — a table surviving a membership change it should not have, a fingerprint that missed a weight, scratch reused across differently shaped inputs. None of those show up in a single call, which is why the property suite does not catch them.

Three policies are covered because three earn it. Round robin and smooth weighted round robin carry a cursor and per-identity credit, so their state is their behaviour and a fresh instance is a different policy rather than a reference for this one. The randomised policies advance an RNG per call and differ from a fresh instance by construction; their laws are distributional. The churn itself is checked, because a comparison against a cache nobody invalidated would hold trivially.

Claims about the tree

Every count in the prose is a claim with an expiry date nobody wrote down. This repository has shipped several past theirs: ten Loom models when there were thirteen, 240 test entry points when there were 259, 642 mutation sites when there were 705, and a performance chapter still calling criterion baselines a roadmap item after they had landed. Each was accurate when written, which is the whole difficulty — prose drifts silently, and stays readable while doing it.

scripts/check-docs-drift.mjs counts the derivable claims from the source and fails when the prose disagrees. A claim whose pattern no longer matches is also a failure, and so is a claim stated twice: a checker that quietly stops checking is worse than no checker, and a duplicated claim leaves a stale copy behind every correction.

It also resolves every local link and every repository path the prose names, skipping the ones git ignores. Generated output is named legitimately — the book builds to site/book/, which the prose says and a clean checkout does not contain — so checking its existence would make the gate answer differently on a developer machine than in CI, which is the one thing a gate must not do. The number skipped is printed rather than passed over quietly, since a gate that skips silently is indistinguishable from one that has stopped checking. Links in docs/ are already validated by the book check, but README and the root documents are outside it, and fenced code is stripped before that check runs — so a chapter telling a reader to run a script is precisely the claim nothing verified. That is the claim most likely to rot after a rename, and the one a reader hits first.

Facts stated in more than one place get checked against each other rather than against nothing. The required CI checks are one fact with three copies — the workflow defines the job names, scripts/protect-main.sh requires them, and releasing lists them — and renaming a job desynchronises the other two silently, at which point a required context that no longer reports blocks every merge or a job quietly stops gating. The comparison runs in every direction, including the one whose absence is silent: a job defined in a gating workflow that nothing requires still runs and still reports, so dropping it from the script and the chapter together would otherwise leave every remaining check agreeing with itself. The MSRV is one fact with five: the manifest declares it, a badge shows it, the README says it, the test matrix pins it, and protection names the resulting job. The manifest is the only copy a compiler enforces, so it is the source and the rest are compared to it.

Some claims cannot be derived by a script that does not run the campaign producing them. The recorded mutation results are checked for internal consistency instead — a breakdown that no longer sums to its own total is drift arithmetic can catch. Counts fixed at compile time, such as the metrics cardinality, are asserted in the crate that owns them rather than scraped, since that is where a change to them happens.

Property testing is one verification layer, not a replacement for mutation, fuzz, model-checking, benchmark, simulation, and coverage gates. The mutation policy and reproducible poise-core campaign are documented in mutation testing. Resource-bounded fuzz targets and the exhaustive in-flight concurrency model are covered in fuzzing and concurrency models.

Mutation testing

Poise requires every terminating, viable poise-core mutation to be caught. This is a behavioral gate, not a coverage percentage target: one unexplained survivor fails the campaign.

The campaign is reproducible with cargo-mutants 27.0.0:

scripts/mutants-core.sh

The wrapper enables all poise-core features and fixes the Proptest seed. Single-worker execution avoids sharing incremental compiler state between mutant copies, and is now the enforced default rather than a flag to remember. .cargo/mutants.toml owns timeouts and a small, reviewed list of equivalent mutations. Each exclusion has an adjacent proof explaining why the rewrite cannot change observable behavior. Do not add score-based or broad file exclusions to make a campaign green.

Resource bounds

A campaign rebuilds and retests the crate once per mutant, 705 times over. It is the most resource-hostile thing in this repository, and an unbounded run is capable of taking a workstation down hard enough to corrupt the git object store. The wrapper enforces its own limits; none of them depend on the caller passing a flag.

• Nested parallelism is capped. cargo-mutants runs --jobs cargo invocations, and each one spawns its own CARGO_BUILD_JOBS rustc processes. The live process count is the product of the two, and neither factor bounds the other, so their defaults multiply to nproc squared on a many-core machine. The wrapper pins one worker and eight build jobs, and clamps any larger --jobs request to a four-worker ceiling.
• Scratch directories stay off RAM-backed filesystems. cargo-mutants copies the source tree and the whole target directory per worker — currently 555 MB each — into TMPDIR. Where /tmp is a tmpfs, that default puts every copy in RAM and pushes roughly 390 GB of copy traffic through unreclaimable pages over a full campaign. The wrapper defaults TMPDIR to target/mutants-scratch and refuses to start if that path is on tmpfs or ramfs.
• Memory is capped with swap disabled. The campaign runs in a transient systemd scope with MemoryMax and MemorySwapMax=0. Measured peak is 1.4 GB per worker against a 4 GB allowance. Disabling swap is the load-bearing half: without it the kernel answers memory pressure by thrashing, which is what makes the whole machine unresponsive rather than just failing the campaign.
• CPU is capped and the campaign is niced, so an interactive session stays responsive. Isolation is preferred but not required: systemd-run --user needs a session bus that CI runners generally lack, so the wrapper probes for a usable scope and falls back to its remaining bounds with a warning when there is none. Set POISE_REQUIRE_ISOLATION=1 to make a missing cage a hard failure instead.
• A wall-clock backstop stops a wedged campaign. A full run takes roughly twenty minutes; the default ceiling is ninety. It interrupts first so cargo can report, then escalates to SIGKILL, and where a systemd scope exists a slightly later RuntimeMaxSec stops the entire cgroup — which is what actually covers descendants, since timeout only ever signals its direct child.

Exceeding the wall clock or the memory ceiling is reported as itself rather than as a mutation failure, so a limit that has genuinely become too tight for a growing crate is visible as a limit. POISE_MUTANTS_JOBS, POISE_MUTANTS_MAX_JOBS, POISE_MUTANTS_BUILD_JOBS, POISE_MUTANTS_MEMORY_PER_JOB_MB, POISE_MUTANTS_TIMEOUT, and POISE_MUTANTS_SCRATCH override the defaults. Raise one only after confirming the growth that needs it; the previous freeze is what these numbers are for.

For a faster follow-up after adding tests, preserve mutants.out and run:

scripts/mutants-core.sh --iterate

mutants.out is a local report and is intentionally ignored. Review its four outcome classes separately:

• caught means a test failed under the mutation;
• missed means a terminating viable mutation survived and fails the gate;
• timeout means a mutation changed the normally sub-second suite into a watchdog-detected hang;
• unviable means the mutation did not compile and is not scored.

The wrapper accepts cargo-mutants’ timeout-only exit status only when missed.txt exists and is empty. This is necessary for loop-control mutations in Maglev, ring hash, and the lock-free in-flight limiter: their defect is the nontermination detected by the watchdog. Baseline failure, any surviving terminating mutant, and all other tool failures remain nonzero.

Baseline

The 2026-08-04 initial inventory generated 656 mutation sites. Before this hardening pass, 417 were caught, 94 were missed, 23 timed out, and 122 were unviable: 81.6% of completed viable mutations were caught.

The hardened source and reviewed exclusions currently examine 705 sites. A complete campaign caught 528, missed zero, found 154 unviable, and detected 23 nonterminating mutants with the watchdog. The hardening work converted 82 of the original survivors into caught tests, proved 11 equivalent and documented them, and removed one performance-only Maglev cursor mutation by making overflow handling explicit. The watchdog outcomes remain deliberately visible rather than excluded.

Re-run the complete campaign whenever selection arithmetic, health/topology boundaries, hashing, cached membership, or load-tracker concurrency changes. Update this baseline only from a clean unmutated test run, and never describe a partial or filtered campaign as the new baseline.

Fuzzing and concurrency models

Poise has three structure-aware libFuzzer targets:

• policy_state_machine mutates backend order, health, weight, and load while exercising general, load-aware, affinity, and stateful policies;
• topology_state_machine generates priority, panic, and locality scenarios and checks every successful decision against candidate metadata;
• probe_pool interleaves probe recording, selection, rejection, and clock advance against the retention contract, checking that capacity holds, that no expired observation informs a decision, that every decision charges exactly one use, and that a rejected decision spends none.

Build the targets without executing them:

cargo +nightly fuzz build

The local smoke wrapper is intentionally resource constrained:

scripts/fuzz-smoke.sh

Each target defaults to 1,000 executions, a 256-byte maximum input, and a 128 MiB libFuzzer RSS watchdog. The wrapper additionally refuses to run without a systemd user manager that can place it in a cgroup with a 25% CPU quota, 192 MiB hard memory limit, no swap, low scheduler priority, and a 30-second process deadline. Set POISE_FUZZ_RUNS only when the machine has enough headroom. Sustained fuzzing belongs on an isolated worker with externally enforced CPU and memory limits; do not turn the local smoke wrapper into an unbounded time campaign.

AddressSanitizer remains enabled. LeakSanitizer alone is disabled because it cannot operate in ptrace-constrained containers.

The lock-free InFlight tracker and shared health state machines are compiled against Loom’s synchronization types and checked across modeled scheduler and memory-order interleavings:

scripts/model-check.sh

This is a separate build under cfg(loom); ordinary builds continue to use std::sync. The models verify in-flight limit and balance laws, single active probe admission, forced-status generation invalidation, passive-circuit failure accrual, bounded probe reuse under concurrent selection, and coherent rolling-window snapshots during concurrent record/clear operations.

Linearizability, not only invariants

Three of the probe-pool models ask a stronger question than the others. An invariant asks whether a bound was exceeded; linearizability asks whether what the threads observed could have happened at all — whether some sequential order of their operations, respecting each thread’s own program order, reproduces every observation against a naive reference implementation.

The distinction has teeth here. Two selectors that both report one use remaining have charged one decrement twice, which is a lost update. It totals two successes against a budget of two, so a counting invariant accepts it, and no sequential order produces it. Carrying the observed remaining-use count into the history is what makes that visible.

The search is exhaustive because these models issue at most a handful of operations. Two further tests check the checker itself, one supplying a history that must be rejected and one a history that must be accepted, because a checker that always answered the same way would pass every model above while proving nothing.

Model-checking bounds

Loom explores a model exhaustively, so its cost is combinatorial in the threads and shared operations a model contains. Today’s thirteen models all complete well inside loom’s default ceiling, but the failure mode of adding one more concurrent step is a state-space explosion rather than a gradual slowdown. The wrapper bounds that the same way the mutation gate is bounded, and for the same reason: an unbounded verification job is capable of taking its machine down.

• Build and test parallelism are capped. The test harness otherwise runs one test per core, so peak memory is per-model cost times core count. The wrapper caps concurrent models with RUST_TEST_THREADS and rustc parallelism with CARGO_BUILD_JOBS.
• LOOM_MAX_BRANCHES is set explicitly rather than left implicit. Exceeding it aborts the model loudly, so this bound cannot silently shrink coverage: a model that outgrows it fails until someone raises it deliberately.
• LOOM_MAX_PREEMPTIONS is deliberately not set. Bounding preemptions makes exploration cheaper by making it partial, and it does so silently. That trades away the property this gate exists to establish.
• Memory, CPU, and wall clock are capped inside a transient systemd scope with swap disabled. The wall clock interrupts first, escalates to SIGKILL, and is backed by a slightly later RuntimeMaxSec on the scope, which stops the whole cgroup rather than only the direct child.

POISE_LOOM_BUILD_JOBS, POISE_LOOM_TEST_THREADS, POISE_LOOM_MEMORY_MAX, and POISE_LOOM_TIMEOUT override the defaults.

Isolation is preferred but not required, and the three wrappers differ on this deliberately. scripts/model-check.sh and scripts/mutants-core.sh probe for a usable scope and fall back to their remaining bounds with a warning when there is none, because both run in CI and systemd-run --user needs a session bus that a runner may not provide — and a CI worker is already an externally limited sandbox. Set POISE_REQUIRE_ISOLATION=1 to turn a missing cage into a hard failure for those two.

scripts/fuzz-smoke.sh has no fallback and exits 78 without systemd-run. That is not an inconsistency: nothing in CI runs it, and a fuzz target is designed to run until something breaks, so an unbounded local fuzz run has no natural end to degrade toward.

Capability showcase

The site/ directory is a dependency-free GitHub Pages experience for engineers evaluating Poise. Its Invariant Orrery translates real verification layers into an inspectable symbolic system:

• closed orbits represent generated property laws;
• fractured forms represent mutations the suite must catch;
• braided paths represent Loom scheduler exploration;
• tiered planes represent the Rust compiler floor;
• dots traveling through the system represent ordinary behavioral examples.

The capability glyphs are independent of those proof states. Selection uses a directional hexagon, affinity an anchor, topology nested planes, health an aperture, discovery radiating snapshots, observability an eye, and runtime integration a bridge. Combining the two axes lets the replay show which kind of proof is exercising which kind of primitive.

Data contract

site/data/latest.json is schema version 1. The checked-in record is explicitly labeled recorded; it preserves the dated full-campaign baseline documented in mutation-testing.md. A GitHub Actions publication replaces it with a ci record generated by scripts/write-showcase-data.mjs.

The generator counts authored Rust test entry points, property laws, Loom model boundaries, and the four cargo-mutants outcome files. Pass, fail, and unavailable states come from the corresponding Actions steps. It never infers success from a missing file and never turns a partial mutation run into the recorded full baseline.

The Pages workflow uses continue-on-error only long enough to export and upload the evidence. Its final verification step still fails the job if ordinary, property, Loom, MSRV, or mutation verification failed. The deployment job runs even then so the broken proof remains visible in the orrery instead of leaving a stale green page.

The same Pages artifact includes this mdBook at site/book/. The Orrery and book share the palette and typography in site/styles/tokens.css; the book adds only its document-specific theme. A single revision therefore owns the visual explanation, the verification record, and the engineering guidance.

To inspect a generated record without replacing the checked-in baseline:

POISE_SHOWCASE_OUTPUT=/tmp/poise-verification.json \
  node scripts/write-showcase-data.mjs

Resource and accessibility contract

The visitor’s browser never runs Rust tests, fuzzers, mutation testing, or model checking. It only replays the bounded JSON record.

• Desktop rendering is capped at 420 particles and 1.5 device-pixel ratio.
• Tablet rendering is capped at 210 particles; small screens use 130.
• Reduced-motion rendering is static and uses at most 140 particles.
• Rendering pauses while the document is hidden or the orrery is offscreen.
• Resizing rebuilds a fixed-size particle array instead of growing it.
• Every canvas interaction has an HTML button and keyboard equivalent.
• Without canvas or JavaScript, the complete semantic atlas and proof ledger remain readable.

Run the site locally from the repository root with any static file server whose document root is site/. For example:

python3 -m http.server 4173 --directory site

Release engineering

Poise uses one version across six crates and publishes them in dependency order. Normal releases are pull-request driven; direct cargo publish from a maintainer laptop is an emergency procedure, not the default.

Current release state

The six crate names were claimed with version 0.1.0 on August 5, 2026, after the repository’s CI, package, Loom, property, and mutation gates passed. New versions remain gated on all of the following:

1. GitHub Actions, protected environments, branch protection, and private vulnerability reporting remain enabled. Branch protection requires a pull request and all nine checks, and exempts nobody; it does not require an approving review, because a single maintainer with write access cannot supply one. Item 3 is therefore a maintainer obligation rather than a mechanically enforced gate, and should be read as one.
2. The complete CI, MSRV, Loom, package, and mutation gates are green on the exact release commit.
3. The release PR’s SemVer decisions, archive contents, changelog, and API compatibility report receive maintainer review.
4. Each package publishes from the protected crates-io environment through a crates.io trusted publisher.

node scripts/check-release-metadata.mjs --publishable enforces the mechanical portion of this list.

Cargo metadata

The shared package identity is defined once:

[workspace.package]
version = "0.1.1"
edition = "2024"
rust-version = "1.85"
publish = true
license = "MIT OR Apache-2.0"
repository = "https://github.com/copyleftdev/poise-rs"
homepage = "https://copyleftdev.github.io/poise-rs/"

Every member inherits license, repository, and homepage. The release metadata gate rejects drift between the workspace and member manifests.

Repository bootstrap

Authenticate the intended GitHub owner, initialize the real checkout, then run:

gh auth login
scripts/bootstrap-github.sh copyleftdev/poise-rs
git push -u origin main
scripts/protect-main.sh copyleftdev/poise-rs

The script creates or configures a public repository, enables issues, discussions, and private vulnerability reporting, disables the wiki and projects, configures merge hygiene, and installs the intended topics. After the initial push, the protection script requires pull requests, resolved conversations, linear history, and the full CI/security check set, and exempts nobody from them. It does not require an approving review; see the note on the release gates above. Review the settings afterward.

Protect main with required pull requests and these CI jobs:

• Format, lint, and docs
• Documentation book
• Test / Rust stable
• Test / Rust 1.85.0
• Deterministic property laws
• Exhaustive scheduler models
• Package archives
• Licenses, advisories, bans, and sources
• RustSec advisory database

Allow GitHub Actions to create pull requests so Release-plz can maintain its release PR. Keep environment approval on crates-io; the bootstrap environment is retired after its token is deleted.

Completed crates.io bootstrap

Trusted publishing could not create the crate names for their first releases. The initial 0.1.0 versions were therefore published in dependency order with a short-lived crates.io token scoped to publish-new and publish-update.

The bootstrap-only GitHub workflow is not a normal release mechanism and must not be rerun. Dependent archives could not be verified against crates.io until their internal dependencies completed their first registry publication, so the bootstrap proceeded from poise-core through the dependency graph and verified each exact registry version before continuing.

After bootstrap, maintainers must complete this one-time handoff:

1. Configure each crate’s crates.io trusted publisher for this repository, .github/workflows/release.yml, and the crates-io environment.
2. Delete CARGO_REGISTRY_TOKEN from GitHub.
3. Delete or lock the retired crates-io-bootstrap environment.
4. Set the repository variable RELEASES_ENABLED=true.
5. Run the regular Release workflow manually once and confirm it is a no-op.

Regular releases then use GitHub OIDC and no long-lived registry secret.

Normal release flow

1. Merge Conventional Commits to main through green pull requests.
2. Release-plz updates or opens chore: release Poise.
3. Review SemVer decisions, API compatibility output, all manifest diffs, Cargo.lock, and CHANGELOG.md.
4. Merge the release PR without bypassing required checks.
5. Release-plz publishes unpublished workspace versions, creates one workspace tag, and creates one GitHub release.

All crates use the poise version group. A change that forces one dependent crate to bump keeps the affected workspace versions coherent.

Because every crate shares one version, the workspace publishes a single v0.1.1-style tag rather than six package-qualified ones. Release-plz has no notion of tagging a version group, so git_tag_enable and git_release_enable are disabled workspace-wide and re-enabled only on poise-core, which every other crate depends on. Enabling either flag for a second package would make that package contend for the same tag name.

Explicit bump hook

Automated release PRs are preferred. For an approved manual override:

node scripts/bump-version.mjs patch
node scripts/bump-version.mjs minor
node scripts/bump-version.mjs 0.2.0

The command refuses a dirty tree unless --allow-dirty is explicit, changes the workspace version and all internal registry requirements, refreshes Cargo.lock, and runs Cargo plus the release-metadata check. The pre-commit hook reruns the coherence check.

Failure and recovery

Crates.io versions are immutable and cannot be overwritten. If a subset of the workspace publishes, do not retag or retry an already published version. Fix the cause, let Release-plz detect the unpublished packages, and resume. Yank only when the published package is harmful; a yanked release remains downloadable by existing lockfiles.

Never print, upload, or persist registry tokens in build artifacts or logs.

Roadmap

Foundation

• Candidate, status, weight, selection, and policy contracts.
• Round robin, random, weighted random, least loaded, P2C, and rendezvous.
• Deterministic tests, generated policy invariants, and rustdoc examples.
• Criterion benchmarks and recorded baselines for poise-core selection and membership-change cost. Dispatch, health, and discovery remain unmeasured.
• Naming, licensing, security policy, MSRV policy, and governance before publication.

Dynamic systems

• Immutable versioned snapshots with keyed backend identity.
• Atomic snapshot publication and graceful draining.
• Smooth weighted round robin with identity-keyed state migration.
• Load trackers for in-flight requests and peak-EWMA latency.
• Rolling success-rate and overload penalties.
• Passive health and circuit breaking with bounded recovery probes.
• Executor-neutral active probe scheduling with threshold transitions.
• Group-relative success-rate outlier detection with safety caps.

Ecosystem integration

• Tower adapter with retained readiness, load tracking, and cancellation.
• Transactional static snapshot-to-Tower reconciliation.
• Runtime-neutral coalescing snapshot streams and Tower reconciliation.
• Optional Tokio active-health and discovery conveniences without making Tokio a core dependency.
• tracing spans and metrics with bounded-cardinality defaults.

Advanced policies

• Weighted rendezvous with proportional capacity and minimal reweight churn.
• Bounded weighted virtual-node ring hash with transactional rebuilds.
• Prime-sized, transactionally rebuilt Maglev lookup tables.
• Weighted rendezvous affinity with prospective concurrent-load bounds.
• Weighted priority tiers, overprovisioned failover, and panic thresholds.
• Health-adjusted locality weighting and cross-locality spillover.
• Retry and hedge selection that excludes already-attempted backends.
• Adaptive concurrency and capacity-aware routing.

Verification bar

Every stable policy should have:

• unit, property, and deterministic replay tests;
• distribution and disruption tests where applicable;
• criterion benchmarks across small and large backend sets;
• documented time, memory, and allocation complexity;
• behavior documented for empty sets, ineligible sets, ties, overflow, membership churn, and seeded randomness;
• comparison against a reference implementation, where the shipped code is optimised away from its specification. Cached and scratch-retaining policies are compared against a policy with no history; for a policy whose implementation is its specification, a reference would be a copy and is not required.

Prequal proposal

Status: partially implemented. The probe pool described under Probe pool contract exists in poise-core as ProbePool, including the candidate-aware selection described under Selecting against a candidate set. No Prequal policy type exists: the hot-cold lexicographic rule, the decision type, and the fallback behavior remain proposals. Nothing in this chapter has appeared in a released crate.

Prequal is proposed as the next selection family: a policy that chooses among replicas using asynchronously collected probes rather than candidate-attached load metrics. It would give the adaptive-concurrency and capacity-aware-routing roadmap entries the signal they need — the entries themselves describe deployments, and a primitive supplies the piece rather than closing them. What this chapter does not propose is set out under What this chapter is not proposing, which is worth reading first if the length here suggests otherwise.

Why probing changes the contract

Every existing Poise policy reads a signal the caller attached to a candidate. LeastLoaded and PowerOfTwoChoices compare LoadMetric samples; BoundedLoadRendezvous compares sampled concurrency against a computed capacity. All of them balance load.

The Prequal result is that load is the wrong quantity to equalize. Equalizing requests-in-flight across replicas of differing speed drives the slow replicas into their queueing regime while the fast ones idle. Latency is the quantity a caller experiences, and it should be minimized subject to a load cap rather than the other way around.

That inverts the signal path. The policy no longer reads a number hanging off a candidate; it reads a pool of recent observations gathered out of band, and each observation names a replica the policy may or may not still consider eligible. This is a new boundary, and it is the reason this proposal exists as a document before it exists as a type.

The hot-cold lexicographic rule

Given a pool of probes, each carrying a replica identity, a requests-in-flight count, and a latency estimate:

threshold = rif_quantile(pool)
cold      = { probe in pool : probe.rif <= threshold }

selection = if cold is non-empty { argmin latency over cold }
            else                 { argmin rif     over pool }

The cold branch optimizes latency among replicas with spare capacity. The hot branch degrades to load balancing precisely when no replica has spare capacity, which is the only regime where equalizing load is the correct objective.

The paper is emphatic that this lexicographic shape is doing the work, not the two signals on their own: hot-cold beat RIF-only control, and it beat every non-trivial linear combination of RIF and latency they tried. A weighted score is therefore not a simpler equivalent of this rule, and should not be offered as one. The ordering encodes a hierarchy rather than a trade-off — latency is worth optimizing, but a replica staying inside its memory allocation is a constraint, and constraints do not average with objectives.

The threshold must be computed by exact rank selection over the bounded pool, not by a floating-point quantile estimator. Poise policies are required to be reproducible and mutation-testable, and an estimator that drifts with accumulation order is neither. Latency comparison would use a total order in the style of LoadScore, which already resolves f64 comparison through total_cmp.

Note what the quantile is taken over. In the paper, a client maintains an estimate of the RIF distribution across replicas from recent probe responses, and Q_RIF is a fixed rank into that estimate. The adaptivity lives in the distribution, not in the rank. For a bounded pool this collapses to an exact rank over the pool’s own entries, which is the same computation Poise already requires for reproducibility — so the paper’s design and this repository’s determinism constraint agree here rather than conflict.

Parameters from the deployment

The paper’s operating points, recorded so future choices are made against evidence rather than invention. These are inputs to a decision, not a contract:

The last row is the one that matters most for this repository. Prequal does not configure a reuse budget; it computes one from the probing rate, the removal rate, the pool size, and the replica count, and randomly rounds a fractional result to preserve its expectation. ProbePoolConfig::max_uses is a raw constant by comparison. That is the right shape for a storage primitive — the pool knows none of those four quantities — but it means the number is only meaningful when something upstream derives it, and that derivation belongs with the policy rather than with a caller’s guess.

That derivation is partial, and any implementation must say what it does outside the domain. The expression is only defined where

n > m    and    (1 - m/n) * r_probe > r_remove

The first condition is the ordinary case — fewer pool slots than replicas — and the pool is pointless without it. The second is the real constraint: it says probes must arrive faster than the pool sheds them. It is not a remote edge. With the paper’s own m = 16, r_probe = 3, r_remove = 1, a replica count of 24 makes the denominator exactly zero, and any smaller fleet makes it negative. Reaching for max{1, ...} does not rescue this; a zero denominator is a division error before the maximum is ever taken, and a negative one silently returns a budget of 1 that is not the formula’s answer but the clamp’s.

The condition has an operational reading: at that probe and removal rate the pool cannot sustain itself, so no reuse budget makes it self-supporting. Reuse is not the lever there — r_probe must rise or r_remove must fall. A policy computing this must therefore validate the domain first and report the combination as a configuration error, in the manner of ProbePoolConfigError, rather than dividing and hoping. Depletion is then a tuning outcome the operator chose, visible through the fallback regime, and not an arithmetic accident.

Probe pool contract

The pool is the novel component and carries the obligations that make the rule safe. It is implemented as ProbePool, and the obligations below are its documented contract rather than an open design question. They are not the whole set: a further obligation, degradation control, is described after them and has no implementation yet.

• Bounded. The pool holds at most a configured number of entries. Insertion past that bound evicts, and no code path grows it, matching the existing prohibition on unbounded internal collections.
• Consumed on use. A probe informs at most a configured number of decisions and is then removed. This is not an optimization. A probe that reports an idle replica and is readable by every concurrent selector produces a stampede onto that replica, which is the classic failure of stale-information balancing. Bounded reuse is what makes the pool safe to share.
• Aged out. An entry older than a configured maximum is not eligible to inform a decision regardless of its use count.
• Not authoritative over eligibility. A probe naming a candidate that is draining, unavailable, or absent from the current membership generation is discarded at decision time. Health and discovery keep precedence; probing is a ranking signal layered on top of them, exactly as load is today.

Bounded reuse only holds if reading an observation and charging its budget are one indivisible step. ProbePool::decide_at therefore takes the ranking function rather than returning the slice: expiry, selection, and charging happen under a single lock, so two concurrent selectors cannot both spend the last use of one observation. That also fixes where eligibility filtering belongs — inside the caller’s ranking function, which is the only place that can see both the observations and the current candidate set.

Degradation, and the obligation still missing

The paper removes probes for three distinct reasons, and the pool as implemented addresses only two of them. Staleness is covered by the age limit, and depletion by bounded reuse. The third is degradation, and it is a selection-induced bias rather than a timing problem: selection consumes the probes reporting lightly loaded replicas first, so what accumulates in the pool over time is disproportionately the probes reporting heavily loaded ones. A pool that only ages and only consumes drifts toward describing the fleet as busier than it is, and the rule reads that drift as fact.

Prequal’s answer is to delete probes at a configured rate r_remove per query, alternating between two rules: remove the oldest, and remove the worst by the same ranking used for selection, run in reverse — the hot probe with the highest RIF if any probe is hot, otherwise the cold probe with the highest latency. Alternating is what makes one mechanism cover both staleness and degradation.

This is the one paper obligation with no counterpart in ProbePool, and it is load-bearing rather than an optimization: without it the pool develops exactly the bias the rule is least able to detect, because a uniformly pessimistic pool still looks internally consistent.

Removing the worst probe requires ranking, and the pool deliberately has no opinion about ranking. The resolution proposed here follows the shape decide_at already established: the caller supplies the ordering, the pool owns the mechanics and the lock. A removal entry point would take a ranking function, would apply it in reverse, and would drop under the same lock that governs selection and charging. No such method exists yet. The policy would keep the rule; the pool would keep retention. r_remove belongs to the policy layer for the same reason b_reuse does — it is denominated in queries, and the pool does not know what a query is.

Fallback threshold

An empty pool is not the only cold condition. The paper falls back to a uniformly random replica when the pool is empty, and reports that it is useful to invoke that fallback “whenever the pool occupancy drops below 2” — a pool holding a single probe offers no choice, so ranking it is a formality that launders one stale observation into a decision. ProbeDecisionError::NoProbes currently reports only true emptiness. The occupancy threshold at which a caller should prefer its fallback is a policy parameter, and the decision must report which branch produced it either way.

Determinism

Prequal would be the first policy whose inputs vary with wall-clock time, which puts pressure on the law that a seeded policy replays exactly.

The law is preserved by keeping the sampling discipline the existing policies already follow: decide takes one coherent view of the pool and reads it once, in the same way BoundedLoadRendezvous samples every eligible LoadMetric exactly once per decision. Given an identical pool state and an identical candidate slice, the selection is identical. Time enters through pool maintenance and never through the selection rule; every time-dependent pool operation has an _at form that takes the reading instant, so tests and simulations drive a synthetic clock rather than the wall clock.

Crate placement

The split follows the existing tracker and policy separation, where PeakEwma lives in load.rs and the policies that read it live in policy/:

The single rule this must not break: the paper couples probe rate to request rate, which requires a scheduler, and poise-core has no runtime. The core may only observe that probing is due; it may never drive it. Any design that places probe scheduling in the core forfeits the runtime neutrality that poise-core, poise-discovery, and poise-health currently guarantee, and should be rejected on that basis alone.

Decisions and errors

decide would return a decision exposing why the selection happened, following BoundedLoadDecision and PriorityDecision:

• selection: the chosen candidate;
• regime: whether the cold branch, the hot branch, or a fallback produced it;
• the number of pool entries that informed the decision;
• the selected entry’s requests-in-flight and latency, and the computed threshold.

A cold start is the interesting case. An empty pool is not an error, and inventing a PickError variant for it would misreport a healthy system that has simply not probed yet. The proposal is instead a fallback policy parameter, defaulting to PowerOfTwoChoices, with the fallback reported through regime so an operator can observe how often selection ran without probe data. Silent fallback would violate the standing requirement that outcomes stay distinguishable.

Empty and wholly ineligible slices keep the standard PickError distinction. Scratch growth returns StateCapacityExceeded, as elsewhere.

What a probe reports

The two fields are not the independent scalars their types suggest, and the server side owes more than reading two counters.

RIF is a counter read. The latency estimate is conditioned on it: when a query finishes, the paper’s server module records that query’s latency tagged with the RIF counter value at its arrival; answering a probe then reports the median of recent latencies at or near the current RIF. At moderate query rates those samples come entirely from the last few milliseconds.

So a probe answers “how fast is this replica right now, at the concurrency it is currently running at,” not “how long did the probe take.” ProbeReading documents its latency as the observed service time for the probe itself, which is a weaker and different claim — a reporter that measures probe round-trip time satisfies the type while violating the contract the rule depends on. The pool cannot enforce this, so it belongs in the documented obligation on whoever produces readings.

Divergence from the paper

Deliberate departures, documented as contract rather than left implicit:

1. Exact rank selection instead of an estimated quantile. Required for reproducible replay and for mutation testing to be meaningful. As noted above, over a bounded pool this coincides with the paper’s construction rather than opposing it.
2. No global assignment state. As with bounded-load affinity, Poise observes a borrowed snapshot and chooses one destination. It does not own the fleet-wide assignment, so it cannot claim the paper’s system-level results.
3. Reuse does not compensate. When a Prequal client sends a query to a replica it holds a probe for, it increments the RIF recorded on that probe, since it has just added load the probe predates. ProbePool charges the reuse budget and returns the reading unchanged. Their own note is that they would like to age the latency estimate the same way and do not, so this is a known-partial mitigation in the paper too. Compensation needs the rule’s units and belongs with the policy; until it exists, a reused probe overstates how idle its replica still is.
4. Eviction compares observation instants. A full pool drops the incoming observation when it is staler than everything retained, rather than always evicting the oldest resident. The paper simply drops the oldest, and does not discuss probes completing out of order. Ours is the stricter rule and costs nothing, but it is ours, not theirs.

The paper’s constants are a starting point and not a contract. Pool size, reuse limit, age limit, probe rate, and quantile must each be chosen deliberately, documented with their operational meaning, and justified against measurements rather than inherited — with the caveat that b_reuse is derived rather than chosen, and choosing it directly is already a departure.

Verification plan

Mapped onto the evidence table in testing:

The laws worth stating explicitly:

• selection is eligible and in bounds, or the precise error;
• if any probed replica is cold, the selection is cold and no cold replica has strictly lower latency;
• if every probed replica is hot, the selection has minimum requests-in-flight;
• raising the quantile never shrinks the cold set;
• the pool never exceeds its bound, and no entry outlives its reuse or age limit;
• under concurrent selection against one idle replica, no more probes are spent on it than the reuse limit permits, which is the property that separates this from naive stale-information balancing;
• a deliberately slow reference implementation of the rule, kept in the test support module, agrees with the optimized path on generated input.

The bound-and-expiry law and the concurrent-reuse law are pool properties and now hold. The probe_pool fuzz target drives arbitrary record, expire, and select interleavings under the limits in fuzzing, and the Loom models cover the concurrent reuse bound. Reference-implementation parity is listed with them, but it validates the selection rule rather than the pool, so it stays pending with the rest of the rule laws and lands with the policy.

Settled by the paper

Three questions this chapter opened with are answered by the source rather than left to taste.

The quantile is a fixed rank, and that is defensible. Q_RIF is a configured constant; what adapts is the estimated RIF distribution it indexes into. A library that does not own the fleet can hold the rank fixed and let the pool supply the distribution, which is what a bounded pool does by construction. Default to the paper’s range and treat 0 as a supported setting rather than a degenerate one, since it selects RIF-only control deliberately.

Probe issuance warrants its own type. A probe carries a RIF count and an RIF-conditioned latency estimate, runs against single-digit-millisecond timeouts, and in synchronous mode carries request information the replica reads. A health probe answers whether a replica should receive traffic at all. Sharing scheduling machinery with poise-health is reasonable; sharing the request and response type is not, and would couple eligibility to ranking in exactly the way architecture forbids.

The fallback is uniform random, and it triggers on occupancy rather than emptiness. That settles its behavior but not its placement. A type parameter defaulting to a probe-free policy keeps Prequal total and keeps the fallback observable through regime; explicit caller composition keeps the type simpler but makes silent fallback easy to write by accident. The former is proposed, on the standing requirement that outcomes stay distinguishable.

What this chapter is not proposing

Prequal, as the paper describes it, is a load balancer: probe scheduling tied to request rate, a reuse budget derived from fleet size, degradation control paced per query, sinkhole heuristics, and a synchronous mode that puts probing on the request path. Poise is not that, and reading the chapter end to end it would be easy to conclude otherwise, because describing a system faithfully means describing all of it.

The line is the same one the rest of the workspace draws. Poise ships primitives: a bounded store with a retention contract, and a selection rule that is a pure function of what it is handed. A balancer is something an application composes from those, and several questions that look open from inside the paper are already answered by that boundary.

Membership is the caller’s. ProbePool is generic over identity and stores what it is given. It is not authoritative over eligibility, and decide takes the ranking function rather than returning the slice, so the only code that sees both the retained observations and the current candidate set is the caller’s selector — which is exactly where an entry naming a departed replica gets dropped. Whether identity means a name or a discovery generation is a decision for whoever owns the membership, and poise-discovery already owns it. The pool needs no opinion, and giving it one would put a second membership model in the workspace.

Rates are the caller’s. A removal rate denominated in queries cannot live in something that never sees a query. The same is true of the reuse budget, which the paper derives from probing rate, removal rate, pool size, and replica count: the pool knows one of those four. It exposes the bound; whatever counts requests chooses it.

Failure detection is health’s. Sinkholing — a replica failing fast enough to look fast — is real, and it is not a probe-pool problem. Passive health, circuit breaking, and outcome windows already observe error rates, and probes are explicitly not authoritative over eligibility, so a sinkholing replica should be excluded before ranking ever sees it. Composing that correctly is the application’s job, and a primitive that quietly filtered on error rate would be duplicating a subsystem that already exists.

Scheduling is a runtime’s. Probe issuance, its coupling to request rate, and the synchronous variant all need a scheduler. poise-core has none and must not acquire one. Those belong in poise-tokio or above, and the rule must stay a function of a pool it did not fill.

What remains in scope is small and concrete: the hot-cold rule as a selection policy alongside the other thirteen, and the retention mechanics the rule depends on.

Selecting against a candidate set

Membership being the caller’s does not mean the composition should be the caller’s to get right unaided. decide_at hands a ranking function every retained observation, including ones naming replicas that have since drained or left, and filtering them is documented as the caller’s responsibility. That responsibility is silently omissible, and omitting it routes traffic to a departed replica — the exact hazard the retention contract warns about, with nothing in the signature to prevent it. Every caller then writes the same filter, and afterwards searches the candidate slice to turn the returned observation back into something dispatchable.

ProbePool::decide_among_at takes the candidate slice and does that work. It intersects unexpired observations with the candidates by identity, drops any that are not eligible, and offers the ranking function only what is left. The decision carries the chosen candidate’s position, so nothing maps back:

#![allow(unused)]
fn main() {
let decision = pool.decide_among(&candidates, |observed| {
    observed
        .iter()
        .enumerate()
        .min_by_key(|(_, entry)| entry.requests_in_flight())
        .map(|(index, _)| index)
})?;
dispatch(&candidates[decision.selection().index()]);
}

The closure now expresses only the ranking, which is the part that is actually the policy.

This is not the pool acquiring a membership model, which the section above rules out. It caches no candidate set, tracks no generation, and defines no eligibility of its own — it asks Candidate::is_eligible, which the caller’s type implements, and compares against the slice handed to it at the moment of the call. It owns no opinion; it declines to offer a footgun.

That also settles what looked like an open question about keying. Identity is compared at decision time against the slice the caller passed, so an entry naming a replica that has left is invisible without any keying scheme, generation tracking, or staleness rule. decide_at remains for callers who want the unfiltered view.

Cold start stays the caller’s to handle, deliberately. The paper falls back below an occupancy of two rather than at emptiness, and len_at reports occupancy so a caller can implement that threshold; the pool does not choose one.

The one open API question

Degradation is a retention property, so it belongs to the pool if it belongs anywhere here — selection consumes the observations reporting idle replicas first, and what accumulates is a pool describing the fleet as busier than it is. Removing the worst entry needs a ranking, which the pool does not have and should not acquire.

The shape that fits is the one decide_at and decide_among_at established: a removal entry point taking the caller’s ranking, applied in reverse, under the same lock. That is the only remaining addition to the pool’s surface this chapter proposes. Pacing it stays outside, for the reason above.

References

• Load is not what you should balance: Introducing Prequal, Wydrowski, Kleinberg, Rumble, and Archer, USENIX NSDI 2024.
• Architecture for the boundary this policy must respect.
• Load and feedback for the tracker contract it extends.