Performance model

Poise documents algorithmic cost and allocation behavior, and does not publish unsupported throughput claims. Criterion baselines now exist for poise-core selection and membership-change cost, and scripts/bench.sh reproduces them. Dispatch, health, and discovery remain unmeasured.

Read the recorded numbers for shape rather than magnitude. The constants belong to one machine and one compiler; how a policy responds to candidate count and to churn is the part that transfers to a different deployment.

Selection cost

n is candidate count, p ring-point count, and m Maglev table size. Eligibility density and membership churn influence constants materially.

The two cached rows are stated carefully because their headline is easy to misread, and this table previously did. A Maglev table lookup is O(1) and a ring lookup is O(log p), but neither is what a call to pick costs: every safe pick first validates the cached table against the candidate slice, which is O(n). Measured cost therefore grows with candidate count for both, and the constant-time lookup is not the term that dominates.

This does not make them slow. Validating a candidate is far cheaper than hashing one, so at 512 members Maglev picks in about a fifth of the time plain rendezvous takes, and about a fifteenth of bounded-load rendezvous. The correction is to the asymptote, not to the ranking: a cached policy is cheap because its per-candidate work is small, not because it avoids per-candidate work.

Recorded baselines

Median pick time from benches/selection.rs, 100 samples per point, on an AMD Ryzen Threadripper PRO 5975WX with rustc 1.97.1. Reproduce with scripts/bench.sh.

Every candidate is eligible in these runs, which is the cheapest case for the policies that scan until they find an eligible member and the most expensive for those that must consider everything. Round robin is the one whose measured shape differs from its worst case for that reason: it is flat here because the next candidate is always eligible, and its O(n) row describes a slice where that is not true.

Everything else tracks candidate count, and the affinity policies separate by constant rather than by shape: rendezvous hashes every candidate, while ring hash and Maglev only validate every candidate and then consult a table. That constant is the whole practical difference between them at 512 members.

Membership-change cost

Cached affinity policies compute a membership fingerprint from the fields that affect their structure. An unchanged membership reuses the table. Relevant identity, eligibility, order, or weight changes rebuild transactionally.

Rebuild failure preserves the previous live cache. Callers should expose rebuild errors and avoid retrying them for every request without backoff or configuration repair.

Measured cost of a pick that triggers a rebuild, from benches/membership.rs, which flips one member in and out on every iteration:

Three things follow, and none of them are visible from the complexity column alone.

Ring hash and Maglev trade places. Ring rebuild is O(p) in ring points, which scale with members, so it grows with the fleet. Maglev rebuild is O(m) in a table size fixed by configuration, so it is nearly flat and its cost at eight members is almost its cost at five hundred. Ring hash is an order of magnitude cheaper on small fleets and several times more expensive on large ones; on this machine they cross somewhere between 64 and 512 members. A deployment choosing between them on churn cost should know which side of that crossing it sits on.

Rebuild dwarfs selection. A 512-member ring rebuild costs roughly three thousand times a steady-state pick at the same size. Rebuilds are amortized across the requests that follow a membership change, so this is not a per-request cost — but it is the cost of a rollout, and a deployment whose membership changes faster than it can amortize a rebuild has chosen the wrong policy, whatever its lookup complexity says.

Smooth weighted round robin has no rebuild penalty at all. Its churn numbers match its steady-state numbers, because identity-keyed state migrates incrementally rather than being rebuilt. That is the property the table calls identity-keyed state migration, and it is the reason it appears here despite caching nothing to rebuild.

Dispatch cost

poise-tower::Balance::poll_ready scans endpoint readiness in O(n) and performs no allocation. A ready service retains its reservation until selected.

call performs the policy’s normal selection cost, acquires the endpoint load guard, and returns a concrete response future without boxing it.

Snapshot reconciliation is control-plane work. Unchanged service generations retain service, readiness, and load state; new generations are built before commit.

Atomic costs

InFlight uses atomic updates for reservation and release. PeakEwma couples atomic concurrency with synchronized estimator state. Metrics uses relaxed atomic saturating counters.

Contention depends on how policy and tracker instances are shared. One global policy mutex may dominate every algorithmic difference in the table above.

Memory bounds

• Discovery snapshots own one immutable membership vector per live revision retained by readers.
• Outcome windows have configured fixed capacity.
• Metrics own 28 counters regardless of traffic diversity.
• Ring and Maglev caches are bounded by validated configuration.
• The showcase renderer is unrelated to library runtime and caps particles and device pixel ratio in the browser.

How to benchmark your deployment

Measure at least:

1. candidate counts at p50, p95, and maximum;
2. eligibility ratios during healthy and degraded operation;
3. membership update frequency;
4. affinity key skew;
5. tracker contention at real worker counts;
6. policy construction and cache rebuild separately from steady-state pick;
7. complete readiness-to-response lifetime, not selection alone.

Use fixed seeds for comparable stochastic runs. Prevent the optimizer from removing decisions. Report allocator activity and tail latency, not only mean throughput.

scripts/bench.sh runs the in-tree benchmarks and accepts criterion’s baseline arguments, so --save-baseline before a change and --baseline after it reports the regression directly.

Deciding what counts as a regression

Criterion tests each benchmark independently, and this workspace measures dozens of points. Reading those labels one at a time produces a gate nobody can trust, for two separate reasons.

scripts/bench.sh --save-baseline before
# make the change
scripts/bench.sh --baseline before
node scripts/bench-regressions.mjs

bench-regressions.mjs applies two filters and calls a regression only what survives both.

Holm-Bonferroni across every comparison in the run. Testing many hypotheses at a five percent level means five percent of each, so the chance that at least one unchanged benchmark is labelled changed grows with the suite. The correction controls the family-wise rate instead, and Holm rather than plain Bonferroni because it is uniformly more powerful and no less valid.

An effect-size floor, defaulting to five percent. Round robin picks in about five nanoseconds; an impeccably significant one percent regression there is fifty picoseconds. Significance answers whether a difference is real, not whether it matters, and only the second question should stop a merge.

The second filter turns out to do most of the work, which was not the expectation going in. Comparing two runs of identical code on this machine produced 48 comparisons; the classifier here, testing criterion’s point estimates against their standard errors, found 39 significant at the uncorrected level and 35 still significant after Holm — with a median effect of 1.69% and nothing above 5%.

Neither the classifier nor criterion is crying wolf there. A hundred samples with tight variance genuinely resolve sub-two-percent differences between runs, caused by machine state rather than by the code, and a correction designed for the case where no difference exists cannot suppress differences that are real but irrelevant. Only a magnitude threshold can.

POISE_BENCH_ALPHA and POISE_BENCH_MIN_EFFECT override the defaults. Raising the floor is the honest response to a noisy machine; lowering alpha is not, and mostly buys silence rather than confidence. Unlike the mutation and model-checking wrappers, it applies no CPU quota, memory ceiling, or nice level: those bound a job whose cost is the problem, whereas here the measurement is the product, and throttling it would not produce a conservative number but a wrong one. A benchmark is bounded by running it on an idle machine. The script warns when the load average suggests otherwise and leaves the decision to the operator.

Interpreting results

A faster isolated pick is not necessarily a better production policy. A policy that reduces backend queueing or preserves cache affinity can save far more time than it spends selecting. Conversely, an O(1) table lookup can be a poor choice when membership changes faster than rebuild cost can be amortized.

Benchmark the control objective, not only the function call.