Back to the log

Estimating Base Rates with the Outside View: Worked Examples‌‌​⁠‌​‌⁠‌​⁠​​⁠​‌​​​‌‍​‍⁠​‍‌⁠‍​‌‍​⁠‍⁠‌‌‌‍​​​‍⁠‌‍⁠​‌​​⁠‍​⁠‌‍​​‍​​‌

All numeric cases below are synthetic teaching examples, not empirical research findings.

Example: on-time launches

A supplied, complete dataset contains 12 on-time launches among 40 comparable attempts: empirical rate = 30%. Under an explicitly chosen Beta(1,1) prior, the next-trial predictive mean is 13/42 = 30.95%. These are different quantities.‌‌​⁠‌​‌⁠‌​⁠​​⁠​‌​​​‌‍​‍⁠​‍‌⁠‍​‌‍​⁠‍⁠‌‌‌‍​​​‍⁠‌‍⁠​‌​​⁠‍​⁠‌‍​​‍​​‌

A narrower class contains 4/8 on-time launches. Do not declare 50% the better estimate merely because it is more optimistic; explain the matching rule and limited sample.

Example: absent events

There were zero incidents among 10 comparable windows. The descriptive rate is 0%; that does not establish impossibility. A Beta(1,1) illustration gives 1/12 = 8.33%, with substantial model and sampling uncertainty.‌‌​⁠‌​‌⁠‌​⁠​​⁠​‌​​​‌‍​‍⁠​‍‌⁠‍​‌‍​⁠‍⁠‌‌‌‍​​​‍⁠‌‍⁠​‌​​⁠‍​⁠‌‍​​‍​​‌

Counterexample

“We are uniquely capable, so historical overruns do not apply” is not evidence of a different reference class. Identify a measurable difference and test whether it predicts outcomes.