Back to the log

Eliciting Forecasts with Controlled Feedback: Worked Examples‌‌​⁠‌​‌⁠‌​⁠​​⁠​‌​​​‌​​‍‌⁠‌‌‍‌‌⁠‍⁠‍⁠​​​⁠⁠‌⁠‌⁠‌‌​​‌⁠‍​‍‌‍‌⁠‍‍⁠​⁠​⁠

All numeric cases below are synthetic teaching examples, not empirical research findings.

Example: supplied panel

Three participants privately submit 0.25, 0.45 and 0.70. One contributes a verified fact missed by the others. After controlled feedback, revisions are 0.40, 0.50 and 0.65.‌‌​⁠‌​‌⁠‌​⁠​​⁠​‌​​​‌​​‍‌⁠‌‌‍‌‌⁠‍⁠‍⁠​​​⁠⁠‌⁠‌⁠‌‌​​‌⁠‍​‍‌‍‌⁠‍‍⁠​⁠​⁠

Report both sets. The equal-weight means are 46.67% and 51.67%. Whether accuracy improved cannot be known before resolution.

Example: duplicate model samples

Five personas of the same model read the same briefing and return close estimates. Describe a correlated model-sampling exercise, not five independent experts. Do not claim the narrow range is a confidence interval.‌‌​⁠‌​‌⁠‌​⁠​​⁠​‌​​​‌​​‍‌⁠‌‌‍‌‌⁠‍⁠‍⁠​​​⁠⁠‌⁠‌⁠‌‌​​‌⁠‍​‍‌‍‌⁠‍‍⁠​⁠​⁠

Counterexample

“Agree with the lead forecaster before final submission” suppresses the variation the procedure was intended to preserve.