The Classical Model (CM) is a performance-based approach for mathematically aggregating judgements from multiple experts, when reasoning about target questions under uncertainty. Individual expert performance is assessed against a set of seed questions, items from their field, for which the analyst knows or will know the true values, but the experts do not; the experts are, however, expected to provide accurate and informative distributional judgements that capture these values reliably. Performance is measured according as metrics for each expert’s statistical accuracy and informativeness, and the two metrics are convolved to deter-mine a weight for each expert, with which to modulate their contribution when pooling them together for a final combined assessment of the desired target values. This chapter provides mathematical and practical details of the CM, including describing the method for measuring expert performance and discussing approaches for devising good seed questions.
|Title of host publication||Elicitation|
|Subtitle of host publication||The Science and Art of Structuring Judgement|
|Editors||Luis Dias, Alec Morton, John Quigley|
|Place of Publication||New York|
|Publication status||Accepted/In press - 19 Jun 2017|
- classical model
- seed questions