Registration for a live webinar on 'Precision medicine treatment for anticancer drug resistance' is now open.
See webinar detailsWe noted you are experiencing viewing problems
-
Check with your IT department that JWPlatform, JWPlayer and Amazon AWS & CloudFront are not being blocked by your network. The relevant domains are *.jwplatform.com, *.jwpsrv.com, *.jwpcdn.com, jwpltx.com, jwpsrv.a.ssl.fastly.net, *.amazonaws.com and *.cloudfront.net. The relevant ports are 80 and 443.
-
Check the following talk links to see which ones work correctly:
Auto Mode
HTTP Progressive Download Send us your results from the above test links at access@hstalks.com and we will contact you with further advice on troubleshooting your viewing problems. -
No luck yet? More tips for troubleshooting viewing issues
-
Contact HST Support access@hstalks.com
-
Please review our troubleshooting guide for tips and advice on resolving your viewing problems.
-
For additional help, please don't hesitate to contact HST support access@hstalks.com
We hope you have enjoyed this limited-length demo
This is a limited length demo talk; you may
login or
review methods of
obtaining more access.
Printable Handouts
Navigable Slide Index
- Introduction
- The series
- Review
- Coming up
- Part 3: outline
- Aggregation
- Multiple experts
- Aggregating expert judgements
- Linear opinion pool
- Cooke’s classical method
- Behavioural aggregation
- But there are more psychological hazards
- How many experts?
- What does an aggregated distribution mean?
- Multiple experts in SHELF
- The SHELF approach
- Case study 1
- Case study 1: excerpt of SHELF1
- Case study 1: SHELF2
- Individual expert judgements
- Fitting
- Group discussion
- Consensus judgements
- Fitted distribution
- SHELF methods
- Methods for individual judgements
- Tertile and quartile methods (1)
- Tertile and quartile methods (2)
- Roulette method
- Example: uninsured drivers
- Preferred method
- Probability method
- Group judgements
- Do experts have to meet?
- Recap
- References
Topics Covered
- SHELF
- Mathematical and behavioural aggregation
- Group judgements
- The rational impartial observer
- Eliciting a single distribution from multiple experts
- Concept of the rational impartial observer (RIO)
- Multiple experts in SHELF
- SHELF methods
- Tertile, quartile, and roulette methods for individual judgements
- Probabilities method for group judgements
Links
Series:
Categories:
External Links
Talk Citation
O'Hagan, A. (2023, March 30). Expert knowledge elicitation with SHELF: multiple experts [Video file]. In The Biomedical & Life Sciences Collection, Henry Stewart Talks. Retrieved December 30, 2024, from https://doi.org/10.69645/LHJF1314.Export Citation (RIS)
Publication History
Financial Disclosures
- Tony O'Hagan acts as a consultant providing training and advice on the use of SHELF.
Other Talks in the Series: Expert Knowledge Elicitation with SHELF
Transcript
Please wait while the transcript is being prepared...
0:00
Welcome to this third
talk in the series,
'Expert Knowledge
Elicitation with SHELF.'
This one is entitled
Multiple Experts.
0:11
There again is the
rundown of the course
and we are in the third talk
here, multiple experts.
In the last talk, number two,
we did one distribution
with one expert,
and now we're going to move
onto multiple experts,
which is the major area
where we actually use SHELF.
The final talk will be
some details of skills and
resources that we might do.
0:34
But before we do so,
let's look back at
part 2. In part 2 -
One Distribution,
One Expert - we found out about
challenges for subjective
probability judgements,
primarily in the form of
psychological heuristics that
can lead to biased judgements.
We saw how SHELF protocol
addresses those challenges.
We have a carefully
constructed sequence of
questions designed to minimise
the potential for bias.
We did that for one expert,
and now we're going to move
on to multiple experts.
1:05
So in part 3, we'll consider
the general problem of
eliciting a single distribution
from multiple experts.
We have to consider then how
to aggregate the experts
together using
either mathematical or
behavioural aggregation.
We'll see how SHELF uses
behavioural aggregation
to do this with
some specific new concepts
like the rational
impartial observer,
we'll deal with those.
Then we'll compare
the various SHELF
methods for eliciting
judgements from individual
experts and from the group.
1:36
This part will have
three parts to it,
the aggregation part,
multiple experts and
the SHELF methods.