Pre-conference workshop II: A Practical Introduction to Standard Setting
Facilitated by
Dr Lee Coombes
Director of Psychometrics and Assessment Development at Kaplan
Lee is currently Director of Psychometrics and Assessment Development with Kaplan, and is currently working on national licensing exams after spending time in assessment roles with both Peninsula Medical School and Cardiff School of Medicine.
Workshop description
Standard setting is a key part of any assessment, so the choice and use of a standard setting method therefore has to be appropriate and fit for purpose. For our assessment’s outcomes to be valid, how we define the minimal acceptable level of competence will change depending on the format of the assessment, its stakes, and the resources we have at our disposal.
This workshop aims to introduce participants to a range of common methods (including Angoff, Ebel, Cohen, Hoftsee, Borderline Regression), along with key concepts such as precision and error, and how they can be used to provide confidence that we have set a defensible standard. It also focuses on the practical application of standard setting methods so participants can use what they learn to set their own standards with confidence.
No prior knowledge or experience of standard setting is required.
Workshop format
I will be preparing some group activities, where those attending carry out some angoff, ebel and hoftsee work, but this can only go ahead if there are enough people there. If we do not have the numbers then these become a facilitated group discussion. Most likely is that it will be a mix of both (as was the case in Braga).
Objectives
- Understand the difference between absolute and relative standards
- Understand the difference between data driven and data independent standards
- Understand the relationship between reliability, the SEm, and confidence
- Learn which method apply to which assessment types
- Learn how to carry out Angoff, Ebel, Cohen, Borderline Regression, Borderline Groups, Contrasting Groups, Hoftsee, and normative methods.
Outcomes
Participants should be able to return and use these methods in their own institutions.
To support that they will receive a copy of the slides, plus the excel spreadsheets I use during the workshop to calculate standards.
Additional information
- Maximum number of participants: 20
email Lee Coombes: drleecoombes@gmail.com
Click here to book your ticket for this Pre-conference workshop
Check out Pre-conference workshop I:Designing and implementing programmatic assessment: does one size fit all?
Check out Pre-conference workshop III: Stimulating the study process with rules and regulations of assessment