José Miguel Pêgo, School of Medicine, University of Minho, Portugal
Workshop description
Multiple-choice items are the current standard for assessment in the cognitive domain of medical education. Despite being an efficient validated method, the development of each item is, still, a complex and time-consuming and, for that reason, a very expensive process.
With the increasing demand for large number of multiple choice items by medical schools, the current approach to item development struggles to meet this high demand. An innovative solution to this challenge is the use of automatic item generation (AIG) to develop new multiple-choice items.
AIG uses computer technology to generate new questions based in cognitive and item models, created by content specialists. While full automated item generation demands the access to IT resources (hardware and manpower) it is possible to develop simpler generators with household tools that are available in every personal computer. The critical step is the development of a cognitive model and the definition of its parameters that can be done on a simple piece of paper.
Attendees will be given the opportunity to work through development of cognitive models and program an excel worksheet to accomplish the algorithm of decision within a set of items.
AIG is a relatively new and lesser known technology, that requires a lot of development and further validation, but, with no doubt, it will be a technology that will shape the future of item development in medical education.
Objectives
- To introduce the concept of AIG and cognitive models
- To provide
- To provide a framework that participants can take to use in their own institutions
References
Gierl MJ, Lai H, Turner SR. Using automatic item generation to create multiple-choice test items. Med Educ. 2012;46(8):757-765. doi:10.1111/j.1365-2923.2012.04289.x.
Pugh D, De Champlain A, Gierl M, Lai H, Touchie C. Using cognitive models to develop quality multiple-choice questions. Med Teach. 2016;38(8):838-843. doi:10.3109/0142159X.2016.1150989.
Paniagua MA, Swygert KA. Constructing Written Test Questions For the Basic and Clinical Sciences. 2016. http://www.nbme.org/publications/item-writing-manual.html.