What is EBMA European Knowledge Testing (EKT)?
The EKT covers all relevant aspects of basic and clinical sciences as well as behavioural sciences and statistics. The EKT is currently available in English and consists of 200 multiple-choice questions, each with one best answer accepted as correct.
The difficulty of the test and cut scoring are tailored to the knowledge level European students should have at the time just prior to receiving their Doctor of Medicine (or equivalent) degree.
Test design and construction
EBMA understands that good quality assurance around item writing and reviewing is paramount in yielding assessment tools with high levels of validity and reliability. Our assessment tools, including the EKT, are written and carefully peer-reviewed by specialists in Europe.
Test composition
Test composition was established by a survey of EBMA board members who developed a blueprint compatible with the blueprint of the International Progress Test. This blueprint comprises key tasks that doctors commonly perform, such as understanding normal and abnormal processes, identification of clinical findings, diagnosis, the prevention and therapy of diseases and community-related aspects. Test assembly is also guided by system-based categories and traditional medical disciplines. Great care is taken to ensure the EKT will mirror current epidemiology in different parts of Europe, although local customization is possible.
Standard setting
Passing standards for the EKT are calculated using the Angoff method. This procedure involves the participation of the board members who assess the importance of each item for a recently graduated doctor. In the Angoff procedure for the EKT, the intraclass correlation
coefficient was 0,95. This is an indication of the high degree of convergence in the opinions of the participating specialists, and evidence of the robustness of the cut scoring procedure.
State-of-the-art measurement
From test construction to score calculations, EKT uses modern psychometric techniques to ensure a high degree of validity and reliability. Standardized scaled scores are calculated using best modelling practices so that they can retain their meaning throughout time and be
comparable between students, institutions and countries. In recent years, the EKT applications in several European countries has consistently yielded reliability coefficients above 0,90.
Specific psychometric analyses are routinely performed on EKT data to evaluate not only the difference in scores but the differences in the item parameters across participating institutions. These analyses are also known as measurement invariance studies. This is done not only to ensure usefulness of scores but also to unveil the potential impact of cross-national differences in laws or guidelines, cross-cultural differences in the propensity to prescribe or intervene, and cross-institutional differences in curricula. EKT pursues a high level of
compliance to the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014), making the psychometric rigor of international large-scale assessment programmes accessible at a feasible, affordable application in your institution.
Feedback reporting to students
Students receive detailed feedback in individual reports containing the student’s full test score and several subscores, according to cluster, category, discipline. The standardized score is determined by the estimated score distribution on the population and informs the position of the test taker in comparison to the peers.
Feedback reporting to schools
Deans and other stakeholders receive detailed reports of the students’ performance benchmarked against a European reference group.
Benefits
Applying the EKT can benefit your students and your school in several ways:
• a rich source of feedback for students and schools via an online interactive graphical report;
• students can present their score report letters as an evidence of performance for prospective employers;
• benchmarking your institution against European performance levels can be a tool for quality assurance, helping on the identification of priorities for curricular governance and faculty development.