Alternative evaluation methods in medical education
Marjo Wijnen-Meijer 11 TUD Dresden University of Technology, Medical Faculty and University Hospital Carl Gustav Carus, Institute of Medical Education, Dresden, Germany
Editorial
Quality assurance and the research and development of teaching are legally required and central components of higher education didactics. This is particularly true in medical education, which is shaped by specific demands and practice-oriented learning processes.
Feedback plays a crucial role in systematically capturing teaching quality, optimizing learning processes, and supporting both teachers and students in their development [1]. Two main types of feedback can be distinguished: feedback to teachers, which reflects their teaching quality and identifies areas for improvement, and feedback to students, which evaluates their performance and progress.
This article focuses on feedback to teachers, which is essential for the development of teaching and the maintenance of teaching quality.
Traditionally, universities use standardized student evaluations (SETs) in the form of standardized questionnaires, which are widely used to assess the quality of courses and teaching performance because of their simple handling and comparability [2]. However, these approaches often capture only snapshots and are limited in significance. In the medical context, where clinical exercises, simulations, and interprofessional learning play a central role, such methods often fail to adequately capture the complex teaching and learning processes. Numerous studies have shown that SETs are often strongly influenced by factors such as student expectations, gender, or ethnicity, and therefore may not always provide an objective assessment of teaching quality [3], [4]. At the same time, external factors such as course size or exam stress can influence the results [5], making them more of a “satisfaction analysis” rather than a true teaching evaluation [6].
In this context, the development and implementation of alternative evaluation approaches is becoming increasingly important. Methods such as realistic evaluation, as well as focus groups or peer feedback, allow for a more differentiated view of the contexts, dynamics, and effects in medical teaching situations.
The realistic evaluation, developed by Pawson and Tilley, focuses on understanding how programs work in specific contexts and the mechanisms that trigger these effects [7]. This method is particularly useful in medical education, as it not only examines outcomes but also the underlying conditions and processes. It is based on the principle: “What works, for whom, in what circumstances, and how?”. In teaching, this approach could represent a shift from purely summative assessments to analyzing the underlying learning processes and contexts. For example, one could investigate why a particular simulation training is successful for some students but not for others. The method requires both qualitative and quantitative data, such as interviews, observations, or performance assessments. Therefore, it is especially useful for studying complex teaching settings, as they occur in medical education.
In this context, realistic evaluation can be applied to analyze the effects of training programs, teaching methods or curriculum changes. The underlying core principles include mechanisms, contexts, and outcomes.
Mechanisms refer to the processes or underlying factors that explain how an intervention works. In a medical simulation, for instance, the mechanism might be the opportunity for learners to practice decision-making in a realistic environment.
Contexts involve the conditions or environments in which the intervention takes place. For example, learner engagement might depend on the availability of instructors or the cultural norms within the institution.
Finally, outcomes are the results or changes of the intervention. These might include improved clinical skills, enhanced teamwork, or increased self-confidence among participating students.
Insights from realistic evaluations help prepare instructional interventions to different types of learners and underlying contexts. In medical education, this could mean adapting teaching strategies for different groups, such as international students, or adjusting clinical instruction to the available resources at a hospital.
Peer observation can also be a valuable source of feedback, highlighting strengths and areas for development in teaching [8]. This can be done using an observation guide, although it is not always necessary. The key is to clearly define the observational focus in advance.
Furthermore, focus groups can be a valuable addition to other evaluation methods in medical education, such as student evaluations. They provide an opportunity to collect qualitative student feedback and offer a more holistic perspective on teaching quality. However, the main disadvantage is that focus groups are time-consuming and require facilitation experience to ensure productive discussions [9]. Despite these challenges, focus groups can be a valuable tool for assessing and improving medical education, as they provide detailed, qualitative insights, encourage communication between students and instructors, and can contribute to more effective, student-centered teaching design [10].
Finally, student learning outcomes are also a useful feedback source for instructors [11]. This can be achieved through the analysis of exam results in the traditional sense, but also by using quiz questions during lectures to gain a current understanding of what students have comprehended and where knowledge gaps still persist.
In this issue, various didactic interventions are evaluated. The article by Papan et al. describes the effect of an elective course on antimicrobial stewardship (AMS) [12]. The article by Ruck et al. evaluated the long-term effect of a smoking cessation counselling course using online questionnaires [13]. Tom Dreyer and colleagues compared the effect of two different teaching formats based on examination results [14]. Junga et al. compare the evaluations of compulsory and elective clerkships [15]. Brinkmann et al. have evaluated Mind-Body Medicine courses at two different universities using questionnaires and focus groups [16]. And Hofhansi et al. describe students’ experiences with supervision, which were collected using questionnaires [17].
Enjoy reading this interesting issue!
Author’s ORCID
Marjo Wijnen-Meijer: [0000-0001-8401-5047]
Competing interests
The author declares that she has no competing interests.
References
[1] Ding C, Sherman H. Teaching effectiveness and student achievement: Examining the relationship. Educ Res Quart. 2006;29(4):40-51.[2] Hornstein HA. Student evaluations of teaching are an inadequate assessment tool for evaluating faculty performance. Cogent Educ. 2017;4(1):1304016. DOI: 10.1080/2331186X.2017.1304016
[3] Arnold IJM. Do examinations influence student evaluations? Int J Educ Res. 2009;48(4):215-224. DOI: 10.1016/j.ijer.2009.10.001
[4] Heffernan T. Sexism, racism, prejudice, and bias: A literature review and synthesis of research surrounding student evaluations of courses and teaching. Assess Eval High Educ. 2022;47(1):144-154. DOI: 10.1080/02602938.2021.1888075,3
[5] O’Donovan R. Missing the forest for the trees: Investigating factors influencing student evaluations of teaching. Assess Eval High Educ. 2024;49(4):453-470. DOI: 10.1080/02602938.2023.2266862
[6] Rindermann H, Kohler J, Meisenberg G. Quality of Instruction Improved by Evaluation and Consultation of Instructors. Int J Acad Dev. 2007;12(2):73-85. DOI: 10.1080/13601440701604849
[7] Pawson R, Tilley N. Realistic evaluation. Thousand Oaks (CA): Sage; 1997.
[8] Georgiou H, Sharma M, Ling A. Peer review of teaching: What features matter? A case study within STEM faculties. Innov Educ Teach Int. 2018;55(2):190-200. DOI: 10.1080/14703297.2017.1342557
[9] Stalmeijer RE, McNaughton N, Van Mook WN. Using focus groups in medical education research: AMEE Guide No. 91. Med Teach. 2014;36(11):923-939. DOI: 10.3109/0142159X.2014.917165
[10] Hsih KW, Iscoe MS, Lupton JR, Mains TE, Nayar SK, Orlando MS, Parzuchowski AS, Sabbagh MF, Schulz JC, Shenderov K, Simkin DJ, Vakili S, Vick JB, Xu T, Yin O, Goldberg HR. The Student Curriculum Review Team: How we catalyze curricular changes through a student-centered approach. Med Teach. 2015;37(11):10081012. DOI: 10.3109/0142159X.2014.990877
[11] Hattie J. Which Strategies Best Enhance Teaching and Learning in Higher Education? In: Mashek DJ, Hammer EY, editors. Empirical Research in Teaching and Learning. Hoboken (NJ): John Wiley & Sons, Ltd; 2011. p.130-142. DOI: 10.1002/9781444395341.ch8
[12] Papan C, Gärtner BC, Simon A, Müller R, Fischer MR, Darici D, Becker SL, Last K, Bushuven S. Stewards for Future: Piloting a medical undergraduate elective on antimicrobial stewardship. GMS J Med Educ. 2025;42(1):Doc9. DOI: 10.3205/zma001733
[13] Ruck J, Tiedemann E, Sudmann J, Kübler A, Simmenroth A. Evaluating the longitudinal effectiveness of a 5A model-based smoking cessation counselling course for medical students in family medicine placement. GMS J Med Educ. 2025;42(1):Doc10. DOI: 10.3205/zma0011734
[14] Dreyer T, Papadopoulos S, Wiesner R, Karay Y. Classroom teaching versus online teaching in physiology practical course – does this lead to different examination results? GMS J Med Educ. 2025;42(1):Doc8. DOI: 10.3205/zma001732
[15] Junga A, Görlich D, Scherzer S, Schwarz M, Schulze H, Marschall B, Becker JC. “Hold the retractor, that's it?” – A retrospective longitudinal evaluation-study of the surgery and the elective tertial in the practical year. GMS J Med Educ. 2025;42(2):Doc3. DOI: 10.3205/zma001727
[16] Brinkmann B, Stöckigt B, Witt CM, Ortiz M, Herrmann M, Adam D, Vogelsang P. Reducing stress, strengthening resilience and self-care in medical students through Mind-Body Medicine (MBM). GMS J Med Educ. 2025;42(1):Doc7. DOI: 10.3205/zma001731
[17] Hofhansl A, Zlabinger G, Bach L, Röhrs J, Meyer AM, Rieder A, Wagner-Menghin M. Medical students’ perception of supervision in MedUniVienna’s structured internal medicine and surgery clerkship program: Subject specific differences and clerkship sequence effects. GMS J Med Educ. 2025;42(1):Doc5. DOI: 10.3205/zma001729
