Portal de Periódicos da CAPES

Audiovisual calibration tools for examiners: the use of a “standardised student” to train undergraduate paediatric examiners

2007; BMJ; Volume: 92; Linguagem: Inglês

1468-2044

S. Wilson, Gareth Tudor‐Williams, H. Davies, Michael D. Blair,

Medical Education and Admissions

Background: Our university recently changed to a new form of formative, in-course assessment for medical students in paediatrics, the mini-clinical evaluation exercise (mini-CEX). All examiners differ in their stringency (the hawk/dove phenomenon), and we saw this as an ideal opportunity to address interrater differences as well as train examiners in mini-CEX. Aims: To evaluate the usefulness of a training DVD at mini-CEX examiner training workshops, as a pilot for assessing the effect of feedback on the scoring tendencies of undergraduate paediatric examiners. Methods: Subjects were an opportunity sample of 27 paediatricians, paediatric registrars and staff grade paediatricians participating in 2 mini-CEX training workshops. They scored a DVD depicting a standardised taking a paediatric history or examining a child at 3 levels of competency. Scores were charted to create a frequency histogram and the results fed back to examiners. The critique of the student was discussed and scores at the extremes explored with examiners. Results of the opportunity sample were compared to a gold standard score derived from the assessment of the same DVD clip by 5 experienced undergraduate paediatric examiners. Results: Interrater reliability was high for the below expectations but low for the meets expectations/borderline (MEBS), indicating wide variation in the expectations of paediatric clinicians examining at undergraduate level. Of a possible 24 marks (minimum 6), the median mark for the MEBS was 16 (range 10-20). Examiners were often unaware whether they were hawks or doves until scoring tendencies were explored in the training session. All workshop participants felt the training DVD helped them to better understand the mini-CEX process. The median of the experienced examiners was also 16, with a much narrower range (14-17) which we have previously reported. Conclusions: Examiners found the DVD scoring session helpful to familiarise themselves with mini-CEX and identify their rating style. Further work is in progress to objectively evaluate whether feedback about scoring tendency and comparison to a score agreed by experienced examiners will modify extreme scoring behaviour.