Portal de Periódicos da CAPES

Fool’s gold and chasing unicorns: USMLE Step 1 has no clothes!

2017; American Physical Society; Volume: 41; Issue: 2 Linguagem: Inglês

10.1152/advan.00016.2017

1522-1229

Heidi L. Lujan, Stephen E. DiCarlo,

Diversity and Career in Medicine

EditorialFool's gold and chasing unicorns: USMLE Step 1 has no clothes!Heidi L. Lujan and Stephen E. DiCarloHeidi L. LujanDepartment of Physiology, Wayne State University School of Medicine, Detroit, Michigan and Stephen E. DiCarloDepartment of Physiology, Wayne State University School of Medicine, Detroit, MichiganPublished Online:25 Apr 2017https://doi.org/10.1152/advan.00016.2017MoreSectionsPDF (40 KB)Download PDF ToolsExport citationAdd to favoritesGet permissionsTrack citations ShareShare onFacebookTwitterLinkedInWeChat Everyone is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid.—Albert Einsteinhans christian andersen tells the tale of an emperor who loved wearing fine garments and consumed all of his peoples' hard-earned money on them. He had a different ensemble for each hour of every day and was, by all accounts, the finest dressed man in the land.One day, two crooks claiming to be weavers convinced the emperor that they were capable of making the finest, lightest, most magnificent fashions the world has ever seen using cloth that was invisible to anyone who was inept or unwise. Impressed, the emperor foolishly paid the swindlers to weave an outfit using their extraordinary cloth.When the garment was finished, a procession was arranged to show off the emperor's new apparel, and the entire city gathered. Dressed in his new outfit and believing he looked wonderful, he paraded before his people.His people believing that the new attire was invisible to anyone who was inept or unwise, offered thunderous applause. None, except a child, was willing to admit that, "The Emperor has no clothes!" Similarly, the United States Medical Licensing Examination (USMLE) Step 1 has "no clothes," since many people believe that its validity for postgraduate medical residency selection decisions is based on empirical data; however, this belief, like the Emperor's new clothes, is unwise and unseen.The USMLE score is one of many factors considered by residency programs in selecting applicants. As such, it is a profoundly "high stakes" exam. High-stakes exams are often the only metric that counts in medical school, and many educators are mainly concerned with test outcomes. The higher the stakes, the more students and professors focus on teaching and learning for the exam. As a result, what is not tested on Step 1 or topics and skills that cannot be tested with a multiple-choice format are not taught, and instruction starts to look like the exam. Furthermore, understandably, students only want to prepare for material that will be tested on Step 1 because of its importance for residency decisions. Of course, this neglects many life skills, including critical thinking, problem solving, communication, interpersonal skills, and compassion. It also creates a competitive environment, a negative attitude toward teaching and learning, and disconnects us from each other and the world around us. This in unfortunate, because it is well documented that patients with physicians who communicate well are more adherent to therapies, more satisfied with care, and less likely to file malpractice suits (4, 7, 15–17, 19). Patients mainly want a doctor who will communicate with them (9). Fortunately, communication skills and other life skills can be taught and evaluated (8).Furthermore, the scores on USMLE Step 1 are not correlated with reliable measures of medical students', residents', or fellows' clinical skill acquisition (11). In fact, correlations of USMLE Step 1 scores and reliable measures of clinical skills acquisition obtained from nine studies involving 393 students from 2005 to 2010 document that using USMLE Step 1 scores for postgraduate residency selection is neither structured, coherent, nor evidence based (11). This also creates a profound contradiction in that we teach students to be critical thinkers, to use evidence-based medicine and diagnostic tests to guide their decision-making processes; while at the same time relying on a test, to make career changing decisions, that is not evidence based and has little validity for residency selection decisions (12).Moreover, many students believe that the test defines them such that their intelligence is determined by the grade on a multiple-choice test. However, learning is a continuous process of integrating what is known with new information, and students must understand that knowledge is contextual. That is, information must be tailored and applied to different situations (2). The multiple-choice format of one question, four answers, and only one correct answer creates the illusion of right and wrong, a binary condition that ignores the infinitely fluid nature of information and that knowledge is complex, tentative, evolving, and something that can be improved upon over time. This is disheartening, as it erodes motivation and discourages life-long learning.The time, stress, and expense associated with Step 1 cannot be overstated, and the social stigma of low scores and possible financial hardships can be overwhelming. As examples, medical students pay approximately $136 million dollars each year in registration fees for the USMLE exams (http://www.usmle.org/performance-data/default.aspx). Tens of millions of dollars are also spent on travel and test preparation expenses. This seems like an unacceptable burden for an exam the holds little validity. Accordingly, it is time to scrutinize our assumptions regarding Step 1 and accept that the value of scores as an objective evaluation of student performance or reliable indicator of student learning is under deep scrutiny and challenge (14).However, adding a collaborative group testing component to the traditional exam format during years 1 and 2 of the medical school experience may reduce the Step 1 insanity and provide an opportunity for life skills to be taught, practiced, and evaluated (1, 5, 6, 10, 13). Student assessment during the collaborative group testing component of the traditional exams during years 1 and 2 can be determined by behavioral, noncognitive life skills that are based on the observation of the student and the student's work during the group component. Specifically, communication, collaboration, and problem-solving skills, decision making, and initiative, critical integration of knowledge, professional attitudes, and ethics can all be practiced by the student and evaluated by the faculty during the collaborative group testing component of the traditional exam.The final grade on the exam can be based on both the performance on the traditional component of the exam and on the behavioral parameters during the group component (10). This approach prevents the entire grade being based on work that rewards only correct answers. This method also rewards students for participation and effort as well as engaging with the material. Accordingly, with the collaborative group testing component, the process of gaining knowledge is as important as the answer. The focus is on social and communication skills and critical thinking and the exchange of ideas (10).Some students and educators may be concerned that collaborative group testing may bias against students who do not have good communication or social skills. However, everyone has the ability to sharpen their communication and social skills through study and practice. The point is that improving our students' life skills will result in patients who are more adherent to therapies, more satisfied with care, and less likely to file malpractice suits (4, 7, 15–17, 19). Furthermore, it should be noted that there are efforts to improve critical thinking and problem-solving skills on the Step 1 exam by developing and reformatting exam questions; however, our thesis was not a balanced discussion because a preacher does not begin a sermon on the evils of alcohol by admitting the comforting effect of a beer after a hard day at the laboratory (3, 18). The defense of Step 1 will be left to its many able advocates.Adding a collaborative group testing component to the traditional exam format during years 1 and 2 of the medical school experience will not completely change the Step 1 insanity. However, reducing competition and enhancing collaboration and communication with a collaborative group component to the traditional exam is an important step to beginning this process.DISCLOSURESNo conflicts of interest, financial or otherwise, are declared by the author(s).AUTHOR CONTRIBUTIONSH.L.L. and S.E.D. conceived and designed research; H.L.L. and S.E.D. drafted manuscript; H.L.L. and S.E.D. edited and revised manuscript; H.L.L. and S.E.D. approved final version of manuscript.REFERENCES1. Cortright RN, Collins HL, Rodenbaugh DW, DiCarlo SE. Student retention of course content is improved by collaborative-group testing. Adv Physiol Educ 27: 102–108, 2003. doi:10.1152/advan.00041.2002. Link | ISI | Google Scholar2. Cortright RN, Lujan HL, Cox JH, Cortright MA, Langworthy BM, Petta LM, Tanner CJ, DiCarlo SE. Intellectual development is positively related to intrinsic motivation and course grades for female but not male students. Adv Physiol Educ 39: 181–186, 2015. doi:10.1152/advan.00117.2014. Link | ISI | Google Scholar3. DiCarlo SE. Teacher quality matters!! Physiologist 53: 89, 92–89, 94, 2010. PubMed | Google Scholar4. Franks P, Jerant AF, Fiscella K, Shields CG, Tancredi DJ, Epstein RM. Studying physician effects on patient outcomes: physician interactional style and performance on quality of care indicators. Soc Sci Med 62: 422–432, 2006. doi:10.1016/j.socscimed.2005.05.027. Crossref | PubMed | ISI | Google Scholar5. Giuliodori MJ, Lujan HL, DiCarlo SE. Collaborative group testing benefits high- and low-performing students. Adv Physiol Educ 32: 274–278, 2008. doi:10.1152/advan.00101.2007. Link | ISI | Google Scholar6. Giuliodori MJ, Lujan HL, DiCarlo SE. Student interaction characteristics during collaborative group testing. Adv Physiol Educ 33: 24–29, 2009. doi:10.1152/advan.90161.2008. Link | ISI | Google Scholar7. Hall JA, Roter DL, Milburn MA, Daltroy LH. Patients' health as a predictor of physician and patient behavior in medical visits. A synthesis of four studies. Med Care 34: 1205–1218, 1996. doi:10.1097/00005650-199612000-00006. Crossref | PubMed | ISI | Google Scholar8. Hausberg MC, Hergert A, Kröger C, Bullinger M, Rose M, Andreas S. Enhancing medical students' communication skills: development and evaluation of an undergraduate training program. BMC Med Educ 12: 16, 2012. doi:10.1186/1472-6920-12-16. Crossref | PubMed | ISI | Google Scholar9. Hurwitz B, Vass A. What's a good doctor, and how can you make one? BMJ 325: 667–668, 2002. doi:10.1136/bmj.325.7366.667. Crossref | PubMed | Google Scholar10. Lujan HL, DiCarlo SE. The flipped exam: creating an environment in which students discover for themselves the concepts and principles we want them to learn. Adv Physiol Educ 38: 339–342, 2014. doi:10.1152/advan.00081.2014. Link | ISI | Google Scholar11. McGaghie WC, Cohen ER, Wayne DB. Are United States Medical Licensing Exam Step 1 and 2 scores valid measures for postgraduate medical residency selection decisions? Acad Med 86: 48–52, 2011. doi:10.1097/ACM.0b013e3181ffacdb. Crossref | PubMed | ISI | Google Scholar12. Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of United States Medical Licensing Examination Step 1 scores in residency selection. Acad Med 91: 12–15, 2016. doi:10.1097/ACM.0000000000000855. Crossref | PubMed | ISI | Google Scholar13. Rao SP, Collins HL, DiCarlo SE. Collaborative testing enhances student learning. Adv Physiol Educ 26: 37–41, 2002. doi:10.1152/advan.00032.2001. Link | ISI | Google Scholar14. Schinske J, Tanner K. Teaching more by grading less (or differently). CBE Life Sci Educ 13: 159–166, 2014. doi:10.1187/cbe.CBE-14-03-0054. Crossref | PubMed | ISI | Google Scholar15. Stewart M, Brown JB, Boon H, Galajda J, Meredith L, Sangster M. Evidence on patient-doctor communication. Cancer Prev Control 3: 25–30, 1999. Crossref | PubMed | Google Scholar16. Tulsky JA, Arnold RM, Alexander SC, Olsen MK, Jeffreys AS, Rodriguez KL, Skinner CS, Farrell D, Abernethy AP, Pollak KI. Enhancing communication between oncologists and patients with a computer-based training program: a randomized trial. Ann Intern Med 155: 593–601, 2011. doi:10.7326/0003-4819-155-9-201111010-00007. Crossref | PubMed | ISI | Google Scholar17. Walsh JM, McPhee SJ. A systems model of clinical preventive care: an analysis of factors influencing patient and physician. Health Educ Q 19: 157–175, 1992. doi:10.1177/109019819201900202. Crossref | PubMed | Google Scholar18. West KM. The case against teaching. J Med Educ 41: 766–771, 1966. PubMed | Google Scholar19. Williams S, Weinman J, Dale J. Doctor-patient communication and patient satisfaction: a review. Fam Pract 15: 480–492, 1998. doi:10.1093/fampra/15.5.480. Crossref | PubMed | ISI | Google ScholarAUTHOR NOTESAddress for reprint requests and other correspondence: S. E. DiCarlo, Wayne State University School of Medicine, 540 E. Canfield Ave., Detroit, MI 48201 (e-mail: [email protected]wayne.edu). Download PDF Back to Top Next FiguresReferencesRelatedInformation Related ArticlesIn the valley of the blind, the USMLE is king 25 Apr 2017Advances in Physiology EducationCited ByRelationship between classroom attendance and examination performance in a second-year medical pathophysiology classChristine A. Kauffman, Megan Derazin, Abdo Asmar, and Jonathan D. Kibble25 September 2018 | Advances in Physiology Education, Vol. 42, No. 4 More from this issue > Volume 41Issue 2June 2017Pages 244-245 Copyright & PermissionsCopyright © 2017 the American Physiological Societyhttps://doi.org/10.1152/advan.00016.2017PubMed28442475History Received 30 January 2017 Accepted 27 February 2017 Published online 25 April 2017 Published in print 1 June 2017 Metrics