Portal de Periódicos da CAPES

The Marvelous Medical Education Machine or How Medical Education Can Be Unstuck in Time

2000; Lippincott Williams & Wilkins; Volume: 75; Issue: Supplement Linguagem: Inglês

10.1097/00001888-200010001-00043

1938-808X

Charles P. Friedman,

Radiology practices and education

The jumping-off point for this paper is actually the second part of its compound title. The concept of becoming “unstuck” in time stems from the initial line of Kurt Vonnegut's popular novel Slaughterhouse Five [Vonnegut 1969]: Listen: Billy Pilgrim has come unstuck in time. In this paper I will actually argue that medical education has become “stuck,” not only in time but also in space and content. It has become stuck in time because events considered to be educational largely occur through interactions that require the learners and the faculty to be simultaneously participating in these interactions. It has become stuck in space because its mechanisms of delivery are largely bound to a specific physical location, the academic medical center with its classrooms and associated health care delivery venues. It has become stuck in content because the topics that are the focus of educational interactions are insufficiently under the control of the students, and the teachers. Increasingly, there is no reason for any of these requirements to be imposed on the educational process. Moreover, medical education remains stuck in an era when much of the rest of human enterprise is becoming unstuck, the result of a sweeping set of cultural changes made possible by information technology and primarily by the phenomenal proliferation of the global Internet [Drucker 1999]. I will further argue in this paper that medical education can gradually be “unstuck” in space, time, and content through appropriate use of emerging technology, with emphasis on simulation methods that have become widespread in the use of training pilots and professionals in other disciplines. Modern flight simulators have become so sophisticated that experienced pilots being certified to fly a new aircraft might have a load of passengers in the back the first time they actually fly the plane [Dawson and Kaufman 1998]. While there will always be a pilot experienced flying this aircraft alongside the neophyte in the cockpit, this practice clearly testifies to the educational power of simulations. Recently, the U.S. Navy adopted the inexpensive Microsoft “Flight Simulator”™ program as standard training for its new pilots, after a trainee who practiced extensively on this program recorded the best performance ever on an initial training flight [Brewin 2000]. The “marvelous medical education machine,” as the concept will be developed in this paper, is the complete simulator for medical education, analogous to the best of contemporary flight simulators. But like Vonnegut's novel, the marvelous machine is currently a work of fiction. It does not exist, although bits and pieces of it do exist, and these suggest what might be possible in the not-too-distant future. In the sections that follow, I will describe the need for the marvelous machine in greater detail, discuss what it can potentially do when built, expose the internal anatomy of the complete machine, review some of the pieces that exist now and how we might build it from here, and finally discuss some of the key educational research questions that will have to be illuminated along the way. This paper, in its entirety, will argue that building the marvelous machine should be a top priority for medical education nationally and internationally. Stuck in Space, Time, and Content To clarify what it means for medical education to be “stuck,” it may be useful to consider education as a process with events that exist in three dimensions (Figure 1). The first dimension can be thought of as physical space, the second time, and the third the biomedical topic that is under consideration. Medical education is stuck in all three dimensions, because teachers and learners have little control over these dimensions: where and when the events occur and what topics are addressed. In the basic sciences, for example, lectures and labs occur in a fixed place and at a scheduled time and on a topic that faculty believes the students need to know about—and then they are over. In the clinical sciences, patients (who remain the primary “teaching material” even though this term is seldom used anymore) appear at a fixed location and at a particular time with the problem they happen to have—and then they leave.Figure 1: Traditional medical education is “stuck” in the dimensions of space, time, and content.This way of doing educational business is so much a part of daily life in an academic medical center that most of us take it for granted; and since our students learn and graduate and become certified as practitioners, it is easy to conclude that there is nothing wrong with being “stuck.” But there are profound reasons for concern. First and foremost, education that is stuck routinely ignores much of what is known about teaching and learning in medicine. Studies of clinical reasoning accumulated over more than 20 years point to the “case specificity” of medical expertise, meaning that proficiency generalizes very weakly from disease to disease and, more generally, from one aspect of medicine to another [Elstein et al. 1978; Schmidt et al. 1990]. As such, the most effective way, and perhaps the only way, of developing proficiency over time is active practice with a wide range of cases and with as many repetitions for each subject/disease area as possible [Issenberg et al. 1999]. In educational environments that are stuck, live patients are the primary source of such practice; yet faculty and students have no control over the patients who walk into the clinic or are admitted to the hospital. Active, appropriate practice under these circumstances can be very difficult to engineer, much less guarantee. Another problem is the expectations of a coming generation of learners that has increasingly “grown up digital” [Tapscott 1998]. Our students who have experienced increasingly sophisticated video games, and who have spent hours with such excellent simulations as Sim City™ and Flight Simulator™, will recognize immediately the potential for similar experiences to enhance their training in medical domains. These learners will intuitively understand that medical education is stuck in space, time, and content. Although they may not use these exact words, they will find being stuck unacceptable. They may articulate this recognition by comparing their medical education experience with their undergraduate experience, wondering why, as the sophistication level of what they are studying is increasing, the sophistication of the technology used to support these studies is decreasing. In the short term, they may accept what they see as antediluvian educational practices, simply because these represent the only pathway to a desired profession, but over time they will demand a different kind of service, the need for which and the practicality of which they see as self-evident. If they cannot get this service from traditional educational institutions, their instincts honed by the Internet culture will lead them to seek it from other sources. Economic pressures on academic medical centers may drive change as well. The problem of providing appropriate practice for trainees exacerbates as health care economics shortens hospital stays and clinic visits, and trainees necessarily have more limited access to patients. Clinical faculty members at academic medical centers and in community settings may perceive that their productivity is judged much more by patient throughput than student learning. An educational system already limited in its ability to provide an appropriate range of “teaching material” may find itself unable to provide appropriately motivated teachers as well. If academic medical centers do not systematically recognize the opportunity afforded by information technology to “unstick” the system, others will. Hafferty has warned that, for a variety of reasons, medical education based in academic centers could lose its social mandate by not addressing in the curriculum a widely-recognized set of social needs, and thus become irrelevant to the needs of the modern world [Hafferty 1999]. Similarly, by remaining obstinately stuck in space, time, and content, academic medical centers could lose what may be called their “technical mandate” to educate because the methods being used no longer make sense to trainees and to society as a whole. Simultaneous loss of social and technical mandates will generate alternative approaches to education that could, over time, become the norm. Such alternatives are already becoming evident, for example, in the Open University's plan to offer a curriculum equivalent to the first two years of the medical curriculum in the United Kingdom [Daniel 1999], and possibly through Internet ventures such as “medschool.com”™ [Medschool.com 2000]. Established academic medical centers can choose to be leaders and active partners in these developments, or not. Some may ask to what extent the technique of standardized or simulated live patients [Ainsworth et al. 1991], which has occupied much of the attention of the medical education research community over the past two decades, offers the capabilities of the marvelous machine. It does, but as a practical matter only to a very limited extent. Standardized patients are expensive and do not offer the economies of scale that, as will be discussed, the marvelous machine so profoundly offers. The largest expense associated with use of standardized patients is the wages they must be paid, and the 20th standardized patient encountered by a student costs almost as much as the first. Standardized patients must be painstakingly trained, and there are significant costs associated with this training that are completely lost once the patient retires from active educational service. And a standardized patient can offer only limited variations on the case he/she was trained to represent. As a trainer for procedures, standardized patients must endure the mistakes of the non-expert. Invasive or risky procedures cannot ethically be performed on them at all. Although they can explain how they feel, standardized patients have no access to what is actually happening inside their bodies, and cannot explain to trainees the consequences of their actions at the organic or cellular levels. Finally, standardized patients cannot easily record what is being done to them by the trainee, so feedback to trainees cannot be related with high precision directly to their actual actions and decisions. So while standardized patients can be enormously valuable sources of practice and tools for assessment, they take medical education only part of the way to where it can and needs to go. They are, for the most part, stuck in space, time, and content. Potential of the Marvelous Machine Remember above all that the marvelous machine does not currently exist. As we consider what the future might hold if medical education begins a steady progression toward the development of the marvelous machine, it is useful to visualize an end-point of this progression. I do not envision, ever, the complete elimination of teaching around live patients in the same sense, although some might disagree, that no novice pilot is likely to receive a license without flying a real plane. Nor does this work envision that neurosurgery residents will perform their first operation solo after five years of practice only on a simulator. I do, however, envision a future where medical trainees, and practitioners for their continuing education, spend increasingly large fractions of their time working on computer-based simulators. The reasons for this are examined below. Later sections explore what must be inside such a machine in order for it to do these things. The marvelous machine is unstuck in the three-dimensional educational space described earlier (see Figure 1) because it can provide tireless practice of medical diagnosis, management, and clinical procedures. It is unstuck in the time dimension because it can be used anytime, for as long as the trainee wishes, and over and over again to provide the kind of meaningful repetition of tasks that is highly desirable. The machine is unstuck in the space dimension because the ideal, fully developed machine can “go” or be accessed anywhere. The Internet can in principle bring the capabilities of the machine to a trainee at home or on campus, anywhere in the world. This capability has enormous implications for the future of medical education as it requires us to think of the medical school not so much as a physical place but as a set of learning resources that can be delivered anywhere [Friedman 1996]. The machine is unstuck in the content dimension because it can address on demand topics and skills of faculty and/or student choice, creating appropriate variants of each case or topic to enable meaningful practice to occur. It can record every element of what happened during a student's work with a case—generating highly specific feedback to the learner on his/her performance and informing student and faculty choices about what further practice each student may need. A further key feature of the marvelous machine are the fortunate economics of its use. Once developed and programmed, there is minimal marginal cost attaching to its operation. The contrast to standardized patients, who are paid a fixed sum per hour, is particularly striking in this regard. To understand the potential of the machine from a somewhat different perspective, consider the potential of such a device to engage learners in “what if” games, which are enormously educational. Students can ask, and get answers from the machine, to the following classes of “what if” questions: What if I did the procedure again, just a little bit differently? The marvelous machine allows students to tinker in a way that enables them to hone their skills and judgment. A student can, for example, explore the consequences of perhaps giving a slightly stronger dose of a drug to a “patient” whose disease is being simulated by the machine. What if I did this in a way I know is wrong? Without the machine, it is difficult to experience the consequences of mistakes as a way to learn to manage them. In the real clinical world, mistakes certainly cannot be purposely made, and when they occur occasionally by accident or oversight they are not often not recognized as such until long after their occurrence. With the machine, students can make mistakes on purpose, knowing that they are mistakes, so they can practice managing the consequences, or just to see what happens. What if I did this 100 times in each of two different ways? One of the most educationally creative ways of using the marvelous machine may be to conduct an “instant clinical trial” by instructing the machine to treat 100 instances of the “patient” one way and another 100 instances a different way. The models built into the mature machine, as will be discussed below, are necessarily and realistically probabilistic and the machine will therefore reflect naturally occurring variability in the way organisms respond to drugs and other external stimuli. What if biology worked just a bit differently? Used in this way, the machine can connect the basic and clinical sciences. In a fully mature version of the machine, students can be given the capability of changing the parameters of the biological models that drive the simulator. The potential of enhancing their understanding of basic biology is significantly enhanced through the ability to see how organisms would act or react if the basic laws of biology were constructed just a bit differently from the way we believe they are. Based on this, for now, somewhat abstract conceptualization of the marvelous machine and using our imaginations to conceive what someday the machine will be able to do, consider how medical education must then be undertaken. Medical education would not look and feel at all as it does now. The rationale for “lockstep” learning wherein students proceed in unison through a relatively rigid curriculum would disappear almost completely, and likely with it would disappear the notion of a four-year curriculum. Indeed, lockstep learning can be seen as an administrative artifact of the lack of a mature marvelous machine. Students could, in principle, begin their predoctoral education whenever they were ready, and authorized, to do so. They would end it when they had proved they had mastered the stated objectives of the curriculum. There might be no need to have students physically on the central campus most, or even some, of the time. Lectures certainly have their place as an educational medium, but the current reliance on lectures as a primary mechanism for conveying information would no longer make sense. Perhaps, with the marvelous machine, we could return to the pre-Flexnerian concept of the part-time student without inheriting the educational inadequacies of the pre-Flexner era. While this paper does not focus on continuing education of physicians and other health professionals, the needs in continuing education are such that the potential effects of the machine on this level of the educational continuum are similarly revolutionary [Barnes 1998]. The Anatomy of the Marvelous Machine Now to more technical specifics. How are we going to build the machine? What is its anatomy [van Meurs et al. 1997], its necessary component parts? As illustrated in Figure 2, the marvelous machine can be seen as having five major components, not counting the “learner” without whom the machine would have no purpose. The specific techniques for developing each of these components are beyond the scope of this paper, but a later section discusses the academic disciplines that contribute to each one.Figure 2: The anatomy of the marvelous machine. First and foremost, the machine has a domain model, which is a mathematical description of the biological phenomena governing the disease or body sub-system of interest. The domain model computes the state of the patient and the effects of the learner's actions on the state of the patient. The mathematical domain model is what makes it possible for the marvelous machine to generate an endless supply of novel cases and other practice opportunities, and it is what largely sets the marvelous machine apart from traditional simulation environments that use “scripted” cases. Typically, these domain models have explicit probabilistic features that reflect the natural variability in disease development and response to clinical interventions. Next is the clinical representation engine. This component is necessary because the output of the domain model is typically a set of numbers that must be translated into clinical observables: statements the patient would make about his/her disease (“I feel tired all the time…”), findings that could be appreciated on physical examination (“The patient is cyanotic…”), and test results (“Biopsy reveals a tumor…). The sensory pathways component takes the findings and creates portrayals of them that are actually seen, heard, or touched by the learner. This component can be seen as the virtual reality aspect of the machine [Hoffman and Vu 1997; Satava and Jones 1998]. In a mature version of the machine, the learner will see the patient and hear his/her statements; and experience his/her physical condition through sight, touch, and hearing. All of these presentations would change as the patient's condition changed, as directed by the domain model in response to actions taken by the learner and/or a natural evolution of the patient condition. The changes might occur in real time, as would be the case if the learner was using the machine to practice a procedure, or in compressed (simulated) time if the learner was using the machine to practice longitudinal management of a chronic disease. The scoring model is the basis of providing performance feedback to the learner. Although there are other ways of approaching this problem, the scoring model typically would compute what is the ideal action for the learner to take at any point in the simulation and compute an instantaneous “score” for learner through a metric that compares the ideal performance with what the learner actually did. In some versions of the machine, the knowledge encoded in the domain model can also be harnessed to power the scoring model. Lastly, a complete educational application using the machine must have a curriculum model. Since the machine's domain model can support learners' practice by constructing cases with specific problems and other characteristics, the curriculum model would represent the set of problems and characteristics on which all learners must have practice, and in which order. For each learner, the curriculum model would maintain records of which aspects of practice had actually occurred. To illustrate how these components of the machine would interact to generate a comprehensive practice experience, we could follow the machine through one conceptual cycle of operation. This example is a bit simplistic, but illustrative of the concepts. Ms. Smith, a medical student, is taking a rotation in clinical oncology and indicates to the machine that she wants to practice on a simulated case. The process begins with the curriculum model determining that she has not completed her minimum quota of practice on managing metastatic breast cancer. The machine may ask Ms. Smith at that point if she would like some further practice in breast cancer management. After an affirmative response, the domain model then generates a case of metastatic breast cancer, represented mathematically, subject to the constraints passed to it by the curriculum model. Because the domain model is inherently probabilistic, many features of the case presented to Ms. Smith are determined by chance and no two cases would be exactly the same. The clinical representation engine then converts the initial state of the patient to a set of clinical findings that can be made known to Ms. Smith, should she request them as part of her initial workup of the patient. Ms. Smith's work then begins. She is told that the patient is in her “clinic” and takes a history, performs an exam and runs tests on the patient. Only those patient findings actually requested by Ms. Smith would be revealed to her. This is mediated through the sensory pathways component of the machine. Ms. Smith would hear the patient's voice responding to questions, see (and, depending on the maturity of the virtual reality component of the machine, perhaps feel) the areas affected by the patient's previous surgery, and see the results of lab tests and imaging studies indicating metastatic disease. Based on Ms. Smith's initial work-up, she then puts the patient on a regimen of chemotherapy. The domain model then computes the effects of the chemotherapy on the course of the patient's disease, mathematically modeling the growth of tumor cells, the reactions of these to the therapy, and any toxicity that may result from the therapy. The scoring model, in the meantime, has assigned and recorded a score (or scores) to the actions Ms. Smith has taken. Assuming that the domain model determines that Ms. Smith's therapeutic regimen would cause toxicity, Ms. Smith would encounter the patient again when that toxicity had developed to the point that the patient would be symptomatic and would return to the clinic. At this later point in simulated time, the domain model will have generated a new set of mathematical parameters describing the patient's updated condition, and will have passed them to the clinical representation engine. The cycle of the machine's operation continues with Ms. Smith having the opportunity to examine the patient again, run more tests, and make decisions to manage the toxicity. Those decisions would be assigned a score, and the simulation would continue until the exercise was completed. Ms. Smith might indicate to the machine that she was finished, or the patient might die or become disease free after a sufficient period for the domain model to conclude a probable cure. It would then be possible for Ms. Smith to initiate a dialog with the scoring model, which would present her score and critique her performance. If Ms. Smith wished, she could run the simulation clock back to a point where her performance was sub-optimal, and play a “what if” game by trying something different and experiencing the consequences of her revised actions. How the Machine will be Built Is the example above science fiction? Partially, but on balance, not. Indeed, a primative version of the simulator described above (see Figure 3) has been developed through the OncoTCap project at the University of Pittsburgh [Day et al. 1998]. A key innovative element of OncoTCap is the development for many specific areas of oncology of a domain model that is powerful enough to drive a simulation of the type described. Indeed, OncoTCap's domain model allows students to play all the “what if” games described earlier. They can try again, just to see if they can do better; they can do something wrong on purpose, just to see what happens or to practice managing the consequences, they can instruct the domain model to run two types of treatment, each with 100 simulated patients, to run a “clinical trial” to see which method is superior; and they can even change the parameters of the domain model to see what the world would be like if biology worked a bit differently than science currently thinks it does.Figure 3: Screen shot of the OncoTCap simulator which has been linked to a clinical presentation engine.Other notable efforts to build elements of the marvelous machine are described below. Still, it is safe to say that building the marvelous medical education machine is rocket science. It is much harder than building a flight simulator, in part because of a major difference between aviation and medicine. As Dawson and Kaufman (1998) have observed, in medicine one must manipulate the environment whereas in aviation the goal is to avoid it. The problems of realistically representing clinical findings, and their evolution over time in the same patient, are enormous. When one loads, on top of that, the virtual reality aspects of creating the sensation of actually interacting with the patient through all senses, the full magnitude of the challenges that lie ahead begins to come clear. So how will the machine be built? First of all, it will be built incrementally. Pieces of it already exist; other pieces will arrive in the near future; and in some sense it will never be complete. It will just get better and better over time. Second, it will be built domain-by-domain. The comprehensive unified mathematical model of human biology, the “Maxwell's Equations of biology,” probably do not exist and, if they do, they are not likely to be discovered anytime soon. What we are therefore likely to see in near future are cancer simulators, anesthesiology simulators, diabetes simulators, surgical simulators, etc. These domain-specific simulations will become increasingly sophisticated, and then at some point in the future, the models will become sufficiently powerful that simulations from different domains will begin to merge. Finally, the marvelous machine will be developed through collaborations among clinical domain experts and scientists in various disciplines. The clinical representation and sensory pathway components are problems that fall to computer scientists and engineers; domain modeling is work for computational biologists and researchers in artificial intelligence; the scoring and curriculum models are the purview of psychometricians and decision analysts. Collaborative efforts to develop components of the marvelous machine abound. Many collaborations, some of which have created mature products, have been ongoing for many years. To cite just a few examples, two models of simulators for anesthesiology have reached a high level of development [Norman and Wilkins 1996]. A group in London has developed a prototype marvelous machine for diabetes [Lehmann 1998]. Groups at Stanford and UC-San Diego have taken important strides in developing anatomical simulations that are the basis for building practice on clinical procedures into the marvelous machine [Hoffman and Vu 1997; Dev et al. 1998], as have groups at Mayo and Walter Reed Hospital in the specific area of GI procedures and endoscopy [Robb 1997]. It is important to acknowledge the CBX (Computer Based Exam) project of the National Board of Medical Examiners, which has created a comprehensive simulation environment of the U.S. medical certification process [Clauser et al. 1998] and an effort underway at the American Board of Family Practice to develop simulations driven by mathematical models [Sumner et al. 1998]. Algorithms that would drive a scoring module of the type described above have also been developed [Downs et al. 1997]. So while the marvelous machine as a whole does not exist, it is very safe to say that significant bit and pieces of it do exist and there are substantial reasons to believe that it can and will be built over time. Conclusion: The Educational Research Challenges A final piece of the challenge of the marvelous machine is the set of educational research questions that must be addressed if the machine is going to be built properly and its value and place in medical education thoroughly understood. By this I do not mean the myriad of technical research challenges that will have to be overcome to build the domain models, scoring models, clinical representation engines, and sensory pathways to the learner. As discussed earlier, these fall properly into the research areas of computer science and engineering, computational biology, and other fields. From an educational research perspective, the key questions map out uncharted territory because of the novelty of what the machine can do. To the extent that the machine represents new technology with the potential to be of benefit, this does not mean that the machine will be of benefit. As with any technology, there is potential for it to leave us less well off than we were before. In the end, no matter how well the technology itself functions, the success of the machine will depend on how the machine is used, the educational engineering of the machine into a comprehensive learning environment in which the machine is but one element. As noted earlier, teaching around live patients is not going to go away, no matter how sophisticated the machine becomes over time. Even though the live lecture as an educational medium is completely stuck in space and time and content, the live lecture will likely prove more durable than its most strident critics would have us believe. The proper use and integration of the machine into medical education can be directed profoundly by research that addresses questions such as: Relative to the domain model, how “good” do these models have to be in order for them to be ready for use in education. For educational purposes it is perhaps sufficient for the domain model to create and evolve cases that are plausible, but not absolutely correct [Friedman 1995]. But how plausible is plausible enough? Relative to the curriculum model, how should an unstuck curriculum be structured? With freedom to learn anywhere, anytime, and on topics of student and/or faculty choice, how much freedom is the right amount of freedom? What should be constrained? To what extent should the domain learning model be one of discovery-oriented? How should more, or perhaps less, freedom be granted to learners as their experience and expertise accumulate over time? Relative to the scoring model, all of the reproducibility and validity issues that arise with any new assessment technique arise with the marvelous machine as well. The score a student receives for working one simulated case—or a battery of cases comprising a certification examination—has to be meaningful. Other questions relating to the scoring model relate to the structure of feedback. What models for presenting feedback to learners, during and after case, are most facilitative of learning? This surface glance at the important educational research questions that attach to the marvelous machine brings this paper to its closing plea. Perhaps this is a plea that is totally unnecessary, but the potential role of the marvelous machine in medical education seems so important that the research community should address itself to it sooner rather than later. Much of the needed educational research can be applied formatively to guide ongoing developmental efforts, and it is not too early to get started. The biggest mistake at this point would be to view the machine parochially as a technical undertaking, leaving its development solely to the “techies” until some point in the future, by which time many key opportunities may be lost. So in some sense, the educational research community faces, on a smaller scale, the same challenge posed by the marvelous machine to the medical education community as a whole. The machine is coming; it is inevitable. It will gradually and by dint of great creative effort unstick medical education in space, time, and content. Those who ignore it run the risk of becoming irrelevant; those who embrace it can do enormous good for the profession and, ultimately, for the health of the public we all serve.