Portal de Periódicos da CAPES

THE CLASSIC: Virtual Reality Surgical Simulator

2006; Lippincott Williams & Wilkins; Volume: 442; Linguagem: Inglês

10.1097/01.blo.0000194681.94882.9a

1528-1132

Richard M. Satava,

Augmented Reality Applications

Richard Satava is an acknowledged expert in the development of leading-edge technology in surgery such as virtual reality surgical simulators, surgery in the space environment, three-dimensional (3-D) imaging, and tele- presence surgery. His education included undergraduate work at Johns Hopkins Medical School at Hahnemann University in Philadelphia, a residency in surgery at the Mayo Clinic, and a fellowship in surgical research also at the Mayo Clinic. He currently is a professor of surgery at the University of Washington and program manager of Advanced Biomedical Technology at the Defense Advanced Research Projects Agency. He also is a special assistant in Advanced Surgical Technologies at the United States Army Medical Research and Materiel Command at Fort Dietrich, Maryland. In the past, he was a professor of surgery at Yale University and at Walter Reed Army Medical Center. His numerous appointments include the White House Office of Science and Technology Committee on Health, Food, and Safety; the Emerging Technologies Committee; and the Informatics Committee of the American College of Surgeons. He has served as president of the Society of Laparoscopic Surgeons and the Society of American Gastrointestinal Endoscopic Surgeons. The paper published here describes Dr. Satava's early experience with virtual reality simulation and his suggestions for the future of this technology. Henry H. Sherk, MD Whenever something is too dangerous, expensive, or distant in time, place, or imagination to physically experience, there have been attempts to simulate the experience. Television and movies convey real or imagined experiences, but they are passive, not interactive. Video games are interactive, but not realistic. The new technology of virtual reality brings a measure of the conveyed experience with interactivity. Virtual reality is a fully three-dimensional computer-generated “world” in which a person can move about and interact as if he actually were in this imaginary place. This is accomplished by totally immersing the person's senses in the “world” using a head-mounted display (HMD) and a DataGlove. The HMD is a helmet fitted with paired wide-angle television screens that sit in front of the eyes and stereophonic speakers placed over the ears so a person wearing the helmet can only see and hear what is generated by the computer. The Data-Glove is an input device (a glove that works like a joystick) which appears as a hand in the “world.” One can “move” through the world by pointing in a direction for travel or pick up and manipulate objects by making a grasping motion with the glove. A world can be anything from a kitchen to an automobile to an abdomen-anything which can be drawn can be experienced three dimensionally. Simulation has been a mainstay for aviators for the past 40 years, ever since the first Link flight simulator. Over the decades, the simulation has become so realistic that pilots now spend their initial hours of “flying” in the simulator, making perfect takeoffs and landings before they ever set foot in an airplane. The benefit and savings in time, cost, equipment, and safety are enormous. The newest generation of flight simulators have incorporated the techniques and concepts of virtual reality to add an even greater level of realism.Fig: No Caption Available.There have been attempts to bring simulation into the medical educational process. Many have included computerized interactive question-and-answer teaching tools (including multimedia); video-game-like manipulations of images have also been used. None of these efforts have achieved any level of realism, particularly for surgical application. There is one exception-the endoscopic simulation of Noar,2 in which the simulator is a modified videoendoscope. Since it is the same instrument which is used to perform an actual procedure, the resulting simulation is amazingly convincing. Technology is now becoming available for a surgical simulator that can provide the necessary interactive realism to serve as an adequate simulator to teach surgical technique: virtual reality. There are two generic requirements for any surgical simulator; it must have accurate detail and must be highly interactive. Specifically, the image must be anatomically precise and the organs must have natural properties such as the ability to change shape with pressure and to behave appropriately in gravity. All the body parts that are represented must be able to be manipulated by grasping, clamping, or cutting, and they must be able to bleed or leak fluids. An anatomically correct image can either be rendered in high-resolution 3-D graphics (cartoon-quality graphics) or digitally scanned in, as from a CT or MRI scan (a video- quality image). Interactivity for a surgeon means multi- sensory input-the ability to grasp and move objects or organs; having instruments that feel and respond with dexterity; and receiving the natural input of pressure, force feedback, and tactile sensation. Stereophonic sound has proven to be a powerful contributor to the feeling of realism. Using human interface technology, the man-machine interface must be constructed in such a way that the images, sensory input, and interactivity combine to convince the surgeon that the illusion in front of him is real; shortfalls from any of these requirements detract from the total believability of the simulation. All simulators are driven by the capabilities of the hardware and software. Powerful as computers are today, they are woefully inadequate for lifelike total surgical simulation. There are five areas that must be addressed to provide a realistic simulation: (1) Fidelity-the image must have enough high resolution to appear real. (2) Object properties-the organs must deform when grasped and must fall with gravity. (3) Interactivity-the surgeon's hand and surgical instruments must interact realistically with the organs. (4) Sensory input-force feedback, tactility, and pressure must be felt by the surgeon. (5) Reactivity-the organs must have appropriate reactions to manipulation or cutting, such as bleeding or leaking fluids. Initial research over the past 5-10 years in video technology, graphics, computer-aided design (CAD) and virtual reality has given us an insight into some of the requirements for a virtual-reality surgical simulator. If graphic images are used, it is estimated a rate of 500,000 polygons/s would be required for a realistic reconstruction of the abdomen; current high-level computer-graphics workstations generate 60,000-100,000 polygons/s. Using CT or MRI scan images would require substantially more computer power. The algorithms for object deformation and gravity are available and continue to evolve. For motion, at least 30 frames/s are necessary to eliminate flicker and response delays; this level of interactivity is available on standard VR systems. The computational power for sensory input and for object reactivity has not been determined. As each of these components is added to make the simulation more realistic there is additional tremendous computer power required; at this time solutions are achieved by trade-off. For example, if the image is very high quality, the motion is too slow or there is no interaction; if the motion in the 3-D world is realistic, there are only cartoon-level graphics. Even research computers with massive parallel processors, like the Connection Machine and Pixel Planes are orders of magnitudes away from the required computing power. The video-display devices are improving; standard television-style video monitors using a cathode-ray tubes (CRT) can generate 1,000 lines for superb resolution, but a head-mounted display (HMD) using a liquid crystal display (LCD) is less clear. Alternate display technologies such as high-definition television (HDTV) and holography are still in the prototype stages and have not been explored for a virtual-reality environment. Finally, the human interface with a simulator must be as “natural” as possible; sitting in front of a 2-D video screen with a keyboard would detract from even the most realistic images in a simulation. MATERIALS AND METHODS This is the first step, a most primitive VR surgical simulator. Using off-the-shelf, state-of-the art VR hardware (Silicon Graphics work-station) and software (Paracomp Swivel 3-D) from VPL Research, a surgical “world” of the abdomen has been created of a torso which contains the stomach, pancreas, liver, biliary tree, gallbladder, and colon as well as a few surgical instruments (scalpel, clamps). The images of organs and instruments are created using 3-D computer-aided design (CAD), resulting in anatomically accurate, cartoon-level (3,000-7,000 polygons) graphics. The display technology is a commercial Eye- phone high-resolution HMD (using LCD displays with only 200 lines of resolution) and interaction is with the Data-Glove, but there is no sensory input of force feedback or tactile sensation. The visual display rate and interaction with the “world” is at nearly 30 frames/s, depending upon the complexity of the graphics involved. RESULTS The virtual abdomen which has been created is realistic because of the anatomic and technical accuracy; however, the computer-generated graphics detract from believability because of the cartoon level of the graphic rendering. When wearing the HMD, some of the clarity is lost because of the limited resolution of the LCD display; a new- generation color high-resolution LCD which is in prototype will alleviate this problem. The interactivity is excellent; instruments can be grasped, opened, and closed, and organs can be clamped. Pulling upon or moving the organs from place to place is not believable, since they do not bend or change shape. The object deformation algorithms which give elasticity and dynamic realism to the organs are available but have not been utilized because they take so much computational power. As the computers become more powerful, this component will be added. Although blood vessels have been included, cutting them does not produce bleeding; it will be much later into development before this aspect of simulation can be incorporated. The organs can be visualized from both inside (like an endoscopic view) and outside (like an open surgical procedure). There are two main applications: anatomy lesson and surgical procedure simulation. Although we are pressing the limits of the computer capabilities, both applications are realtively simplistic; it is more like the Walt Disney movie Alice in Wonderland. An extremely interesting application for anatomy is that the abdomen can be explored by “flying” around the organs (a bird's-eye view from the diaphragm) or behind them (watching the bile duct pass through the pancreas under the duodenum) or even inside them. (A trip starting from inside the esophagus, down the stomach, through the duodenum, up the bile duct, and out the gall bladder is a popular route.) While this may seem like trivial play, these extraordinary perspectives impart a deeper understanding and appreciation of the interrelationship of anatomical structure which cannot be achieved by any other means, including cadaveric dissection. The simulation of a surgical procedure is primitive. The organs are obviously not real, there is no deformation or reaction to grasping, there is no sensory input of pressure or texture, and the instruments are crude and respond sluggishly. Because of the construction of the world, the organs must be moved around like static blocks, when they are grasped they break off at the given points where they were drawn. And of course there is no bleeding, bile leakage, or organ peristalisis. Yet the basic framework for a totally interactive simulator is in place, with the action initiated and performed by the surgeon using motions that are actually used during surgery. While the limitation of the system is that it is more like a video game because of the cartoon graphics, low-resolution HMD, and non-natural interface, as programming progresses and computational power increases the simulation will achieve greater realism. DISCUSSION Virtual reality provides the first opportunity to combine 3-D visual imagery with interactivity at a level that would permit realistic simulation of complex anatomic dissection or performance of surgical procedures. It can make learning surgical anatomy easier by allowing the student to explore the interrelations of various organ systems in perspectives not available through other standard teaching techniques. Eventually it will be possible to “dissect” a virtual cadaver, resulting in a very effective educational tool. The surgical simulator can be used in surgical residency training and research. A resident can practice a surgical procedure repeatedly until perfect before performing it on a patient. A researcher can attempt new surgical procedures repeatedly before the first attempts on an animal model. Teaching surgical skills becomes easier. Risks are fewer. Fewer animals are needed for research. In the current conventional configuration (HMD and DataGlove), the simulator is a “stand-alone” teaching system. It is envisioned that in the future, the simulator (or the Virtual Abdomen) could be connected to a Green Telepresence Surgery System,1 which does not use a HMD and DataGlove. This is an ideal human interface, for the same surgical workstation which will be used to perform real operative procedures through telepresence surgery could access a simulated virtual abdomen and practice surgical procedures on the computer-generated model. Also, a surgical simulator could be networked (like the military battle- training simulator SIMNET), so multiple surgeons could practice or do surgical research in the computer-generated virtual environment. These advantages are appealing in this era of animal-rights sensitivity and of fear of exposure to blood-borne diseases such as AIDS and hepatitis.