Portal de Periódicos da CAPES

A Conversation with … Nigel Packham PhD, NASA Safety Expert who Analyzed the Space Shuttle Columbia Disaster

2020; Lippincott Williams & Wilkins; Volume: 479; Issue: 2 Linguagem: Inglês

10.1097/corr.0000000000001408

1528-1132

Seth S. Leopold,

Health Systems, Economic Evaluations, Quality of Life

While contemporary healthcare systems seem focused more intently than ever on delivering value—defined as the best-possible results at the lowest-possible cost—I believe that if we asked our patients what they seek in a healthcare encounter, they might instead ask for the best-possible result at the lowest-possible risk. Surgeons are intimate with risk. It’s central to every decision we make, and a part of every intervention we consider. But our intimacy with risk doesn’t protect us from forming some basic misapprehensions about how risk works. Human nature means we’ll get this wrong some of the time. In recent editorials in Clinical Orthopaedics and Related Research®, I’ve discussed some examples of this, including the natural human tendency to have greater confidence in situations where we have less knowledge or ability [4], the temptation to dichotomize kinds of risk that clearly occur on sliding scales [6], and even the desire to attribute risk to causes that can’t possibly be responsible for the complications in question [5]. So perhaps we should look outside our specialty for different ways to think about risk. In this month’s “A Conversation with …”, I’m pleased to introduce CORR’s readers to Nigel Packham PhD, the National Aeronautics and Space Administration (NASA) Safety Expert who analyzed the Space Shuttle Columbia disaster [8], which tragically killed all seven crewmembers in 2003 when it disintegrated on re-entry.Nigel Packham PhD (Photo credit: NASA).Among many other things, Dr. Packham is an expert in “risk trades”, or the behaviors we need to assume when we can’t eliminate all risk. Indeed, with risk trades, we need to understand that there may be alternate paths to success, but that each of those alternate paths has its own degree of associated risk. How one acts in the face of a risk trade varies depending on whether one has moments or months to make one’s decision. Because of this, and because of the stakes involved, I believe all of us can learn from Dr. Packham, who is the Johnson Space Center’s Safety and Mission Assurance Associate Director, and who has looked at risk trades going back as far as the first lunar landing more than 50 years ago [13]. When risk goes bad—whether in surgery or spaceflight—we get disasters. And the analysis of these disasters is inadequately handled in most surgical settings. For example, although our morbidity and mortality (M&M) conference serves several important social functions [1, 3], to the degree that M&M often involves pinning responsibility on the person closest to the fan when the stuff hit it, we don’t learn as much as we might from these conferences. Contrast M&M with the top-to-bottom, all-angles approach that the National Transportation and Safety Board (NTSB) applies in the wake of aviation or railway disasters [12]. Analogously, Dr. Packham was principally responsible for a seminal report to the Johnson Space Center on the space shuttle program [8], which contextualized lessons in terms of cultures, processes, and products that could lead to (or impair) safety. Even before that, he was recognized at NASA with the 2009 Category 1 Quality and Safety Achievement Recognition for his leadership in the safety of human space flight, and, specifically, for taking steps to ensure that his findings from the Space Shuttle Columbia investigation resulted in lasting institutional change [2]. On a lighter note: Did I mention that Dr. Packham once led a crew of four who locked themselves inside a 20-foot diameter chamber for 3 months to test air, water, and waste recovery processes for long-duration missions [7, 9] (and in so doing managed to use sweat, feces, and urine in previously unimaginable—at least to me—ways)?Dr. Packham was one of four crew members who locked themselves inside a 20-foot diameter chamber for 3 months to test air, water, and waste recovery processes for long-duration missions [7, 9]. (Photo credit: NASA.)And he’s not just any rocket scientist; I believe Dr. Packham’s personal background may give him a perspective of particular relevance to readers of a surgical journal. In addition to his expertise in risk, he grew up in a medical household: His father was a urologist, his mother was an ophthalmologist, and his brother is a retired family physician. Dr. Packham “gets it” when it comes to what we do. Please join me in the conversation that follows with Dr. Nigel Packham, international expert on safety and risk trades in outer space. Seth S. Leopold MD:Can you explain “risk trades” briefly, and perhaps give an example of how a surgeon might use that concept to advantage as he or she thinks about the decisions he or she makes with patients? Nigel Packham PhD: The example I always give is the International Space Station (ISS), a joint venture of 15 countries from across the globe, was first launched in 1998, and has been continuously inhabited by astronauts since 2000 until present day. Obviously, just as human beings age, so do the machines and apparati that maintain the ISS in a habitable condition. When a critical pump that circulates fluid to cool systems inside the ISS showed signs of failing, the ISS Program was faced with a multitude of options, each of which carried with it its own degree of risk. Since the pump resided on the outside of the ISS, in the near vacuum of space, a spacewalk or extravehicular activity (EVA) would have to be performed to repair or replace it, if that was the chosen pathway. Other pathways under consideration included finding alternate means to cool critical equipment in the event that the pump permanently failed (but resulting in having to turn off other systems), or even decrewing the ISS until critical spares could be launched from the ground. Data had indicated that the inlet and outlet seals to the pump were leaking, but no prediction on useful remaining life could be made. Performing an EVA is the most-risky activity that crewmembers have to perform during their stay on-orbit. So, a risk trade had to be performed to compare the risk of allowing the pump to continue on with degraded performance, inevitably losing cooling capacity over time, versus performing an EVA to fix the problem. Eventually, the Program made the informed decision to perform the repair during an EVA. Consider the medical equivalent to this scenario, and please accept my sincere apologies in advance for the limited and most probably very wrong picture that I am about to paint. A 90-year old patient presents with hypotension. Angiography suggests mitral valve prolapse. What options are available to the cardiologist, or more importantly, the patient? Surgery? Pharmacological treatment? Further tests? No further treatment? The medical professional will use all of his/her knowledge to evaluate the risks associated with each of these potential courses of action, describe them to the patient and hopefully inform them of the risks associated with each respective approach. The medical professional would then recommend a particular course of action. Perhaps without realizing it, what they have just accomplished is a risk trade. The risk of say, performing surgery vice doing nothing at all except for treating the symptoms. Dr. Leopold:Even though surgeons deal with risk every day, I think we sometimes misperceive it. Sometimes, we may feel that it doesn’t apply to us[4], sometimes we conceptualize it incorrectly, such as when we treat sliding-scales of risk as though they’re binary[6], and sometimes our cultural blinders get in the way[5]. What are the most-common misunderstandings you’ve observed about how people perceive, treat, or react to risk, and what are some good heuristics to use to avoid those errors? Dr. Packham: What I have come to understand from my interactions with the medical community over several decades is that survivability rates are calculated using data in a statistical fashion. When the layman hears that the survivability for an individual surviving for 5 years after a specific procedure is 75%, they might rightly assume that it means that 3 out of 4 individuals will survive at least 5 years. What is often (and rightly so) omitted from that discussion is the confidence (statistically speaking) in that number. Unlike the medical case, NASA uses a probabilistic approach to defining risk. In many ways, this approach is driven by NASA’s small sample size as compared to medical data on a particular procedure, but it appears to have served us well. This probabilistic risk assessment allows the NASA engineer to assess, numerically, the likelihood/risk that a certain event will happen in a specific time period. Once individual risks for various events are known, an overall risk number can be crafted for the entire mission. For the majority of the Shuttle missions flown after the Columbia accident in 2003, a target of 1:200 for loss of the crew was considered to be an acceptable risk, which the Program Manager could accept. If the probabilistic risk assessment showed the risk to be greater than 1:200 (that is, a smaller number, like 1:50), then it did not mean that the mission could not be undertaken, simply that the risk had to be accepted by an authority higher than the Program Manager (up to and including the NASA Administrator). What does this imply? That just because the risk becomes greater than that acceptable by a Program Manager, it does not mean that the mission can’t be flown, simply that a higher authority has to accept the risk. It should also be noted that just because a threshold is crossed, it does not mean that the crew will be lost. There’s simply a higher risk that the crew could be lost. And while we are on the subject of the crew, they have the final decision on whether or not to fly the mission regardless of what the risk numbers are saying, or whatever the Program Manager or NASA Administrator might say. In practice, over the 30 years that I have been working at NASA, I have yet to hear an astronaut decline to get in a Shuttle cockpit because of any risk numbers. It therefore becomes incumbent upon the safety and mission assurance professional to act as the “angel on the shoulder” to combat the devil on the other side. This brings up another good point. The accuracy of your method of estimating risk depends on when you do the analysis and what tools you have available to you when you perform it. A disturbing example is the risk of the first Shuttle flight back in 1981. Using all of the analytical tools available to the Agency at the time of the first flight, the loss of crew was estimated to be somewhere between 1:1000 and 1:10,000. One could argue, especially considering that the later flights threshold was 1:200, that those odds were pretty good. Now fast forward to the end of the Shuttle Program—some 30 years and 135 flights later. Now using all the data accrued over the 30 years, plus the analytical tools that had developed over those years, a retroactive estimate of the risk of losing the crew of the first Shuttle flight was an astounding 1:12! Dr. Leopold:As you might know, surgeons evaluate their failures, but the most-common venue for this—something called the M&M conference—looks very different from the kinds of root-cause analyses and other processes that NASA and the NTSB might use when they assess disasters[11, 12]. M&M can be quaintly reminiscent of old-time aviation or the law of the sea; the captain of the ship takes responsibility for what almost always are systems-level breakdowns. What can surgeons learn from the approaches you use in the Safety and Mission Assurance directorate at NASA to learn from our complications? Dr. Packham: It is remarkably timely that I write upon my return from NASA’s Day of Remembrance—a day in which we celebrate the lives of the crews of the three missions that we have lost—Apollo 1, Challenger, and Columbia. Apart from a somber reminder of the 17 souls that we have lost, it serves to reinforce our absolute necessity to ensure that we don’t do the same thing again. Something very important happened after the Columbia accident in 2003, which failed to happen after Challenger in 1986. NASA purposefully chose to look at the accident from the viewpoint of crew survivability. I was fortunate enough to be asked to lead a small team of specialists who were tasked with understanding in the greatest detail, what happened to the crew module and the crew of Columbia as she disintegrated over Texas. Although the technical disagreements were many, they were certainly to be expected. After all, it’s not often that an accident happens at an altitude of about 200,000 feet and at a velocity approaching Mach 20. What was not expected, however, was the extraordinary amount of pushback from several communities within the Agency when I insisted that this report be publicly available. My reasoning was that we had learned some extraordinarily important lessons that had direct transportability to other space-faring ventures. The counterarguments were equally remarkable both for their blunt nature and their incredulity. The basic message was “we don’t tell bad things about this Agency”. You should now know that when Challenger happened, the debris that was collected was immediately placed inside a concrete silo at the Kennedy Space Center and was left sealed. It has only been opened once since 1986, to retrieve a specific piece of debris for an exhibit at the Kennedy Space Center visitors center. Having not started with the Agency until 1991 (5 years after the Challenger accident), I can only surmise that the voices calling for “no bad news” were stronger then than they were after Columbia, but the concerning thing was that they were still there in 2003. So, as an additional reinforcement of our desire to inform whoever would listen about our mistakes, we published “Significant Incidents and Close Calls in Human Spaceflight” [10] in 2008, which shows graphically where not only our fatalities have occurred by phase of the mission, but also every case that we know of where an incident had the potential to either cause harm to a crew or result in a fatality. This product, which is now interactive (https://sma.nasa.gov/SignificantIncidents/index.html) is updated every 6 months in an attempt to keep current with new spaceflight ventures. Finally, the Columbia story—a presentation by one of the Columbia crew survival team members—has now reached an audience of more than 75,000 people, worldwide. Dr. Leopold:Your work has many parallels with the work surgeons do: It has life-and-death implications, over the span of a career perhaps you’ve looked back on some decisions you’ve made with regret, it must be hard to get back into the game after a serious or costly error. What approaches have you used in those circumstances? Dr. Packham: It may be worthwhile to explore how NASA “got back in the game” after the accidents that we experienced. Specifically, for the Shuttle Program, which spanned 30 years from 1981-2011 with 135 flights, we decided to take a retrospective look at how the safety community responded after Challenger and Columbia. We took an estimated measure of the size and effectiveness of the safety community throughout those 30 years (and actually even before the first flight in 1981). Essentially a grand “M&M Board”, the product, the “Shuttle Legacy Graphic” [11] clearly shows that with success comes complacency. As the flight rate—the number of missions flown each year—increased, the size and effectiveness measure steadily declined. With five missions flown in 1984, nine in 1985, and roughly the same planned for 1986, the Challenger accident left the Agency grounded for 2 years until return to flight. But what was obvious was a rapid and large increase in the size of the safety community after the accident. Fast forward 17 years, and the same uncanny relationship appeared prior to and following the Columbia accident. After the second return to flight in 2005, there were only 6 years remaining in the Program. Disturbingly, the same gradual erosion of the size and effectiveness of the safety community was evident leading one to ask whether we would have had another accident had we continued the Program for say another 5 years. This saw-tooth nature of the measure becomes even more apparent when you represent the measure as the width of a bar over time, as we did in the JSC Safety and Mission Assurance Space Shuttle Program Legacy Report [11]. The resultant “turbine chart” as it has now become known identified a multitude of environmental conditions that were in play over the course of those 30 years. I show this graphic when I visit organizations that one wouldn’t normally associate with human spaceflight (oil and gas, the chemical industry, commercial aviation, and various aspects of the practice of medicine). I challenge each of them to look into their histories to see how their safety communities reacted to accidents and failures, and more importantly, to investigate what environmental factors were in play when those failures occurred. A classic example could be the oil and gas industry’s response to the Deepwater Horizon event in 2010. I know firsthand that the job recruitment for safety professionals went rampant. Indeed, the Chief Safety Officer for the Shuttle Program, who had worked for me since I had joined the safety program, was enticed away from a long and highly successful civil service career to become an executive in an oil and gas company’s safety organization. That individual rejoined the NASA community 6 years later after the environment had once again shifted (lower oil prices). While NASA’s responses to their Shuttle accidents was entirely understandable, it was also somewhat misguided. The idea that simply doubling or tripling the size of your safety community may appear to be appropriate, simply having three times the number of people doing the things they always did without a change in an understanding of the true risks involved is simply shortsighted. A parallel could be found in various mandates handed down to the medical community in terms of the number patients that had to be seen in a given workday. While the bureaucrats who own these mandates might clearly believe that they result in more patients being seen, they fail to see the underlying fallacy that a shorter visit with the healthcare provider means an inherent increase in risk to those same patients. Dr. Leopold:You lived in a medical household. Your father was a urologist; your mother, an ophthalmologist; and your brother, a family physician. Looking back, is there anything you took from your experiences at home that you now bring to your current role at NASA? Dr. Packham: In my late teens I had many occasions to see my father operate. Most of these were associated with my driving him to the hospital, late at night, to see patients who were not at all well. Although I did not recognize it at the time (late 1970s to early 80s), when I look back today, I now recognize that the surgeon in the operating room was the person in charge, and that everything revolved around that surgeon. Although I think of my father as someone who welcomed the opinions of others, when it came to surgery, it was very much his way. Today, I hope that autocracy has softened somewhat, without negating the fact that the surgeon is still the accountable body. In my world, even though the Commander of a space mission has the authority to make “command” decisions, in practice, such decisions are rarely based upon the Commander’s simple authority. After all, how can one individual know everything about such a complex engineering marvel to have the audacity to make informed decisions by themselves? The answer is that he or she can’t and shouldn’t make decisions when they involve multiple competencies represented by each team member’s expertise. Only then can a true understanding of risk inform the right decision.