Portal de Periódicos da CAPES

Defining Quality in Radiology

2007; Elsevier BV; Volume: 4; Issue: 4 Linguagem: Inglês

10.1016/j.jacr.2006.11.014

1558-349X

C. Craig Blackmore,

Advanced X-ray and CT Imaging

The introduction of pay for performance in medicine represents an opportunity for radiologists to define quality in radiology. Radiology quality can be defined on the basis of the production model that currently drives reimbursement, codifying the role of radiologists as being limited to the production of timely and accurate radiology reports produced in conditions of maximum patient safety and communicated in a timely manner. Alternately, quality in radiology can also encompass the professional role of radiologists as diagnostic imaging specialists responsible for the appropriate use, selection, interpretation, and application of imaging. Although potentially challenging to implement, the professional model for radiology quality is a comprehensive assessment of the ways in which radiologists add value to patient care. This essay is a discussion of the definition of radiology quality and the implications of that definition. The introduction of pay for performance in medicine represents an opportunity for radiologists to define quality in radiology. Radiology quality can be defined on the basis of the production model that currently drives reimbursement, codifying the role of radiologists as being limited to the production of timely and accurate radiology reports produced in conditions of maximum patient safety and communicated in a timely manner. Alternately, quality in radiology can also encompass the professional role of radiologists as diagnostic imaging specialists responsible for the appropriate use, selection, interpretation, and application of imaging. Although potentially challenging to implement, the professional model for radiology quality is a comprehensive assessment of the ways in which radiologists add value to patient care. This essay is a discussion of the definition of radiology quality and the implications of that definition. IntroductionSince the publication of the Institute of Medicine's 1999 report To Err Is Human: Building a Safer Health System [1Kohn L. Corrigan J. Donaldson M. To err is human: building a safer health system. National Academy Press, Washington, DC1999Google Scholar], there has been increased interest in quality in medicine. The report's authors estimated that there may be as many as 100,000 preventable deaths each year at medical centers in the United States. Preventable deaths are caused by errors of omission, commission, communication, and other sources. This report highlighted the potential benefit to health care in the United States from improvements in the quality of care [1Kohn L. Corrigan J. Donaldson M. To err is human: building a safer health system. National Academy Press, Washington, DC1999Google Scholar, 2Kohn L.T. The Institute of Medicine report on medical error: overview and implications for pharmacy.Am J Health Syst Pharm. 2001; 58: 63-66PubMed Google Scholar].Coincident with increased awareness of a quality deficit is the continued rise in health care costs. As of 2003, the United States was spending 15.2% of its gross domestic product on health care [3World Health OrganizationThe world health report 2006. World Health Organization, New York, NY2006: 185Google Scholar]. This is by far the highest of all developed nations in the world, at least 50% higher than the number two country, Switzerland, and nearly double the average among developed countries of 8.3% [3World Health OrganizationThe world health report 2006. World Health Organization, New York, NY2006: 185Google Scholar]. Despite these expenditures, however, the World Health Organization in 2006 reported that the United States lags behind most other developed nations in the important health outcomes of life expectancy and infant mortality and, as of 2000, had only the 37th best health care in the world [3World Health OrganizationThe world health report 2006. World Health Organization, New York, NY2006: 185Google Scholar]. It is not unexpected that payers repeatedly question the value and the quality of the health care being provided in this country. Coincident with these rising health care costs is an escalation in growth in imaging procedures. Imaging procedure volume is expected to nearly double in the decade from 1998 to 2008.One response to the inadequate quality and the high expense of US medical care has been the establishment of pay-for-performance programs. These programs feature lower payments to lower quality providers and presumably increased payments to providers that can demonstrate higher quality. The goal of such programs is to lower cost but also, importantly, to force improvement in provider quality.Pay-for-performance programs require quality metrics to function. In response to a proposed 4.9% cut in professional fee components from the Centers for Medicare and Medicaid Services, the American Medical Association negotiated an agreement whereby the planned cut would be canceled, and in response, the American Medical Association would produce 140 quality metrics during calendar year 2006, which could be implemented as part of pay-for-performance schemes [4Moser J.W. Wilcox P.A. Bjork S.S. et al.Pay for performance in radiology: ACR white paper.J Am Coll Radiol. 2006; 3: 650-664Abstract Full Text Full Text PDF PubMed Scopus (36) Google Scholar].In radiology, the above-mentioned factors have led to increased interest in the development of radiology quality metrics. The ACR now has a metrics committee charged with developing radiology quality metrics [4Moser J.W. Wilcox P.A. Bjork S.S. et al.Pay for performance in radiology: ACR white paper.J Am Coll Radiol. 2006; 3: 650-664Abstract Full Text Full Text PDF PubMed Scopus (36) Google Scholar]. An additional independent group, the Radiology Quality Summit, convened for the first time in September of 2005 in Sun Valley, Idaho, with a manifesto emphasizing developing methods of measuring and improving radiology quality [5Johnson C.D. Swensen S.J. Applegate K.E. et al.Quality improvement in radiology: white paper report of the Sun Valley Group meeting.J Am Coll Radiol. 2006; 3: 544-549Abstract Full Text Full Text PDF PubMed Scopus (35) Google Scholar].The implications of the development of radiology quality metrics, however, go beyond pay-for-performance and current reimbursement schemes. As organized radiology moves toward defining radiology quality, in effect, radiology is defining for itself the role and responsibilities of radiologists. By defining quality radiology, the ACR and others are defining, on some level, the standards that must be met to function as a radiologist. In the past, quality has primarily been defined through credentialing. Certification from the American Board of Radiology, possibly supplemented by a certificate of added qualification, was the basic quality standard in radiology. The development of these new quality metrics will supplement and in some ways supplant credentialing as the measure of radiologists' qualifications.On an even broader level, proposed quality metrics will define roles that radiologists should function in and therefore will define the profession of radiology for future generations. As turf battles continue in radiology, the quality metrics defined by radiology can be used to argue either for or against different providers' providing imaging services. Nonradiologists could evaluate themselves under ACR or other radiology metrics and argue that they practice quality radiology. Thus, if radiologists are to argue that they provide superior quality radiology, they must ensure that quality metrics encompass the breadth of activities that differentiate true quality as it relates to patient care.In current discussions of radiology quality metrics, two broad frameworks have evolved. These can be labeled the radiologist "production" approach and the radiologist "professional" approach. The objectives of this essay are to describe and contrast these differing perspectives on radiology quality and to define quality metrics under the different models. I also seek to explain why radiology as a profession needs to be proactive in defining quality.Radiologist Production ModelIn the radiologist production model, radiologists are an integral part of a production process whereby images are produced and interpreted, and results are communicated. Patients arrive for imaging, and radiologists carry the responsibility for supervising the technical aspects of the imaging, the accurate interpretation of the imaging studies, the generation of reports of the examinations and findings, and the communication of the results to referring clinicians. Radiologists are responsible for the safety of patients while they are in radiology departments, as well as the satisfaction of patients, their family members, and their providers, with the whole imaging experience. The final product of this process is the radiology report. Quality markers are easily defined from the technical aspects of imaging and from the production of the report, and include radiation dose, the number of repeat images required, patient satisfaction, the ease of scheduling, the accuracy of interpretation, and the promptness and clarity of communicated reports (Table 1). In this manufacturing chain, a radiologist may serve in a managerial role and also in a technical role as an interpreter of images. Because of the parallels to manufacturing, many of the same approaches to quality, such as total quality management [6Seltzer S.E. Kelly P. Adams D.F. et al.Expediting the turnaround of radiology reports: use of total quality management to facilitate report signing.AJR. 1994; 162: 775-781Crossref PubMed Scopus (51) Google Scholar] and Six Sigma [7Cavagna E. Berletti R. Schiavon F. Scarsi B. Barbato G. Optimized delivery radiological reports: applying Six Sigma methodology to a radiology department.Radiol Med (Torino). 2003; 105: 205-214PubMed Google Scholar], may be appropriate.Table 1Radiologist production model metricsStep in Production ChainElementsSample MetricsPatient accessTime and ease of scheduling an appointmentFirst and third available appointmentsCommunication between referring provider and radiology departmentBillable indications provided with imaging ordersPreimaging planningCommunication with patientsCompliance with preimaging instructions (NPO, bowel preparation, hold medications, etc)Patient experience before imagingWait timesPatient imagingSafetyHand washingRadiation doseRepeat rates, CT dose estimates, ACR technical standardsProtocol selectionImaging callbacks, inadequate studiesContrast administration safetyContrast reaction, nephropathy, extravasationInterpretationExpert interpretationAccuracy, peer double reads, imager credentialingReport generationTimeliness of reportTime from imaging to report finalizationClarity of reportUse of structured reports, referring provider satisfaction with reportCommunication of important, unexpected findingsDocumentation of communicationNote: CT = computed tomography; NPO = nothing by mouth. Open table in a new tab The radiologist production model is clearly important and encompasses much of the daily activity of practicing radiologists. Important and obvious metrics can be derived from this process, measured, and potentially used as targets for quality improvement and error avoidance. Many common practice management metrics, including report turnaround times, scheduling delays, repeat imaging rates, and the use of film markers, are easily translated into quality metrics under the production model. In addition, patient satisfaction can be measured and is one reflection of quality, as is the safety of patients and the appropriate minimization of radiation dose. The accuracy of interpretation (or at least interobserver agreement) is also measurable to some extent using approaches such as the ACR's RADPEER® system and is an important component of the radiologist production model.Although it is important, the radiologist production model is limited. The intrinsic assumption of the production model is that the final product is a timely and accurate radiology report produced in conditions of maximum patient safety and communicated to a patient's health care provider. The radiologist production model suggests that if the function of a radiologist is to produce an accurate report, then the use of radiologists has no advantage over other approaches that can produce reports of similar accuracy, safety, and timeliness. The report itself becomes a commodity, and the radiologist's role in patient care is limited to producing this fungible output. By focusing on the report as the output of diagnostic imaging practice, a radiologist's value is defined as purely in terms of being part of the production chain. By extension, radiologists, though skilled, are labor and replaceable with less expensive labor either through nonphysician providers and outsourcing or through the further development of computer-assisted diagnostic software.It is also important to note that current reimbursement is based solely on this production model. Radiologists are paid on the basis of the number and complexity of imaging studies they interpret, but they are paid only upon the production of the final product, which is the radiology report.Radiologist Professional ModelBy contrast, the radiologist professional model uses a broader definition of the role of radiologists and therefore how quality in radiology should be measured. Under the radiologist professional model, radiologists are physicians who are experts on the use of imaging for diagnosis, specialists in imaging acquisition and interpretation, and consultants on the application of imaging information to clinical care. Accordingly, radiologists have an explicit role in using imaging to improve the health of patients. This includes the functions of radiologists under the production model but also incorporates sharing responsibility for determining which subjects should be imaged, what imaging modalities and approaches should be used, how imaging studies should be interpreted, and how imaging study results should change patient care. A radiologist professional cannot practice in isolation, and effectiveness in the radiologist professional model requires interaction with the other medical providers sharing responsibility for a patient's care.The radiologist professional model as a concept is not likely to spark controversy among radiologists. No doubt many of us see ourselves as working in this model. However, the new focus on quality metrics means that radiologists and other physicians will now be evaluated on the various aspects of quality. For radiologist professionals, this means that they must now take responsibility for and be held accountable for which patients are imaged, how they are imaged, and even how the information is used in patient care. If radiologists are to function as physicians and professionals and bear partial responsibility for all of these functions, then quality in radiology should also include the measurement of these roles (Table 2).Table 2Radiologist professional model metricsComponent of Professional ModelElementsSample MetricsWho should be imaged?Use of evidence-based imaging practiceAdherence to clinical prediction rules, compliance with local and national practice guidelinesWhat imaging approach?Selection of optimal imaging strategyDiagnostic yield, adherence to clinical prediction rules, compliance with local and national practice guidelines, rate of additional (induced) imaging studiesImaging productionProduction of imaging studies and reportsAll the metrics in Table 1How are studies interpreted?Balance of sensitivity and specificityRate of false-positive and false-negative diagnoses, diagnostic yieldTailoring of interpretation to clinical scenarioRatios of adverse false-positive outcomes (ie, negative laparotomy result, negative biopsy result) vs rate of adverse false-negative outcomes (ie, delayed diagnosis)Patient outcomeEffect of imaging on patient careRates of specific therapeutic interventions after imaging (ie, use of thrombolytics for nonhemorrhagic stroke, biopsy or appropriate follow-up for lung nodule, repositioning of endotracheal tube after misplacement identified on chest radiograph)Effect of imaging on outcomeRates of specific medical errors after imaging (ie, negative laparotomy for appendicitis, nontherapeutic knee laparoscopy, intracranial hemorrhage after thrombolytics for stroke) Open table in a new tab Who Should Be Imaged?With training in imaging physics and in-depth understanding of imaging, radiologists should be uniquely suited to influence decision making as to which patients undergo imaging. Radiologists should be involved in the development of evidenced-based guidelines on local and national levels, advising clinicians on the ideal use of imaging. Radiologists should also be involved in the research that underlies such guidelines, including the development, validation, and implementation of the clinical prediction rules to determine which subjects need imaging [8Blackmore C. Clinical prediction rules in trauma imaging: who, how and why?.Radiology. 2005; 235: 371-374Crossref PubMed Scopus (32) Google Scholar]. Clearly, radiologists are involved in such processes, although the lead role in the development of clinical prediction rules for imaging has often come from outside radiology [9Stiell I. Wells G. Vandemheen K. et al.The Canadian C-spine rule for radiography in alert and stable trauma patients.JAMA. 2001; 286: 1841-1848Crossref PubMed Scopus (861) Google Scholar, 10Stiell I.G. Greenberg G.H. McKnight R.D. et al.Decision rules for the use of radiography in acute ankle injuries Refinement and prospective validation.JAMA. 1993; 269: 1127-1132Crossref PubMed Scopus (290) Google Scholar, 11Hoffman J. Mower W. Wolfson A. Todd K. Zucker M. Validity of a set of clinical criteria to rule out injury to the cervical spine in patients with blunt trauma.N Eng J Med. 2000; 343: 94-99Crossref PubMed Scopus (0) Google Scholar, 12Haydel M. Preston C. Mills T. Luber S. Blaudeau E. DeBlieux P. Indications for computed tomography in patients with minor head injury.N Engl J Med. 2000; 343: 100-105Crossref PubMed Scopus (784) Google Scholar].Under the production model, a radiologist is a participant in a manufacturing chain, a passive role whereby studies are ordered by referring physicians and performed by the radiologist without questioning. The radiologist professional model implies a more active role influencing the process of image ordering. This may occur on the local level through medical-centered committees, case conferences, personal relationships with referring clinicians, and hospital administration, and it may occur on the national level through payers, organized medicine, and government. Unfortunately, despite the depth of knowledge in imaging that radiologists possess, influencing the selection of subjects for imaging has not always been a role in which radiologists are recognized, and certainly not one for which they are compensated.The radiologist professional model demands that the determination of the quality of radiology practice include assessments of whether inappropriate patients are imaged. When there is strong evidence supporting the determination of appropriate imaging, inappropriate imaging can be directly measured and used as a quality marker. For example, in emergency radiology there are internationally validated clinical prediction rules for trauma to the ankle [10Stiell I.G. Greenberg G.H. McKnight R.D. et al.Decision rules for the use of radiography in acute ankle injuries Refinement and prospective validation.JAMA. 1993; 269: 1127-1132Crossref PubMed Scopus (290) Google Scholar], knee [13Stiell I.G. Greenberg G.H. Wells G.A. et al.Prospective validation of a decision rule for the use of radiography in acute knee injuries.JAMA. 1996; 275: 611-615Crossref PubMed Google Scholar], head [12Haydel M. Preston C. Mills T. Luber S. Blaudeau E. DeBlieux P. Indications for computed tomography in patients with minor head injury.N Engl J Med. 2000; 343: 100-105Crossref PubMed Scopus (784) Google Scholar, 14Stiell I.G. Wells G.A. Vandemheen K. et al.The Canadian CT head rule for patients with minor head injury.Lancet. 2001; 357: 1391-1396Abstract Full Text Full Text PDF PubMed Scopus (1043) Google Scholar], and cervical spine [9Stiell I. Wells G. Vandemheen K. et al.The Canadian C-spine rule for radiography in alert and stable trauma patients.JAMA. 2001; 286: 1841-1848Crossref PubMed Scopus (861) Google Scholar, 11Hoffman J. Mower W. Wolfson A. Todd K. Zucker M. Validity of a set of clinical criteria to rule out injury to the cervical spine in patients with blunt trauma.N Eng J Med. 2000; 343: 94-99Crossref PubMed Scopus (0) Google Scholar], all of which can be used to determine which subjects do not need imaging, on the basis of simple clinical criteria. It is relatively easy to perform simple chart audits in such patients to ensure that these clinical criteria for appropriate imaging are met in subjects who are imaged. A simple quality metric can be derived consisting of the number of patients in whom an indication is documented as the numerator and the total number of patients imaged for a given clinical scenario as the denominator. For example, the New Orleans Criteria are an internationally validated clinical prediction rule to determine which subjects with minor head trauma require imaging [12Haydel M. Preston C. Mills T. Luber S. Blaudeau E. DeBlieux P. Indications for computed tomography in patients with minor head injury.N Engl J Med. 2000; 343: 100-105Crossref PubMed Scopus (784) Google Scholar]. This clinical prediction rule is much more detailed and exclusive than the lengthy list of indications that will currently lead to reimbursement. Adherence to the New Orleans Criteria can serve as a quality metric for the performance of imaging on only indicated patients in emergency departments.Ideally, all imaging would be based on strong evidence. Unfortunately, however, the evidence basis for much of radiology practice is limited [15Medina L.S. Blackmore C.C. Principles of evidence based imaging.in: Medina L.S. Blackmore C.C. Evidence-based imaging: optimizing imaging for patient care. Springer-Verlag, New York, NY2006Google Scholar, 16Blackmore C.C. Medina L.S. Evidence based radiology and the ACR Appropriateness Criteria.J Am Coll Radiol. 2006; 3: 505-509Abstract Full Text Full Text PDF PubMed Scopus (41) Google Scholar]. The book Evidence-Based Imaging [15Medina L.S. Blackmore C.C. Principles of evidence based imaging.in: Medina L.S. Blackmore C.C. Evidence-based imaging: optimizing imaging for patient care. Springer-Verlag, New York, NY2006Google Scholar], as well as the US Public Health Service Task Force on Preventive Services and the Cochran Collaboration, serve as important repositories for the evidence supporting imaging. Barring strong evidence, including validated clinical prediction rules, consensus must be reached as to the appropriate indications for imaging. The consensus-based ACR Appropriateness Criteria® are an important starting point for guiding the use of imaging, and there are also many other guidelines from other medical societies and interest groups. Once imaging indications are agreed on at either the local or the national level, these too can form the basis of assessing the quality of the selection of subjects for imaging.Intrinsic to the role of radiologists in determining whether imaging is appropriate for a given patient is the potential for discouragement, or even the denial of imaging services by radiologists in those situations in which appropriate indications are not identified. Denial of imaging is not a role in which radiologists may be comfortable and realistically must be a collaborative process, involving radiologists as well as referring clinicians, that occurs before the patient arrives for imaging. Furthermore, such decisions about who is to be imaged are best made in advance at the local or national level rather than on an individual level.It also needs to be acknowledged that under the production model that currently serves as the basis for reimbursement, there is no financial incentive to radiologists or clinicians to limit the use of imaging. Intrinsic to the radiologist professional model is the development of a mechanism to align financial incentives with the appropriate use of imaging. Pay for performance represents a potential method of realigning these incentives, because the appropriate use of imaging can be rewarded as quality care, whereas the inappropriate use of imaging could be penalized [4Moser J.W. Wilcox P.A. Bjork S.S. et al.Pay for performance in radiology: ACR white paper.J Am Coll Radiol. 2006; 3: 650-664Abstract Full Text Full Text PDF PubMed Scopus (36) Google Scholar]. Using the example of head computed tomography (CT) cited above, radiologists or radiology groups that perform imaging on patients who do not meet one of the clinical criteria described by the New Orleans group could be considered to be practicing lower quality radiology, with the associated financial consequences.Which Imaging Is Appropriate?A second role for radiologist professionals is the determination of which specific imaging tests are appropriate if subjects require imaging. This is a consulting role in which a radiologist works with a referring clinician to determine the best diagnostic approach for a patient with a given set of clinical symptoms and signs. Such consultation may occur locally, often informally, in the course of daily practice, or it may occur at a broader level through the development of evidence-based guidelines. Again, however, the radiologist professional model requires more than informal and occasional consultation; it demands responsibility for the selection of the correct imaging approaches for patients. Quality in radiology therefore is defined not simply by the availability of a radiologist for consultation, but by the reality of appropriate imaging strategies in subjects referred to the radiologist for care.An example of this approach is the use of cervical spine CT in trauma patients. The determination of which victims of trauma require imaging can be reliably made on the basis of clinical prediction rules developed by the Ottawa group [17Stiell I. Wells G. Vandemheen K. et al.The Canadian C-spine rule for radiography in alert and stable trauma patients.JAMA. 2001; 286: 1841-1848Crossref PubMed Google Scholar] and the National Emergency X-Radiography Utilization Study [18Hoffman J. Mower W. Wolfson A. Todd K. Zucker M. Validity of a set of clinical criteria to rule out injury to the cervical spine in patients with blunt trauma.N Engl J Med. 2000; 343: 94-99Crossref PubMed Scopus (964) Google Scholar]. Once the decision has been made, however, that imaging is appropriate, there are two commonly used approaches, radiography and CT. Computed tomography has been shown to be cost effective in high-risk patients, but the high imaging cost and high radiation dose render CT inappropriate for use in all subjects. A validated clinical prediction rule can be used to identify subjects who are at high risk for cervical spine fracture (>4%), in whom CT is the appropriate choice. On the basis of such information, the appropriate use of CT of the cervical spine in trauma patients can be assessed by looking at the diagnostic yield of the imaging study. If CT is appropriate in the patients who are at 4% or greater risk for fracture, the diagnostic yield of CT should be at least 4%. A lower yield would imply that CT is being used too broadly and that quality has been sacrificed. Of course, this approach requires evidence as to the appropriate diagnostic yield for an imaging study, which is unfortunately lacking for many imaging issues [19Blackmore C.C. Avey G.A. Imaging of the spine in victims of trauma.in: Medina L.S. Blackmore C.C. Evidence based imaging: optimizing imaging for patient care. Springer-Verlag, New York, NY2006Google Scholar].How Should Studies Be Interpreted?Next in the quality assessment under the radiologist professional model is how imaging studies are interpreted. In the radiology production model, imaging reports are fungible, and quality might be judged simply on the basis of accuracy. However, in the radiologist professional model, radiologists provide individualized care for specific patients and are able to adjust their interpretations as appropriate for given clinical scenarios.There is enormous variability in the interpretation of diagnostic imaging evaluations. This variability can be thought of as occurring in two categories. The first of these is voluntary variability, meaning that this variability is based on the choices of an individual radiologist as to whether to emphasize sensitivity or specificity. In other words, different radiologists make the choice to operate at different points on the receiver-operating characteristic curve. A radiologist who emphasizes sensitivity will have important differences in the interpretation of studies with a radiologist who chooses to emphasize specificity. The other component of variability is involuntary variability, which is a function of differences in the skill, the equipment, the interpreting environment, and even the imaged patient's characteristics.The voluntary component of variability is another point at which a radiologist professional can influence the medical care of a patient. Using utility theory, one can determine the optimal point on the receiver-operating characteristic curve at which a study should be interpreted from the prevalence of disease in the population and the ratio of the utilities of false-positive and false-negative im