Cover story: Verification and Validation: The Chasm between Protocol and Practice
2012; Lippincott Williams & Wilkins; Volume: 65; Issue: 3 Linguagem: Inglês
10.1097/01.hj.0000412693.87680.7e
ISSN2333-6218
Autores Tópico(s)Hearing Loss and Rehabilitation
ResumoFigure: © Images.com/Dave CutlerAlthough the two largest audiology associations stressed the importance of verification and validation in hearing aid fittings years ago, audiologists are largely noncompliant with recommendations to use these best practices.Figure: Larry E. HumesThe American Speech-Language-Hearing Association first emphasized their value in 1998, followed by the American Academy of Audiology in 2006. In just the past five years alone since AAA adopted its practice guidelines, nearly half of audiologists surveyed by ASHA said they do not use verification and validation processes. Two ASHA surveys indicated that only about 55 percent of audiologists performed verification with REM, with no indication of how frequently this service was provided. Validation of outcomes with self-reported questionnaires was practiced by only about 36 percent of the audiologists surveyed. (Audiology Survey: Frequency Report [2008, 2010]. ASHA: Rockville, MD.) The survey didn't ask about the use of aided and unaided speech-understanding measures by audiologists, so it is possible that a higher percentage of audiologists make use of some form of validation other than a self-reported survey. ASHA surveyed a little more than 2,000 audiologists, about 80 percent of whom indicated that they fit and dispense hearing aids. These disappointing figures on verification and validation measures by practicing U.S. audiologists are consistent with those reported for dispensing audiologists in a much smaller sample. (Hear Rev 2006;13[6]:16.) So where to start? Despite renewed interest in the importance of verification and validation for optimizing successful outcomes for hearing-aid wearers (ASHA Leader 2009 Sept 1; http://bit.ly/HumesAmos; Hear Rev 2010;17[4]:12, 2011;18[4]:12, 2011;18[6]:10; Hear Rev 2011;18[7]:10), it's best to begin with the basics. Verification generally refers to the process of confirming that the hearing aid is acoustically working as desired on the hearing aid wearer's ear. One can verify the function of hearing aids on the patient in terms of desired gain and maximum output, in which “desired” function is typically prescribed by an established algorithm or conceptual framework. Although not the only means of verification, the most widely adopted and recommended procedures for verification make use of real-ear measurement (REM), especially for matching measured gain to prescriptive targets. Current practice guidelines for the American Speech-Language-Hearing Association and the American Academy of Audiology, for example, endorse REM as the recommended procedure for gain verification. Verification in some cases can be performed to confirm the function of specific electroacoustic features of the hearing aids, such as the function of directional microphones. (Ear Hear 2004;25[2]:147; Hear Rev 2011;18[4]:12.) Validation, on the other hand, establishes that well fitted hearing aids, confirmed via verification, lead to positive outcomes for the hearing aid wearer. ASHA and AAA advocated this important process in hearing aid fitting protocols, and it is somewhat independent of verification. Certainly, it is accepted that verification of appropriate acoustical function of the hearing aid on the wearer's ear does not guarantee positive outcomes for the wearer. Outcomes are typically documented by using behavioral measures, such as speech-understanding scores, in unaided and aided listening conditions, or by obtaining self-reported measures of hearing-aid benefit, satisfaction, or usage. Typically, a complete evaluation of outcomes requires both types of measures because evaluations of speech-understanding improvements are not strongly correlated with self-reported outcome measures. (Ear Hear 1992;13[3]:131; Trends Amplif 2003;7[2]:41; Hearing Care for Adults 2006. Stafa, Switzerland: Phonak AG; 2007.) A plethora of evidence supports the importance of verification and validation processes, and the focus here is on the verification and validation of hearing aid fits in older adults, who purchase about two-thirds of the hearing aids sold in the United States. Somewhat surprisingly, the scientific evidence backing verification and validation is limited and weak, despite recent endorsements by various experts in audiology and in the hearing-aid industry, myself included. VERIFICATION Using a common internationally recognized yardstick to measure the quality of scientific evidence (Evidence-based Medicine: How to Practice and Teach EBM. 2d ed. Edinburgh: Churchill Livingstone; 2000) or a similar system tailored to evaluating evidence in audiology (J Am Acad Audiol 2005;16[7]:419; Evidence Based Practice in Audiology. San Diego: Plural Publishing; in press), the evidence supporting the verification process is sparse, and the quality of the evidence is generally low. A systematic review of the literature (J Am Acad Audiol 2005;16[7]:461) for the verification of maximum output using unaided loudness discomfort levels (LDLs) only identified three studies that adequately addressed this issue, and even then, the level of evidence for each was low. Many have argued, however, that it is not necessary to obtain unaided LDLs to limit maximum output. (Ear Hear 1998;19[4]:255; 1998;19[4]267.) Instead, one can estimate maximum output from thresholds, and then verify the adequacy of the output setting behaviorally, typically by presenting stimuli at 80-90 dB SPL at various frequencies and asking the wearer to indicate whether these stimuli are uncomfortably loud. Recent research, however, raises questions about the validity of this behavioral verification process for maximum output. (Int J Audiol 2010;49[1]:14.) Using REM for gain verification doesn't fare much better. Only limited and weak evidence supports this recommended practice. Mueller conducted a systematic review focusing on differences in prescriptive procedures, all of which employed REM to verify gain rather than on the importance of the use of REM per se on outcomes. (J Am Acad Audiol 2005;16[7]:448.) Of course, at the root of using REM for gain verification is that when initial fittings differ from the target prescription, the audiologist will seek to improve the match to target with fine-tuning adjustments. Importantly, in a descriptive study, Aazh and Moore demonstrated that such fine-tuning of gain does, in fact, result in a higher percentage of matches to gain targets. (J Am Acad Audiol 2007;18[8]:653.) Initially, 36 percent of the fittings for 42 ears were within (±10 dB) prescribed gain targets at all frequencies from 250-4000 Hz, but this improved to 83 percent with fine-tuning of hearing-aid gain. It appears that audiologists are capable of fine-tuning the gain of hearing aids to obtain a better match to prescribed targets for most hearing aid wearers. The central question, however, is, what is the evidence that doing so leads to improved outcomes? Once again, the evidence base is surprisingly sparse, a conclusion supported recently by the systematic review of the literature performed by Knudsen et al. (Trends Amplif 2010;14:127.) In their thorough review of the literature on factors influencing hearing aid satisfaction and usage in adults, as well as seeking help for a hearing aid purchase, only two studies identified by their review involved aspects of the fitting process, and neither involved the initial fine-tuning of gain with REM. Some studies have addressed the impact of incorporating post-fit fine-tuning several weeks after the fitting (Am J Audiol 2001;10[1]:13; Eur Arch Otorhinolaryngol 2009;266[6]:907; Trends Amplif 2011 Dec 28 [Epub Ahead of Print]), but this fine-tuning was typically driven by wearer complaint, and does not pertain to the use of REM to fine-tune hearing-aid gain to obtain better initial matches to target. Cox, Alexander, and Gray, on the other hand, assessed a variety of patient and protocol characteristics, including the impact of the quality of the match between observed and prescribed gain on a range of self-reported outcome measures in 205 hearing-aid wearers. (Ear Hear 2007;28[2]:141.) Although the quality of fit, which is directly related to the success of the REM fine-tuning process at fitting, was significantly related to the “success component” of hearing aid outcome, this factor made minor contributions, explaining only about 10 percent of the individual differences in success. Unfortunately, the state of the science today supporting protocols, including verification of maximum output and gain, is underwhelming. Much additional research is needed. Analyses of recent MarkeTrak VIII retrospective data hint, however, that verification may be an important part of a comprehensive protocol (Hear Rev 20120;17[4]:12; 2011;18[6]:10), but well-designed prospective randomized control trials are sorely needed to provide definitive evidence in support of this process. Audiology is not unlike other health care fields in which strong evidence for aspects of clinical practice has been found to be lacking. In the absence of an adequate evidence base, a field often turns to the weakest form of evidence, “expert opinion,” as the only available form of evidence. This appears to be the case for the REM-verification component of the recommended hearing-aid fitting protocol. VALIDATION The expert opinion on validation in the AAA guidelines notes that “as audiologists continue to compete in the health care marketplace, they must demonstrate that treatments reduce activity limitations, decrease participation restrictions, and improve health-related quality of life. Only by measuring the outcomes can audiologists be assured that treatments make a difference and patients have benefited from their care.” (AAA, 2006; Page 43.) Validation is not considered an optional process but a required one, and the results of that process are as or more important than the process of validation itself. Fundamental to the process of validation, however, are the central questions of what aspects of outcome should be measured, how they should be measured, and when they should be measured. The most recent professional guidelines from AAA make no specific recommendations for number or type of outcome measures to obtain or how or when they should be measured. These choices are left to the discretion of the audiologist. Perhaps, in the absence of specific recommendations, audiologists are less able to implement an appropriate validation process. A complete review of these issues is beyond the scope of this article, but research summaries from the past dozen years or so are available on hearing aid outcome measures conducted at Indiana University under my direction. (Seminars in Hearing 2009;30[2]:112; http://bit.ly/Humes2009; Hearing Care for Adults 2009-The Challenge of Aging, Stafa, Switzerland: Phonak AG; 2009; Evidence Based Practice in Audiology. San Diego: Plural Publishing; in press.) These studies sought to determine what should be measured and when it should be measured, when it comes to hearing aid outcome measures, and to identify factors contributing to individual differences in outcome among wearers. The target population in all studies was adults aged 60-79 with bilateral mild-to-moderate high-frequency sensorineural hearing loss typical for their age, the group that most frequently purchases hearing aids in the United States. A detailed comprehensive fitting and evaluation protocol consistent with the ASHA and AAA best-practices standards of care was followed and the technology held constant within each study. The latter constraint was imposed because the focus was on individual differences in outcome. These Indiana University studies were most interested in what about the individual wearer determined successful outcomes rather than which various technologies affected outcome. After the completion of several such studies of hearing aid outcome, it became possible to compare group data across studies or technologies retrospectively. This was possible in part from the use of a common core battery of hearing aid outcome measures in each study and the use of reasonably large samples in many of the studies. My colleagues and I published a comparison of outcome measures for several hearing aid technologies, ranging from low-cost analog single-channel linear Class-D devices with output-limiting compression in the earliest study to higher-end digital six-channel WDRC open-fit devices with directional microphones. (Seminars in Hearing 2009;30[2]:112; http://bit.ly/Humes2009; Hearing Care for Adults 2009-The Challenge of Aging, Stafa, Switzerland: Phonak AG; 2009.) The groups fitted with each of the technologies in these studies were typically comprised of at least 50 individuals with similar mild-to-moderate hearing losses, ages, genders, and proportions of new hearing aid wearers. All wearers were fit bilaterally and outcome measures were obtained at four to six weeks post-fit. Very few group differences in hearing aid outcome were observed for performance comparisons across technologies, and no differences were observed on self-reported measures of outcome, including measures of hearing aid benefit, satisfaction, and usage. A similar lack of differences associated with technology was recently reported for measures of self-reported benefit obtained from large groups of hearing aid wearers fitted with either linear or wide-dynamic-range-compression technologies. (Ear Hear 2010;31[1]:47.) Given the lack of differences across most outcome measures and technologies, it was possible to pool the results across the Indiana University studies to form a large dataset of outcome measures from 368 older hearing aid wearers with mild-to-moderate hearing loss. Eleven outcome measures were included in this analysis: four scale scores from the HAPI (speech in quiet, speech in noise, speech with reduced cues, and miscellaneous; J Speech Hear Res 1984;27[1]49), three scale scores from the GHABP (helpfulness, satisfaction, and usage; J Am Acad Audiol 1999;10[2]:80), one global satisfaction score, dispenser-related items excluded, from the MarkeTrak V survey (Hearing Journal 2000;53[1]:38), daily usage calculated from the hearing aid diaries maintained by the wearers, and two measures of speech-recognition performance for the CST (aided performance and benefit, or aided-minus-unaided performance; Ear Hear 1987;8[5 Suppl]:119S). Factor analysis was used to reduce the redundancy in this set of outcome measures. (Factor Analysis, 2nd Ed. Mahwah, NJ: Lawrence Erlbaum Associates; 1983; Trends Amplif 2003;7[2]:41.) Given the high subjects-to-variables ratio (368:11), an excellent fit was obtained with three oblique (correlated) principal components accounting for about 75 percent of the variance and communalities all above 0.7, except for one outcome measure (CST benefit, communality=0.6; Factor Analysis, 2nd Ed. Mahwah, NJ: Lawrence Erlbaum Associates; 1983; Trends Amplif 2003;7[2]:41). The component weights resulted in easily interpreted factors. The three dimensions or components of hearing aid outcome identified via the factor-analysis solution, as well as the association of each of the 11 outcome measures with each of these outcome dimensions, are illustrated by the Venn diagram. (Figure 1.) All of the “helpfulness” and satisfaction ratings were weighted heavily on the first factor, and this has been interpreted as “benefaction,” a combination of self-reported benefit and satisfaction.Figure 1: Hearing Aid Outcome.The second factor was interpreted as hearing aid usage because both measures of usage were weighted most heavily on this factor. The third factor was interpreted as aided speech recognition because the aided CST score was the most heavily weighted outcome variable for this factor. Note that objective benefit (CST_ben) — the difference between the aided and unaided CST scores — lies in the region between the usage and aided-speech-recognition factors. This outcome measure was weighted about equally on these two factors (and as a result, has a lower communality than the other 10 outcome measures, which means that this outcome measure may not be well represented by this three-factor solution). The relative sizes of the circles in the Venn diagram represent the proportion of variance represented by each factor, and the amount of overlap among the circles represents the correlations among these factors. Benefaction is the largest contributor to “outcomes,” and is correlated (r=0.47) with usage, the next largest contributor. Aided speech-recognition contributes least to hearing aid outcomes, and is not appreciably correlated (r<0.12) with either of the other two outcome dimensions. These factor analyses of hearing aid outcome measures from a battery of 11 such measures obtained from 368 older adults wearing hearing aids and a variety of technologies suggest that only three to four outcome measures are needed to obtain a well-defined picture of hearing aid outcome. These measures include one measure of benefaction, one measure of hearing aid usage, and one measure of aided speech recognition. Because the latter may be difficult to interpret as a single percent-correct score, however, it would be best to obtain unaided speech-recognition scores as well and to look for significant improvements. Norms compiled from 333 of the 368 hearing aid wearers, prior to the completion of the most recent study of 35 older adults wearing open-fit devices have been published elsewhere for many of the outcome measures used in these Indiana studies. (Seminars in Hearing 2009;30[2]:112; http://bit.ly/Humes2009; Hearing Care for Adults 2009-The Challenge of Aging, Stafa, Switzerland: Phonak AG; 2009.) These norms make it possible to measure outcome and counsel the wearer on how he is performing compared with “typical” older adults wearing two well-fit hearing aids, in which “well fit” means good matches to prescribed insertion-gain targets using REM and behavioral evaluation of maximum output settings. Very favorable outcomes have been observed for all the technologies evaluated in the Indiana studies, with median ratings of the hearing aid being “helpful” for speech communication, median ratings of being “satisfied” with the hearing aids and their function, and median usage of slightly more than eight hours per day. (Seminars in Hearing 2009;30[2]:112; http://bit.ly/Humes2009.) One could argue that these positive outcomes are due to the universal use of the best-practices hearing aid fitting and evaluation procedures, including REM verification, counseling, and validation, in the Indiana University studies. Use of such practices does not guarantee positive outcomes for all wearers, however. This is illustrated in the bar graphs of outcome measures for two self-reported benefit measures (HAPI for speech in noise and speech in quiet) and global MarkeTrak satisfaction ratings, excluding dispenser-related items. (Figure 2.) Even though the “typical” or “average” outcomes for these older adults are very good, the outcomes are not universally good for all hearing aid wearers. Roughly one-third of the 368 hearing aid wearers from these studies reported being less than “satisfied” with their hearing aids and reported that their hearing aids were less than “helpful” in noise.Figure 2: Outcome Measures.When the factors that might explain these individual differences were explored in the most recent Indiana University study, findings very similar to those observed by Cox et al were obtained. (Ear Hear 2007;28[2]:141.) The mismatch between target gain and observed gain, after fine-tuning, was a significant predictor of outcome, but only for the outcome dimensions involving aided speech understanding, and even then, this measure accounted for less than 10 percent of the individual differences in outcome. A direct linkage between the quality of the fit in matching to prescriptive targets as verified by REM and the measured outcomes remains elusive. Perhaps, however, the quality of the fits in the Indiana studies and the 2007 Cox study were all very good, yielding positive average outcomes and minimizing the relative contributions of this factor to the measured outcomes. Further research is needed to examine this issue. FastLinks • Read ASHA's Guidelines for Hearing Aid Fitting for Adults at http://bit.ly/ASHAguidelines. • The American Academy of Audiology Guidelines for the Audiologic Management of Adult Hearing Impairment can be accessed at http://bit.ly/zB0B7h. • Read the Hearing Journal's MarkeTrak report on consumer satisfaction at http://bit.ly/wThHBk. • Click and Connect! Access the links in The Hearing Journal by reading this issue on thehearingjournal.com. • Comments about this article? Write to HJ at [email protected]. • Follow us on Twitter at twitter.com/hearingjournal.
Referência(s)