Accuracy and Reliability of Internet Resources Providing Information on Obstructive Sleep Apnea
2018; American Academy of Sleep Medicine; Volume: 14; Issue: 10 Linguagem: Inglês
10.5664/jcsm.7380
ISSN1550-9397
AutoresJong Min Lee, Woo Ho Ban, Hyun‐Kyung Park, Eunbyeol Na, Sug Kyung Kim, Hyeon Hui Kang, Sang Haak Lee,
Tópico(s)Mobile Health and mHealth Applications
ResumoFree AccessObstructive Sleep ApneaAccuracy and Reliability of Internet Resources Providing Information on Obstructive Sleep Apnea Jongmin Lee, MD*, Woo Ho Ban, MD*, Hyun Kyung Park, MS, Eunbyeol Na, BS, Sug Kyung Kim, MS, Hyeon Hui Kang, MD, Sang Haak Lee, MD, PhD Jongmin Lee, MD* Division of Pulmonary, Allergy and Critical Care Medicine, Department of Internal Medicine, Seoul St Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea , Woo Ho Ban, MD* Division of Pulmonary, Allergy and Critical Care Medicine, Department of Internal Medicine, Incheon St Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, Republic of Korea , Hyun Kyung Park, MS CC&I Research, Seoul, Republic of Korea , Eunbyeol Na, BS CC&I Research, Seoul, Republic of Korea , Sug Kyung Kim, MS CC&I Research, Seoul, Republic of Korea , Hyeon Hui Kang, MD Division of Pulmonary, Critical Care and Sleep Medicine, Department of Internal Medicine, St Paul's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea , Sang Haak Lee, MD, PhD Address correspondence to: Sang Haak Lee, Division of Pulmonary, Critical Care and Sleep Medicine, Department of Internal Medicine, St Paul's Hospital, College of Medicine, The Catholic University of Korea, 180 Wangsan-ro, Dongdaemungu, Seoul, Republic of Korea+82 2 961 4500+82 2 968 7250 E-mail Address: [email protected] Division of Pulmonary, Critical Care and Sleep Medicine, Department of Internal Medicine, St Paul's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea Published Online:October 15, 2018https://doi.org/10.5664/jcsm.7380Cited by:2SectionsAbstractPDFSupplemental Material ShareShare onFacebookTwitterLinkedInRedditEmail ToolsAdd to favoritesDownload CitationsTrack Citations AboutABSTRACTStudy Objectives:To evaluate the readability, content, and quality of internet information about obstructive sleep apnea (OSA).Methods:Three major search engines (Google, Yahoo, and Bing) were used to retrieve the first 200 hits for the term "sleep apnea." The website contents were assessed using established guidelines, while the website quality was evaluated using the validated DISCERN instrument and Journal of the American Medical Association benchmarks.Results:In total, 243 websites met the inclusion criteria for this study. The websites were divided into five categories: scientific resource, foundation, news/ media report, commercial website, and personal commentary. The mean Flesch-Kincaid grade level, which indicated the readability of the websites, was 7.5. The quality of most websites was poor, with news/media reports and commercial websites exhibiting poorer quality than the others. The Health On the Net (HON) code, which certifies the reliability and credibility of online content regarding human health, was applied only to 14.3% websites; the reliability of all these websites was poor. The content score for the diagnosis and management of OSA was lower for news/media reports and commercial websites than for scientific resources. News/media reports exhibited the worst content scores.Conclusions:Our findings suggest that most online health resources regarding OSA are not reliable and are inaccurate. Health care providers should be aware of the high variability in online information concerning OSA.Commentary:A commentary on this article appears in this issue on page 1643.Citation:Lee J, Ban WH, Park HK, Na E, Kim SK, Kang HH, Lee SH. Accuracy and reliability of internet resources providing information on obstructive sleep apnea. J Clin Sleep Med. 2018;14(10):1717–1723.BRIEF SUMMARYCurrent Knowledge/Study Rationale: Patients generally use the internet as a resource for health information. However, no study has evaluated the content and quality of online information on obstructive sleep apnea (OSA).Study Impact: The results of this study demonstrate that websites related to OSA are often incomplete and inaccurate, and the low quality of online information about this condition is a significant barrier to patient education. This study highlights the need to ensure the accuracy and quality of online health information concerning OSA.INTRODUCTIONThe internet has become a common source for health-related information and medical queries.1 In 2014, 84% of the United States population used the internet, with 72% of that population searching for health information.2 In total, 35% adults in the United States attempt to obtain a diagnosis of a medical condition for themselves or others by seeking health information on the internet.2 The internet is an attractive resource for patients because it provides information on almost all health-related topics immediately. However, there are concerns about the accuracy and reliability of online health information, and there is a possibility that the patients may be misinformed.3,4Obstructive sleep apnea (OSA) is a common breathing disorder characterized by narrowing of the upper airway that impairs normal ventilation during sleep.5 Patients with untreated OSA may be at an increased risk of motor vehicle accidents, metabolic dysregulation, and cardiovascular diseases such as hypertension, coronary artery disease, congestive heart failure, arrhythmias, and stroke.6,7 Therefore, the recognition and treatment of OSA is important for improving the quality of life and lowering the mortality and morbidity of patients with OSA. Although clinical guidelines for the diagnosis and management of OSA were published by the American Academy of Sleep Medicine in 2009 and updated in 2017,5,8 there is a lot of misleading information about OSA on the internet, which can result in serious consequences for patients.Thus far, few studies have examined the quality of online health information about OSA. Therefore, the objective of the current study was to analyze the readability and quality of online health information concerning OSA.METHODSData SourcesWe used three well-known internet search engines for our analysis of online health information about OSA. Google, Yahoo, and Bing ( http://google.com, http://yahoo.com, http://bing.com) searches for "sleep apnea" were performed on September 1, 2016. After deleting the history and cookies of the web browser, searches were conducted using the United States version of each search engine.Website SelectionThe first 200 English language websites providing information on OSA in each search engine were collected. Websites that required registration or enrollment fees to access information, duplicate websites, websites that only include videos, and scientific journal articles for medical professionals were excluded.Evaluation of the Readability and Accuracy of WebsitesThe website contents were crawled using Python software 2.7.12 (Python Software Foundation, Beaverton, Oregon, United States). All websites were classified into five categories: scientific resource, foundation, news/media report, commercial website, and personal commentary. The readability of websites was evaluated using the Flesch Reading Ease Score (FRES) and the Flesch-Kincaid grade level,9,10 while the quality of information provided by each website was assessed using publishing benchmarks laid by the Journal of the American Medical Association (JAMA) and the DISCERN instrument. The JAMA benchmarks provide a means for the qualitative assessment of websites for the presence of authorship, references (attribution), date of last update (currency), and disclosure of conflicts of interest (disclosure).11 The DISCERN instrument is a validated tool designed to judge the quality of written information about treatment choices.12 The instrument comprises 16 questions rated from 1 (low quality/ not addressed) to 5 (high quality/fully addressed). Questions 1 through 8 evaluate the reliability, questions 9 through 15 evaluate the quality of information regarding treatment options, and question 16 evaluates the overall publication.12 In addition, each website was evaluated for the Health on the Net Foundation (HON) code. HON is a nonprofit organization that reviews health information websites for eight points of conduct: authoritative, complementarity, privacy, attribution, justifiability, transparency, financial disclosure, and advertising policy.13 The content score was created by two authors (J Lee and WH Ban) using a predefined scoring system based on clinical guidelines (Table 1). To avoid discrepancies in findings, the DISCERN and content scores were rechecked by two independent reviewers.Table 1 Criteria for scoring the content of websites providing information on obstructive sleep apnea.Table 1 Criteria for scoring the content of websites providing information on obstructive sleep apnea.Statistical AnalysisStatistically significant differences between groups were analyzed using chi-square tests, analysis of variance, Bonferroni (Dunn) t tests, Kruskal-Wallis tests, or Fisher exact tests as appropriate. Data are expressed as mean ± standard deviations or numbers and percentages. A two-tailed value of P < .05 was considered statistically significant. All statistical analyses were performed using SAS software, version 9.4 (SAS Institute Inc., Cary, North Carolina, United States), and a graph was plotted using the R 3.4.1 version (R Foundation, Vienna, Austria).RESULTSCharacteristics of WebsitesAfter the exclusion of duplicates and websites that did not meet the inclusion criteria, 243 websites were considered relevant for this study (Figure 1). Table 2 shows the characteristics of websites classified according to the search engine. Commercial websites were the majority in Yahoo and Google, whereas news/media reports were the majority in Bing. FRES and the Flesch-Kincaid grade level were 57.1 ± 13.7 and 7.5 ± 2.5, respectively. There were no significant differences according to the search engine. The mean DISCERN total score, mean JAMA benchmark score, and mean content score were 49 ± 14, 1.9 ± 1.3, and 10.1 ± 4.1, respectively (Table 3), with significantly higher values for Google websites than for the other websites (P = .0002, P < .0001, and P = .0486, respectively). Most websites did not have HON certification (85.19%).Table 2 Characteristics of websites providing information on obstructive sleep apnea.Table 2 Characteristics of websites providing information on obstructive sleep apnea.Table 3 Reliability and quality of the content of websites providing information on obstructive sleep apnea.Table 3 Reliability and quality of the content of websites providing information on obstructive sleep apnea.Figure 1: Search results and selection of websites providing information on obstructive sleep apnea.The first 200 English language websites providing information on obstructive sleep apnea in each search engine were collected. Websites that required registration or enrollment fees to access information, duplicate websites, websites that only include videos, and scientific journal articles for medical professionals were excluded. After the exclusion of duplicates and websites that did not meet the inclusion criteria, 243 websites were considered relevant for this study.Download FigureReliability and Quality of WebsitesThe mean DISCERN total score for scientific resources was significantly higher than that for news/media reports and commercial websites (P < .0001), whereas individual scores for reliability (questions 1–8) and treatment choices (questions 9–15) were significantly higher for scientific resources than for commercial websites (P < .0001). The overall quality (question 16) of scientific reports was higher than that of news/media reports, commercial websites, and personal commentaries (P < .0001). There was no significant difference between DISCERN scores for scientific resources and those for foundations. The mean JAMA benchmark score was significantly lower for commercial websites than for scientific reports, foundations, and news/media reports (P = .0016), with no significant difference between commercial websites and personal commentaries.Relationship Between the Presence of an HON Code and the Website QualityAs mentioned previously, approximately 14% of websites were HON accredited. Table 4 shows the relationship between the presence of an HON code and the website quality. There were no significant differences between the HON-positive and HON-negative websites in each website category. The DISCERN total score, JAMA benchmark score, and content score were significantly higher for HON-negative websites than for HON-positive websites (P = .0009, P < .0001, and P = .01, respectively).Table 4 Characteristics of websites providing information on obstructive sleep apnea according to the HON code.Table 4 Characteristics of websites providing information on obstructive sleep apnea according to the HON code.Content ScoresTable 5 shows the content scores for websites in each category. The mean total content score was the highest for scientific resources (12.3 ± 2.9) and the lowest for news/ media reports (7.2 ± 4.1; Figure 2), and it was significantly higher for scientific resources than for news/media reports and commercial websites (P < .0001). Scores for definitions and symptoms were significantly lower for news/media reports than for all other categories except personal commentary (P < .0001). News/media reports and commercial websites exhibited significantly lower scores for diagnosis and management than did scientific resources (P = .0005 and P = .0002, respectively). A list of the websites containing higher content scores (> 14) is provided in Table S1 in the supplemental material.Table 5 Content scores by the category of websites providing information on obstructive sleep apnea.Table 5 Content scores by the category of websites providing information on obstructive sleep apnea.Figure 2: Content scores according to the category of websites providing information on obstructive sleep apnea.The mean total content score was the highest for scientific resources and the lowest for news/media reports, and it was significantly higher for scientific resources than for news/media reports and commercial websites (P < .0001). ANOVA = analysis of variance.Download FigureWebsite Characteristics Based on the Search Results RankingTable S2 in the supplemental material shows the type, readability, and quality of websites according to their place in the search results, and whether they appeared earlier (group 1) or later (group 2). The type of website was significantly different between the two groups (P < .0001); scientific resources were the majority in group 1, whereas commercial websites were the majority in group 2. There was no significant difference between readability scores (FRES and the Flesch-Kincaid grade level) for group 1 and those for group 2. The mean total DISCERN and JAMA score of group 1 was significantly higher than that of group 2 (P = .001 for DISCERN, P = .018 for JAMA, respectively).DISCUSSIONIn the current study, we evaluated the readability, content, and quality of internet information about OSA and found that most online health resources regarding OSA are not reliable and are inaccurate.Of late, with the widespread provision of internet services and increased popularity of online medical information, the number of patients accessing and applying this information has increased.14 However, the reliability and quality of currently available medical information concerning OSA has not been evaluated. To the best of our knowledge, the current study is the first to investigate the readability and quality of health information about OSA on the internet.The current study included 243 websites providing health information about OSA. Commercial websites were the most common (37.0%), followed by scientific resources (22.2%). According to the JAMA benchmark and DISCERN total scores, the Yahoo websites were the most reliable, whereas the Bing websites were the least reliable. This could be attributed to differences in the proportion of scientific resources and foundations, which deliver relatively accurate and reliable information, between Yahoo and Bing.The FRES is based on a ranking scale of 0 to 100, and the higher score represents content that can be more easily understood. Scores between 60 and 80 should generally be comprehensible for 12 to 15 year olds. The F-K grade level refers to the American school grade needed to comprehend the material on the website. In this study, the overall means for the FRES score and F-K grade level of the websites were 57.1 and 7.5, respectively. The National Institutes of Health recommends that the reading grade level for written health material should be between the seventh and eighth grade levels for the average adult.15 Accordingly, the readability of websites providing information about OSA was satisfactory. In other words, the information was easy to comprehend by average consumers.We identified 14 content items corresponding to established OSA features and treatments. The content score was higher for scientific resources and foundations than for news/media reports, commercial websites, and personal commentaries. Medical consumers most frequently encounter commercial websites, which transmit medical information for a profit; therefore, these websites are very likely to provide industry-directed and inadequate information to medical consumers. In particular, we found that content scores for diagnosis and management, which are important for the readers to decide a treatment plan by themselves, were low for commercial web-sites. Several treatment options, including continuous positive airway pressure, oral appliances, lifestyle modifications, and surgery, should be considered for the successful management of OSA, and personalized and combination approaches should be evaluated according to the individual patient's characteristics.16 Therefore, websites should provide comprehensive medical information, including various treatment options and data that allow for the self-estimation of anatomical and economical characteristics. In addition, for the readers' convenience, we provide a list of the websites that contains higher content scores (> 14) in Table S1 in the supplemental material.In the current study, not only the quantity of content but also the reliability and quality of information were poorer in commercial websites than in scientific resources and foundations. In particular, DISCERN scores for questions 9 through 15, which are associated with the treatment choice, were significantly lower for commercial websites than for scientific resources. Therefore, medical consumers may receive unreliable information concerning the diagnosis and treatment of OSA through commonly exposed commercial websites, resulting in inefficient medical use.We found that only 14.8% of websites about OSA were certified by HON. HON is an independent organization that assesses whether websites provide understandable, accessible, and trustworthy information. Previous studies have established HON certification as a standard for the reliability of information.17 However, DISCERN, JAMA benchmark, and content scores for websites with HON certification were significantly lower than those for websites without HON certification in the current study. This unexpected finding suggests that it is very difficult for patients to obtain accurate and reliable information from the internet. Medical consumers should be aware that even certified websites may not contain accurate medical information.Although we used United States versions of each search engine in this study, there could be a concern that our results were influenced by our location (Seoul, Republic of Korea). To address that concern, we asked our colleague in Philadelphia, Pennsylvania to search "sleep apnea" in the three search engines in the same manner as this study. Although it was impossible to compare the result statistically, the concordance rate between Korea and the United States was relatively high (Google: 83.2%, Bing: 93.1%, Yahoo: 87.2%).As Table S2 in the supplemental material shows, the type, reliability, and quality of the websites were significantly different according to their ranking in the search results. It is well known that search engines report sites in an order based on the particular wording and other aspects of the websites. A search engine results page is generated based on a series of metrics that determines their relevance to the searched term.18 Webpages that score well on a search engine's algorithmic test are displayed higher in the list. These algorithms are generally based on factors such as the content of a webpage, the trustworthiness of the website, and other external factors.19 Furthermore, every major search engine accepts paid listings, and this guarantees that a paid site will appear in the top results for a related keyword. Therefore, medical consumers should be careful when they search health information online.This research has some limitations. First, potential selection bias may have played a role during website selection and observer bias may have occurred during the assessment of the websites for readability and reliability. Second, an appropriate method for assessing the quality of content pertaining to OSA was not used. Third, we limited our research to online documents. However, other information sources such as You-Tube, television, radio, and word of mouth are also significant. Therefore, future studies should focus on minimizing these biases and developing appropriate scoring systems for measuring the quality of information available through the internet. Moreover, additional studies need to be carried out in order to evaluate information from the other types of sources previously mentioned.Although the quality of internet resources is inconsistent, the internet is still one of the easiest ways for patients to procure information about OSA. Therefore, it is important to develop appropriate standards for certifying such websites, so that patients can obtain reliable health-related information from the internet. In addition, health care providers should attempt to explain the diagnosis, prognosis, risk factors, and treatment of OSA in a manner that meets the patient's expectation levels.CONCLUSIONSIn conclusion, our results suggest that most online health resources regarding OSA are not reliable and are inaccurate. Although medical information about OSA provided by websites is vast and readable, it is not very reliable, with most sites providing insufficient and biased medical information. As a result, patients may face difficulty in procuring appropriate and precise information from the internet, leading to adverse economic and individual health consequences. Therefore, web providers should give more comprehensive and sufficient information to medical consumers and establish a well-organized certification system to ensure the provision of good-quality and reliable medical information about OSA.DISCLOSURE STATEMENTWork for this study was performed at St Paul's Hospital, College of Medicine, The Catholic University of Korea. All authors have seen and approved this manuscript. The authors report no conflicts of interest.ABBREVIATIONSFRESFlesch Reading Ease ScoreHONHealth on the Net FoundationJAMAJournal of the American Medical AssociationOSAobstructive sleep apneaREFERENCES1 Carroll AE, Zimmerman FJ, Rivara FP, Ebel BE, Christakis DAPerceptions about computers and the internet in a pediatric clinic population. Ambul Pediatr; 2005;52:122-126, 15780015. CrossrefGoogle Scholar2 Susannah F, Maeve DHealth Online 2013. Accessed August 28, 2016 http://www.pewinternet.org/2013/01/15/health-online-2013. Published January 15, 2013. Google Scholar3 Fisher JH, O'Connor D, Flexman AM, Shapera S, Ryerson CJAccuracy and reliability of internet resources for information on idiopathic pulmonary fibrosis. Am J Respir Crit Care Med; 2016;1942:218-225, 26849779. CrossrefGoogle Scholar4 Grewal P, Williams B, Alagaratnam S, Neffendorf J, Soobrah RQuality of vascular surgery Web sites on the Internet. J Vasc Surg; 2012;565:1461-1467, 22801107. CrossrefGoogle Scholar5 Kapur VK, Auckley DH, Chowdhuri Set al.Clinical practice guideline for diagnostic testing for adult obstructive sleep apnea: an American Academy of Sleep Medicine clinical practice guideline. J Clin Sleep Med; 2017;133:479-504, 28162150. LinkGoogle Scholar6 Budhiraja R, Budhiraja P, Quan SFSleep-disordered breathing and cardiovascular disorders. Respir Care; 2010;5510:1322-1332discussion 1330-1322, 20875159. Google Scholar7 Tregear S, Reston J, Schoelles K, Phillips BObstructive sleep apnea and risk of motor vehicle crash: systematic review and meta-analysis. J Clin Sleep Med; 2009;56:573-581, 20465027. LinkGoogle Scholar8 Epstein LJ, Kristo D, Strollo PJet al.Clinical guideline for the evaluation, management and long-term care of obstructive sleep apnea in adults. J Clin Sleep Med; 2009;53:263-276, 19960649. LinkGoogle Scholar9 Flesch RA new readability yardstick. J Appl Psychol; 1948;323:221-233, 18867058. CrossrefGoogle Scholar10 Kincaid JP, Fishburne RP, Rogers RL, Chissom BSDerivation Of New Readability Formulas (Automated Readability Index, Fog Count And Flesch Reading Ease Formula) For Navy Enlisted Personnel. Millington, TN: Institute for Simulation and Training, University of Central Florida; 1975. Google Scholar11 Silberg WM, Lundberg GD, Musacchio RAAssessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor--Let the reader and viewer beware. JAMA; 1997;27715:1244-1245, 9103351. CrossrefGoogle Scholar12 Charnock D, Shepperd S, Needham G, Gann RDISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health; 1999;532:105-111, 10396471. CrossrefGoogle Scholar13 Barker S, Charlton NP, Holstege CPAccuracy of internet recommendations for prehospital care of venomous snake bites. Wilderness Environ Med; 2010;214:298-302, 21168781. CrossrefGoogle Scholar14 Hu X, Bell RA, Kravitz RL, Orrange SThe prepared patient: information seeking of online support group members before their medical appointments. J Health Commun; 2012;178:960-978, 22574697. CrossrefGoogle Scholar15 How to write easy to read health materials. Accessed August 28, 2016 https://medlineplus.gov/etr.html. Google Scholar16 Carberry JC, Amatoury J, Eckert DJPersonalized management approach for OSA. Chest; 2018;1533:744-755, 28629917. CrossrefGoogle Scholar17 Ball MJTwenty years of health on the net: committed to reliable information. Stud Health Technol Inform; 2016;225:738-740, 27332324. Google Scholar18 Jansen BJ, Spink AInvestigating customer click through behaviour with integrated sponsored and nonsponsored results. International Journal of Internet Marketing and Advertising; 2009;51-2:74-94. CrossrefGoogle Scholar19 Juon C, Greiling D, Buerkle CInternet Marketing Start to Finish: Drive Measurable, Repeatable Online Sales with Search Marketing, Usability, CRM, and AnalyticsIndianapolis, IN: Que Publishing; 2012. Google Scholar Previous article Next article FiguresReferencesRelatedDetailsCited by Nutrition guidance for cancer patients and survivors—A review of the websites of Irish healthcare and charitable organisations and cancer centresKeaver L, Callaghan H, Walsh L and Houlihan C European Journal of Cancer Care, 10.1111/ecc.13216, Vol. 29, No. 2, Online publication date: 1-Mar-2020. The Internet: Sleep On ItWorsnop C Journal of Clinical Sleep Medicine, Vol. 14, No. 10, (1643-1644), Online publication date: 15-Oct-2018. Volume 14 • Issue 10 • October 15, 2018ISSN (print): 1550-9389ISSN (online): 1550-9397Frequency: Monthly Metrics History Submitted for publicationJanuary 26, 2018Submitted in final revised formJune 18, 2018Accepted for publicationJune 29, 2018Published onlineOctober 15, 2018 Information© 2018 American Academy of Sleep MedicineKeywordsinterneteducationobstructive sleep apneaACKNOWLEDGMENTSThe authors thank JB Park at University of Pennsylvania, for supporting our study.PDF download
Referência(s)