Portal de Periódicos da CAPES

An Interview With the Food and Drug Administration About Draft Patient-Focused Drug Development Guidance 3: Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments

2023; Elsevier BV; Volume: 26; Issue: 6 Linguagem: Inglês

10.1016/j.jval.2023.04.006

1524-4733

Elisabeth Oehrlein, R. L. Bent, Fraser D. Bocell, Lili Garrard, Laura Lee Johnson, Naomi B. Knoble, David S. Reasner, Michelle E. Tarver, Kevin P. Weinfurt, Theresa Mullin,

Pharmaceutical studies and practices

Editor's NoteThe following interview provides Food and Drug Administration's (FDA) responses to Value in Health's clarification request on Draft Patient-Focused Drug Development (PFDD) Guidance 3: Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments. The questions are based on ISPOR's comments on the draft guidance and other topics that have emerged during open meetings and workshops. The interview was conducted by Elisabeth Oehrlein, PhD, MS, Associate Editor at Value in Health. The interview participants were: Robyn Bent, RN, MS; Fraser Bocell, PhD; Lili Garrard, PhD; Laura Lee Johnson, PhD; Naomi Knoble, PhD; David Reasner, PhD; Michelle Tarver, MD, PhD; US Food and Drug Administration (FDA), Silver Spring, MD, USA; and Kevin Weinfurt, PhD, FDA Advisor, Duke University Medical Center, and Duke Clinical Research Institute. In general, Value in Health does not accept unsolicited requests for interview articles. We will consider future articles in this series if we believe our readers can benefit from clarification of high-priority government documents. The following interview provides Food and Drug Administration's (FDA) responses to Value in Health's clarification request on Draft Patient-Focused Drug Development (PFDD) Guidance 3: Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments. The questions are based on ISPOR's comments on the draft guidance and other topics that have emerged during open meetings and workshops. The interview was conducted by Elisabeth Oehrlein, PhD, MS, Associate Editor at Value in Health. The interview participants were: Robyn Bent, RN, MS; Fraser Bocell, PhD; Lili Garrard, PhD; Laura Lee Johnson, PhD; Naomi Knoble, PhD; David Reasner, PhD; Michelle Tarver, MD, PhD; US Food and Drug Administration (FDA), Silver Spring, MD, USA; and Kevin Weinfurt, PhD, FDA Advisor, Duke University Medical Center, and Duke Clinical Research Institute. In general, Value in Health does not accept unsolicited requests for interview articles. We will consider future articles in this series if we believe our readers can benefit from clarification of high-priority government documents. The interview below captures responses from FDA staff to questions regarding the development and application of the PFDD Guidance 3.Question 1.PFDD Guidance 3 builds upon the 2009 patient-reported outcome (PRO) guidance. Can you provide a brief history of the guidance series and the major changes since the 2009 guidance?Since the guidance "Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims"1Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims.http://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-reported-outcome-measures-use-medical-product-development-support-labeling-claimsDate accessed: February 27, 2023Google Scholar was published in 2009, a lot has happened that has informed the content of the recent "Patient-Focused Drug Development: Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments Draft Guidance"2Patient-focused drug development: selecting, developing, or modifying fit-for-purpose clinical outcome assessments.https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-selecting-developing-or-modifying-fit-purpose-clinical-outcomeDate accessed: February 27, 2023Google Scholar (PFDD Guidance 3) and the PFDD methodologic guidance series3Patient-focused drug development guidance series for enhancing the incorporation of the Patient's voice in medical product development and regulatory decision making.https://www.fda.gov/drugs/development-approval-process-drugs/fda-patient-focused-drug-development-guidance-series-enhancing-incorporation-patients-voice-medicalDate accessed: February 27, 2023Google Scholar as a whole.We recognized that guidance was needed not just for PRO measures but for all types of measures meant to reflect how patients feel, function, or survive. This required thinking about a framework that was relevant across a broad range of measures.Additionally, significant methodologic advances in the field of health measurement have been published in the years since the 2009 guidance, and the new guidance expounds on flexible options for clinical outcomes assessment (COA) development.Industry and patient groups have highlighted the need for a principled and pragmatic guidance that offers flexibility.The past decade of experience with COAs has highlighted additional areas where guidance would be helpful.Finally, there have been evolutions in how other fields think about validity and the use of measures for high-stakes decisions such as those we face in a regulatory setting.All these experiences and developments informed PFDD Guidance 3 and the series as a whole. Once all 4 of the PFDD guidances are final, they will replace the 2009 PRO guidance. If you wonder why something is not in PFDD Guidance 3, it can likely be found in one of the other guidance documents in the PFDD guidance series. We focused PFDD Guidance 1 on collecting data, and Guidance 2 is about looking at those data and how to identify and understand what is important to patients. Guidance 3 takes the next step, covering approaches using that information for selecting, modifying, developing, and gathering validity evidence that the COA measures what is important to patients. Guidance 4 will start by assuming you have a fit-for-purpose measure and now wish to use the COA to construct an endpoint for use in clinical research. Guidance 4 will cover COA-based endpoints and determining clinically meaningful differences.Back to Guidance 3. Parts of PFDD Guidance 3 will look familiar. The roadmap to a fit-for-purpose COA was expanded to stress the need to search for existing measures that might be used with or without modification. A revised conceptual framework distinguishes, as the Scientific Advisory Committee of the Medical Outcomes Trust did in 2002, between a conceptual model of the concept of interest and a measurement model that represents how the proposed COA is intended to work. Other ideas in the draft guidance may be less familiar to some readers—namely, the replacement of different types of validity with types of validity evidence and the presentation of that evidence in an explicit rationale to support whether the COA is fit-for-purpose. Both ideas arose from advances in validity theory. The original validity framework for health measurement was taken from educational and psychological measurement traditions, specifically those reflected in the 1974 Standards for Educational and Psychological Tests. In response to many of the same practical challenges faced by health measurement (eg, trying to decide the type and amount of validity evidence needed), subsequent editions of those standards were modified, with the most recent being published in 2014. PFDD Guidance 3 incorporates insights from modern validity theory to help navigate the challenges of COAs.PFDD Guidance 3 was also informed by an important idea from the work of ISPOR's Emerging Good Practices for Outcomes Research Task Force—specifically, how the relationship between the measured concept of interest and the meaningful aspect of patients' health may be more or less direct. This issue becomes particularly salient as we consider guidance for types of COAs beyond PRO measures.Question 2.How did patient and other stakeholder (eg, medical product developer, clinician, instrument developer) comments and questions received on the Guidance 3 discussion document guide the FDA in updating and providing additional clarity in this draft guidance?Great question. Before issuing each of the guidance documents within the PFDD Methodologic Guidance Series, FDA held a public meeting and provided a discussion document for stakeholders to react to. In addition to the public meeting, we also opened a docket to receive comments. What we heard from these meetings and dockets has informed the guidance documents. As we mentioned, in the case of PFDD Guidance 3, we heard that people wanted a guidance that did not rely on a one-size-fits-all approach. We heard that people wanted more of an insight into FDA's thinking and not just a checklist. Because many elements of classical test theory were included in the 2009 PRO guidance, many users felt FDA was not open to other methods and asked specifically for the inclusion of item response theory and other types of modeling and methods. In other venues, including Center for Devices and Radiological Health (CDRH) workshops over the years, groups alluded to opportunities and challenges to implementing the 2009 guidance recommendations4Food and Drug Administration/American Academy of Ophthalmology workshop on developing novel endpoints for premium intraocular lenses; public workshop.https://www.federalregister.gov/documents/2013/09/06/2013-21711/food-and-drug-administrationamerican-academy-of-ophthalmology-workshop-on-developing-novel-endpointsDate accessed: February 27, 2023Google Scholar and we kept these comments in mind when drafting PFDD Guidance 3. We also received feedback from FDA staff and decision makers across the centers about questions they frequently hear during sponsor meetings and issues they frequently observe when reviewing applications—so we tried to incorporate those experiences into the guidance as well. Patients and patient advocacy groups participated in the public workshop in 2018 and contributed to the docket for that meeting. We used their feedback as we created the draft, including more details on how FDA will assess whether a COA is fit-for-purpose, talking about special considerations for pediatric disease and more. Finally, as we refined the scope of each guidance, we leveraged the feedback that we received to move topics between guidances to places where the topics fit more naturally. For example, we moved discussion of meaningful differences, which in regulatory settings is typically related to an endpoint, to Guidance 4.Question 3.As the science of COAs continues to evolve, what kind of flexibility can we anticipate, especially for programs that are already underway? Could you provide an example?For programs already underway, it is not the intent of PFDD Guidance 3 to disrupt the development pipeline. Rather, as a draft guidance, PFDD Guidance 3 presents an opportunity for all of us in the clinical research community to learn from one another about how best to develop measurement strategies that are both principled and pragmatic. We hope that organizations try to test drive the rationale-based approach and share with us what they learn. The framework outlined in PFDD Guidance 3 is intended to promote greater flexibility and efficiency. Flexibility and efficiency are principles that have been reinforced not only in the PFDD Guidance 3 but also in FDA public meetings, reports, and publications. As was always the case, sponsors are welcome to explore and propose approaches believed to have merit.Additionally, a goal of the guidance was to present a framework that was general enough that it could still accommodate advances in COAs. For example, a performance outcome measure might be developed with novel technology, but the process of describing and defending the rationale for its use would still be the same. The specific components of the rationale would be tailored to reflect how that performance outcome measure is supposed to work, its chief assumptions, and potential threats. Hence, the basic framework would still apply.Question 4.What are common misconceptions about COAs? About the PFDD Guidance 3?One common misconception about COAs that we encounter is that sponsors can wait until a phase III trial to introduce a COA into the development program. FDA encourages sponsors to engage with FDA as early as possible to discuss a COA strategy and try out candidate measures in earlier phase trials. Another misconception about COAs that FDA encounters is the belief that a new COA must be developed for every situation. As PFDD Guidance 3 makes clear, sponsors should first consider whether an existing measure, with or without modification, could be used. This helps to conserve critical research resources, including patients' time and effort participating in COA development studies.With respect to misconceptions about PFDD Guidance 3, we have heard that some believe that PFDD Guidance 3 raises the bar for validation. The intent of the guidance is not to increase or decrease the level of evidence required for a COA to be considered fit-for-purpose. Rather, PFDD Guidance 3 is intended to help sponsors to frame the evidence in terms of the rationale for the use of a measure. It is hoped that this will facilitate communication about any validation concerns and will support efficient allocation of resources toward collecting evidence where it is needed most. Relatedly, the guidance is not intended to disrupt established measurements used in clinical care, but if those measurements do not appear to be working well in a research context and you are contemplating developing another measure, this is an opportunity to consider another approach.Another response we have heard is that content validity is no longer important in this guidance. The idea that the content of the measure should capture the concept of interest is still absolutely critical for determining whether a measure is fit-for-purpose. In fact, it is one of the first components of the generic rationale presented in the guidance, but, following modern validity theory, this idea is no longer called "content validity." Instead, we state the key assumption—that the selected COA measure captures all the important aspects of the concept of interest—and then look for evidence supporting this. To support this part of the rationale, it is necessary to have a well-defined conceptual model of the concept of interest well supported by data from patients and/or caregivers. Hence, data about the patients' experiences are still vital in helping us to understand the conceptual model of the concept of interest. Collecting this information is the focus of PFDD Guidances 1 and 2 of the series, which cover qualitative, quantitative, and mixed methods to identify what is important to patients and how to collect comprehensive and representative input. It is that information sponsors then use as evidence to describe how exactly the different aspects of the concept of interest are reflected in the measure. Question 5.PFDD Guidance 3 introduces the term "evidence-based rationale." Could you please explain this concept? How does it overlap with "fit for purpose"?One way to think about the rationale is to ask, "What needs to be true about this measure for the proposed use to be reasonable?" or "How is this measure supposed to work?" For example, one important part of the rationale for a PRO measure is that patients understand the instructions and items as intended by the measure developer. If that were not true, the measure does not work as intended and so we would have serious concerns about using the scores as the basis for an endpoint. Evidence for that part of the rationale could come from cognitive interviews with patients in the target population. Because different types of measures might be designed differently, the rationale may need to be tailored to each measure based on how each measure is supposed to work.When the rationale for a proposed use of COA scores is comprehensive and well supported by evidence, then the COA can be considered fit-for-purpose. That is, the level of validation (reflected by the comprehensive and well supported rationale) is sufficient to support the proposed use of the COA. Often, the proposed use is in a specific regulatory context and whether the measure is fit-for-purpose is a regulatory conclusion specific to an individual medical product development program or to a broader context of use evidenced in a Drug or Device Development Tool qualification.5Drug development tool (DDT) qualification programs.https://www.fda.gov/drugs/development-approval-process-drugs/drug-development-tool-ddt-qualification-programsDate accessed: February 27, 2023Google Scholar,6Qualification of medical device development tools: guidance for industry, tool developers, and Food and Drug Administration staff.https://www.fda.gov/regulatory-information/search-fda-guidance-documents/qualification-medical-device-development-toolsDate accessed: February 27, 2023Google ScholarIt is important to note that the rationale-based framework is intended to be both principled and pragmatic. It is principled because there needs to be a comprehensive rationale for a proposed measure, but it is pragmatic because the degree of evidence needed to support any part of the rationale can depend upon 2 things: (1) how confident we already feel about each part of the rationale and (2) the costs and burdens that would be involved in generating more evidence than we already have. Hence, we need to understand how the measure is supposed to work and potential threats to its use (the rationale) but be judicious in the type and amount of evidence required to support each part of the rationale depending upon the situation.Question 6.The draft guidance refers to 8 components comprising an evidence-based rationale for proposing a COA as fit-for-purpose. Could you walk us through examples relevant to drug development? An example illustrating a PRO and a performance outcome measure would be very helpful.An example of using a rationale to inform measure development was presented by R.J. Wirth in the FDA's public webinar on PFDD Guidance 3 (https://www.fda.gov/drugs/news-events-human-drugs/public-webinar-patient-focused-drug-development-selecting-developing-or-modifying-fit-purpose) (Table 17Wirth R.J. Presenting a case for validity: two examples from the MiCOAS project. Patient-focused drug development: selecting, developing, or modifying fit-for-purpose clinical outcome assessments – draft guidance.https://www.fda.gov/media/161838/downloadDate accessed: February 27, 2023Google Scholar).Table 1Validity table example: Pain Intensity Scale from Wirth R.J.7Wirth R.J. Presenting a case for validity: two examples from the MiCOAS project. Patient-focused drug development: selecting, developing, or modifying fit-for-purpose clinical outcome assessments – draft guidance.https://www.fda.gov/media/161838/downloadDate accessed: February 27, 2023Google Scholar presented at FDA's Public Webinar Patient-Focused Drug Development: Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments Draft Guidance.ComponentJustificationHeadache pain intensity should be assessed by a PRO measure.A direct report of headache pain intensity is best reported by individual experiencing the headache pain (observer-reported outcomes may be possible).The PIS captures all the important aspects of headache pain intensity.Head pain intensity is a narrow concept that is sufficiently covered by the language of the single question asking about pain intensity. In everyday English, the response options (none, mild, moderate, and severe) cover the entire range of pain intensity. Ongoing qualitative research is being conducted to further support this component.Patients understand the instructions and item of the PIS as intended by the measure developer.Wording and response options are very common and have been used in multiple trials successfully. Ongoing qualitative research is being conducted in support of the instructions, item, and response options.Scores of the PIS are not overly influenced by processes/concepts that are not part of headache pain intensity.Qualitative research suggests that people experiencing headache pain can respond to questions about headache pain intensity without being meaningfully influenced by other factors.Scores of the PIS are not overly influenced by other symptoms of migraine.Qualitative research suggests that people experiencing headache pain can respond to questions about headache pain intensity without being meaningfully influenced by other symptoms of migraine.The method of scoring responses to the PIS is appropriate for assessing headache pain intensity.The score is the ordinal numeric code assigned to each answer (ie, none = 0, mild = 1, moderate = 2, severe = 3).Scores from the PIS correspond to the specific health experiences the patient has related to headache pain intensity.Previous research has repeatedly found self-report of pain to accurately reflect an individual's experience.Scores are sufficiently sensitive to reflect changes in the PIS within patients over time.It has been demonstrated empirically in numerous clinical trials that the PIS is sensitive to change and treatment effects.Differences in COA scores can be interpreted and communicated clearly in terms of the expected impact on people's day-to-day lives.Qualitative research suggests people understand movement between none, mild, moderate, and severe and that this movement is relevant to their treatment goals.COA indicates clinical outcomes assessment; FDA, Food and Drug Administration; PIS, pain intensity scale; PRO, patient-reported outcome. Open table in a new tab An example of the same basic approach applied to an existing performance outcome measure was published in the following:Weinfurt KP. Constructing and evaluating a validity argument for a performance outcome measure for clinical trials: an example using the multi-luminance mobility test. Clinical Trials, 2022;19(2):184-193.Question 7.Could you please expand on the FDA's current thinking about how researchers can address interfering influences when it is not possible to completely remove an influence? For example, double blinding for rare disease studies or transitioning from a paper-based COA to electronic COA is preferred because it increases accessibility to patients.PFDD Guidance 3 acknowledges that there is no such thing as a perfect measure or pure scores. There will always be multiple factors in addition to the concept of interest that influence scores on a COA—what is sometimes called construct contamination. The key question is whether any of these influences have a large enough impact on the scores that the scores are no longer useful for their intended purpose. On a case-by-case basis, sponsors are encouraged to consider what the biggest threats might be to the interpretation and use of the COA scores as the basis for an endpoint and then to collect evidence to evaluate how strongly the influence interferes with the scores, and as you observed, there are sometimes ways to reduce the impact of interfering influences through randomization, masking, and other study design decisions. COA indicates clinical outcomes assessment; FDA, Food and Drug Administration; PIS, pain intensity scale; PRO, patient-reported outcome. Question 8.This guidance is a joint effort among Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, and CDRH. What is FDA doing internally to ensure harmonization across centers?FDA has existing infrastructure to facilitate harmonization of implementation across centers, including joint working groups, shared participation in COA qualification teams, and partnering on regulatory science efforts, and as with all tricenter guidances, PFDD Guidance 3 is written so that it can be implemented similarly across centers, except where there are regulatory statutes that would necessitate center-specific implementation.Question 9.CDRH applies a "least burdensome" approach, which means "the minimum amount of information necessary to adequately address a relevant regulatory question or issue through the most efficient manner at the right time." How does CDRH's least burdensome approach apply in the context of PFDD Guidance 3?As part of CDRH's commitment to incorporate patients' experiences into the work we do, the PFDD Guidance 3 helps to facilitate the appropriate use of COA in medical product clinical studies. The least burdensome definition considers several factors, including the type of information, the different approaches to generating or providing information, and when during the total product life cycle information should be generated or provided to the FDA. In line with these considerations, it is important to note that the use of a COA is generally voluntary but may be specifically recommended in certain standards and guidance documents. In January 2022, CDRH finalized the guidance, "Principles for Selecting, Developing, Modifying and Adapting Patient-Reported Outcome Instruments for Use in Medical Device Evaluation,"8Principles for selecting, developing, modifying, and adapting patient-reported outcome instruments for use in medical device evaluation: guidance for industry and Food and Drug Administration staff, and other stakeholders.https://www.fda.gov/regulatory-information/search-fda-guidance-documents/principles-selecting-developing-modifying-and-adapting-patient-reported-outcome-instruments-useDate accessed: February 27, 2023Google Scholar which provides recommendations about the importance of ensuring the PRO instruments are fit-for-purpose and outlines best practices to help ensure relevant, reliable, and sufficiently robust PRO instruments are developed, modified, or adapted using the least burdensome approach. The 2022 Principles Guidance will still contain important considerations and will not be replaced by the PFDD guidance series. Just as the Principles Guidance uses the term fit-for-purpose to describe a flexible approach for PROs, PFDD Guidance 3 expands this flexible approach to all COAs. Importantly, PFDD Guidance 3 provides a common approach and language to facilitate discussions that will help ensure the use of a COA is fit-for-purpose for a given context of use in a clinical study. The 2022 Principles Guidance details key principles and factors to consider when selecting, developing, modifying, or adapting a PRO instrument that is fit-for-purpose. We note the PFDD Guidance 3 is still a draft and not currently for implementation purposes, and FDA appreciates and will consider all comments to the docket as we collectively work to finalize the guidance. Author Contributions: Concept and design: Oehrlein Acquisition of data: Oehrlein Administrative, technical, or logistic support: Oehrlein Conflict of Interest Disclosures: Dr Oehrlein is employed by Applied Patient Experience, LLC, which has contracts with a variety of nonprofit organizations, pharmaceutical companies, and academic institutions. Dr Oehrlein is an editor for Value in Health and had no role in the peer-review process of this article. Funding/Support: The authors received no financial support for this research. Contributors: Laura Lee Johnson, PhD; Lili Garrard, PhD; Michelle Tarver, MD, PhD, Fraser Bocell, PhD; Kevin Weinfurt, PhD; David Reasner, PhD; Naomi Knoble, PhD; Robyn Bent, RN, MS; Theresa Mullin, PhD, US Food and Drug Administration, Silver Spring, MD, USA.