The Limitations of Patient-Reported Outcome Measurement in Oncology

Generic patient-reported outcome measures (PROMs) for cancer have outlived their utility. More relevant and accurate PROMs are required in order to demonstrate the true value of oncology interventions to patients.

Patient-reported outcome measures (PROMs) cover a wide range of different outcomes including perceived clinical severity, disease burden, satisfaction with treatment, quality of life, and utility. Each of these outcomes requires a different type of PROM. Patient-reported outcomes (PROs) have been defined as “a report coming directly from patients about how they feel or function in relation to a health condition and its therapy without interpretation by healthcare professionals or anyone else.”1 Unfortunately, this definition is rather restrictive as it focuses on only one type of outcome—health-related quality of life (HRQL).

-----

Related Content

Ipilimumab Combination Significantly Improves Melanoma Outcomes

Outcomes of a Clinical Pathways Program

-----

It should be noted that PROs should not be confused with patient-centric outcomes. Whereas “patient-reported” indicates that the information is provided by the patient, “patient-centric” implies that the information collected is of specific concern to the patient. In fact, most PROMs do not collect patient-centric data. 

Too often, PROMs are selected and used without a clear justification,2,3 and few PROMs are of adequate psychometric quality to produce publishable data. To demonstrate the true value of oncology interventions to patients, more relevant and accurate PROMs are required.

PROMS COMMONLY USED IN ONCOLOGY

Unlike in other medical specialties, the only PROMs used in oncology assess HRQL. In the 1980s,4 two generic cancer outcome measures, the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ-C30)5 and the Functional Assessment of Chronic Illness Therapy, Functional Assessment of Cancer Therapy - General (FACIT FACT-G), were developed.6 These PROMs were produced independently by physicians to collect information of relevance to physicians; consequently, they focus on HRQL outcomes. Both are generic oncology measures with rather crude add-ons intended to be specific to different cancers or symptoms. Additionally, the EORTC QLQ-C30 and FACIT FACT-G measure vastly different sets of outcomes and cannot be used interchangeably.7

Available evidence suggests that the EORTC QLQ-C30 is a relatively poor measure of HRQL.8-10 It fails to meet most of the criteria generally recommended for selecting instruments for use in clinical trials. The lack of adequate patient involvement during the development of the instrument is reflected in poor response rates in practice.11 There is no evidence that the measure has been able to show differences in efficacy between alternative active interventions. This is due to its inadequate psychometric properties.

The FACIT FACT-G also predominantly measures HRQL, although a few items do go beyond and assess patients’ emotional response to their illness. Still, there is limited evidence of its dimensionality or psychometric qualities, as the published data relate to early versions of the measure.6 In fact, for some modules there appear to be no published data to support their quality.

Despite these shortcomings, a recent review of PROMs used in oncology12 found that the EORTC QLQ-C30 (used mainly in Europe) and the FACIT FACT-G (preferred in North America) remain the most commonly used PROMs in this field. Yet, since these were first developed, the ability to measure PROs has advanced considerably in terms of theoretical underpinnings, instrument development methodologies, and statistical modelling.

Where other measures have been used in cancer studies, they are also generic HRQL PROMs, developed for use in other diseases or individual symptom scales that are not specific to a particular cancer.13 Disease-specific measurement is a fundamental requirement of good PROMs—without this specificity, important issues will be missed and irrelevant questions asked. Given that there are almost 200 different forms of cancer,14 the absence of high-quality, holistic PROMs specific to individual cancers is striking.

THE NEED FOR PROMS IN ONCOLOGY

A recent review of the use of PROMs in trials of the 16 most commonly used regulatory-approved treatments for advanced or metastatic breast cancer highlighted the lack of effective outcome measurement.15 The review identified 1727 publications that included PROMs. Of these, more than 1700 were judged to be unsuitable for review, as they did not summarize treatment benefits and/or toxicity of a selected treatment using a PROM. Most studies failed to report the results of the PRO analyses. None of the assessments in the reviewed studies compared different interventions, only reporting change from baseline in treated groups. Consequently, there is no PRO evidence available comparing the effectiveness of the regulatory-approved treatments for metastatic breast cancer.

Other groups have reported similarly disappointing results in oncology.16-18 King et al19 found that evidence on the effectiveness of PROMs in patient management of brain cancer was inconsistent, although they did believe that such data did aid patient–physician communication. 

A fundamental problem with PRO measurement in oncology is an overreliance on HRQL measures. Patrick and Erikson20 define HRQL as “the value assigned to duration of life as modified by the impairments, functional states, perceptions and social opportunities that are influenced by disease, injury, treatment or policy.” This means that HRQL measures take no account of other influences on the quality of the lives of patients, such as personality, financial resources, education, or the availability of family members for support. 

Another major shortcoming of HRQL PROMs is that they assess a range of symptoms and functional impairments resulting from disease, generating a profile of these outcomes. This makes interpreting changes in HRQL difficult, as some outcomes may improve while others deteriorate. According to the European Medicines Agency, a single domain, such as pain or fatigue, is not considered to be a HRQL outcome and cannot be the basis for a claim of HRQL improvement, even though it is patient-reported.21 Instrument developers often try to overcome this problem by adding together scores from the different outcomes. Unfortunately, there is no scientific justification for doing so, and the “total” scores are largely meaningless.22

HRQL PROMs have prospered because they have been preferred by the pharmaceutical industry, which traditionally sought to show the benefit of their products to clinicians and to the US Food and Drug Administration whose primary concerns are clinical efficacy and safety rather than patient value. European health authorities are now requiring evidence of patient benefit in addition to safety and efficacy, exemplified in comments made by European Medicines Agency executive director, Guido Rasi, who explained the shift by saying, “As patients live with their condition on a day-to-day basis, their views on the therapeutic effect of a medicine and its impact on their quality of life… may differ from those of other stakeholders.”23

There is a clear need for new PROMs for oncology that are modern, disease-specific, of high quality, and capable of determining the impact of interventions from the patients’ perspective. The characteristics necessary for such PROMs are provided in Box 1.

Only one non-HRQL measurement model has been widely operationalized in health research. This needs model, which relates outcomes to needs fulfilment,24 grew out of qualitative research conducted to determine how disease and its treatment affects the lives of patients. Symptoms such as pain and fatigue, as well as functions including employment, hobbies, and socialization, are important insofar as they influence the fulfilment of basic human needs. Subsequent analyses demonstrated that these needs can be combined into a coherent unidimensional scale capable of determining the impact of both clinical and non-clinical interventions.25 In contrast to HRQL PROMs, needs-based measures are patient-centric, disease-specific, and based on a coherent outcome measurement model. The development of such measures for use in oncology could help to determine the true, holistic impact of cancer and the value of its treatment for patients.

THE EVOLVING ROLE OF PROMS IN ONCOLOGY

Health care systems around the world are changing. There are too many demands for services and too few resources to fund them. Three major developments can be identified that attempt to address this problem: the development of care pathways, outcomes-based reimbursement (OBR), and big data analytics.

According to Gebhardt and colleagues,26  “clinical pathways are an efficient means of ensuring that the best, most evidence-based treatment is being used for patients.”They later state that it is important to establish patient-centric metrics such as the success of an intervention, survival rates, and treatment toxicity; notably, these are not patient-reported outcomes. In their review of the development and implementation of two pathways introduced at the UPMC Cancer Center (Pittsburgh, PA), they failed to show any evidence of improvement in patient value. Qualitative research conducted to evaluate the state of care pathways in US health care settings found that PROs were not included as key data sources for care pathway development or as commonly used evaluation metrics for such pathways.27 Furthermore, 62% of key stakeholders cited failure to demonstrate patient outcomes as a barrier to pathway expansion.27 These studies suggest that evaluation of clinical pathways focuses primarily on standardizing practices and creating savings for providers and payers rather than on whether they produce benefit to patients.

The move from fee-for-service to an OBR model also requires the ability to measure patient value by means of PROMs. Yet again, it seems that patient outcomes are not being used in such models. As Gupta et al argue,28 “The current metrics used for value-based reimbursement...are surrogate measures that do not measure value directly.” Payers need to consider both clinical and non-clinical interventions when designing programs to maximize potential outcomes. Non-clinical interventions such as social care, education programs, wearable technology, health apps, exercise regimes, and lifestyle changes, all have a role to play in improving patient value, especially when combined with effective clinical treatment. By definition, HRQL outcomes are unable to determine the impact of such interventions.

Big data analytic studies also need holistic unidimensional PROMs that are independent of the nature of the intervention. Rumsfeld and colleagues29 conclude, “If big data analytics are shown to improve quality of care and patient outcomes…big data will fulfill its potential as an important component of a learning health-care system.” Cichosz et al30 also emphasize that predictive models based on big data “must demonstrate impact, namely, their use must generate better patient outcomes.” Surprisingly, a quick review of the big data in health literature highlights the fact that little attention has been paid to the inclusion of PROMs. Indeed, outcomes are rarely defined in discussions of big data analytics.

These major developments all require the availability of disease-specific PROMs that go beyond the assessment of HRQL outcomes. While the HRQL PROMs used in oncology provide information of value to the clinician, they do not necessarily indicate what is of primary concern to the patient. HRQL outcomes are also of limited value to clinical pathway development and evaluation, OBR, and big data analytics, as they do not assess non-clinical interventions.

PROMs that are relevant for patients and payers, and that better exemplify patient value, are essential for implementing and evaluating modern approaches to health service provision. Complementary measures of HRQL can have value for clinicians. However, it is questionable whether the dated, generic HRQL measures such as the QLQ-C30 and the FACT-G provide an adequate quality of measurement.

CONCLUSION

PRO measurement in oncology lags compared with other clinical specialties. Innovative approaches to health care delivery, evaluation, and reimbursement are welcomed, but they rarely consider patient outcomes in a meaningful manner. Too often, emphasis is placed on motivating physicians, easing commissioning, and reducing costs, irrespective of the effects these changes have on the patient. There is a clear need for the development of high-quality, disease-specific PROMs that assess the true concerns of patients and that evaluate the impact of both clinical and non-clinical interventions on a variety of outcomes. Only then will it be possible to demonstrate the true impact of oncological and other interventions on the lives of patients. 

For other takes on this topic, read the Counterpoints, "Incorporating Routine Patient-Reported Outcomes Assessment Into Cancer Care: Building Momentum" and "The Pivot Toward Patient-Centeredness in Medicine and Oncology." 

References

1.    Patrick DL, Guyatt GH, Acquadro C; Cochrane Patient Reported Outcomes Methods Cochrane Review Group. Patient-reported outcomes. In: Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of
Interventions. Version 5.1.0. 2008. http://handbook.cochrane.org/index.htm#chapter_17/17_patient_reported_outcomes.htm. Accessed August 18, 2016.

2.    Friedlander ML, King MT. Patient-reported outcomes in ovarian cancer clinical trials. Ann Oncol. 2013;24(Suppl 10):x64–x68.

3.    Dawson J, Doll H, Fitzpatrick R, Jenkinson C, Carr AJ. The routine use of patient reported outcome measures in healthcare settings. BMJ. 2010;340:c186.

4.    Chang SK, Hawes KA. The adequacy of the Karnofsky Rating and Global Adjustment to Illness Scale as outcome measures in cancer rehabilitation and continuing care. Prog Clin Biol Res. 1983;120:429-443.

5.    Aaronson NK, Ahmedzai S, Bergman B, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85:365-376.

6.    Cella DF, Tulsky DS, Gray G, et al. The Functional Assessment of Cancer Therapy scale: development and validation of the general measure. J Clin Oncol. 1993;11:570-579.

7.    Kemmler G, Holzner B, Kopp M, et al. Comparison of two quality-of-life instruments for cancer patients: The Functional Assessment of Cancer Therapy-General and the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-C30. J Clin Oncol. 1999;17:2932-2940.

8.    Hjermstaad MJ, Fossa SD, Bjordal K, Kaasa S. Test/Retest of the European Organisation for Research and Treatment of Cancer Core Quality-of-Life Questionnaire. J Clin Oncol. 1995;13:1249-1254.

9.    Wisloff F, Eika S, Hippe E, et al. Measurement of health-related quality of life in multiple myeloma. Br J Haematol. 1996;92:604-613.

10.    Tannock IF, Osoba D, Stockler MR, et al. Chemotherapy with mitoxantrone plus prednisone or prednisone alone for symptomatic hormone resistant prostate cancer: A Canadian randomised trial with palliative end points. J
Clin Oncol. 1996;14:1756-1764.

11.    Juul T, Petersen MA, Holzner B, Laurberg S, Christensen P, Grønvold M. Danish population-based reference data for the EORTC QLQ-C30: associations with gender, age and morbidity. Qual Life Res. 2014;23:2183-93.

12.    Coulter A, Potter C, Peters M, Fitzpatrick R. Cancer PROMs: A scoping study. MacMillan Cancer Support web site. http://www.pssru.ac.uk/archive/pdf/5079.pdf. Accessed August 18, 2016.

13.    Annant M, Randeep G, Ashutosh P, et al. Quality of life measures in lung cancer. Indian J Cancer. 2005;42(3):125-132.

14.    Cancer Types. National Cancer Institute Web site. http://www.cancer.gov/types. Accessed August 18, 2016.

15.    Turner-Bowker DM, Hao Y, Foley C, et al. The use of patient-reported outcomes in advanced breast cancer clinical trials: A review of the published literature. Curr Med Res Opin. 2016;22:1-37.

16.    Dirven L, Taphoorn MJ, Reijneveld JC, et al. The level of patient-reported outcome reporting in randomised controlled trials of brain tumour patients: A systematic review. Eur J Cancer. 2014;50:2432-2448.

17.    Krohe M, Hao Y, Lamoureux RE, et al. Patient-reported outcomes in metastatic breast cancer: A review of industry-sponsored clinical trials. Breast Cancer (Auckl). 2016;10:93-102.

18.    Fallowfield L, Payne H, Jenkins V. Patient-reported outcomes in metastatic castration-resistant prostate cancer [published online ahead of print June 28, 2016]. Nat Rev Clin Oncol. 2016. doi: 10.1038/nrclinonc.2016.100.

19.    King S, Exley J, Parks S, et al. The use and impact of quality of life assessment tools in clinical care settings for cancer patients, with a particular emphasis on brain cancer: insights from a systematic review and stakeholder consultations. Qual Life Res. 2016;25:2245–2256.

20.    Patrick DL, Erikson P. Health Status and Health Policy. Oxford: Oxford University Press; 1993.

21.    European Medicines Agency. Reflection paper on the regulatory guidance for the use of health related quality of life (HRQL) measures in the evaluation of medicinal products. http://www.ispor.org/workpaper/emea-hrql-guidance.pdf. Published July 27, 2005. Accessed August 18, 2016. 

22.    Schröder A, Oernboel E, Licht RW, Sharpe M, Fink P. Outcome measurement in functional somatic syndromes: SF-36 summary scores and some scales were not valid. J Clin Epidemiol. 2012 Jan;65(1):30-41.

23.    Patients to discuss benefit-risk evaluation of medicines with the Committee for Medicinal Products for Human Use [press release]. European Medicines Agency. http://www.ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/news/2014/09/news_detail_002172.jsp&mid=WC0b01ac058004d5c1. Published September 26, 2014.

24.    Hunt SM, McKenna SP. The QLDS: A scale for the measurement of quality of life in depression. Health Policy. 1992;22:307-319.

25.    Rouse M, Twiss J, McKenna SP. Co-calibrating quality-of-life scores from three pulmonary disorders: implications for comparative-effectiveness research. J Med Econ. 2016;19:596-603.

26.    Gebhardt BJ, Beriwal S, Heron DE. Facilitating physician adherence to clinical pathways across a multi-site oncology practice network. J Clin Pathways. 2016;2:33-40.

27.    Chawla A, Westrich K, Matter S, Kaltenboeck A, Dubois R. Care pathways in US healthcare settings: Current successes and limitations, and future challenges. Am J Manag Care. 2016;22:53-62.

28.    Gupta D, Karst I, Mendelson EB. Value-Based Reimbursement: Impact of Curtailing Physician Autonomy in Medical Decision Making. AJR Am J Roentgenol. 2016;206:276-2799.

29.    Rumsfeld JS, Joynt KE, Maddox TM. Big data analytics to improve cardiovascular care: promise and challenges. Nat Rev Cardiol. 2016;13:350-359.

30.    Cichosz SL, Johansen MD, Hejlesen O. Toward big data analytics: Review of predictive models in management of diabetes and its complications. J Diabetes Sci Technol. 2015;14: 27-34.

31.    Tennant A, Conaghan PG. The Rasch measurement model in rheumatology: what is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Rheum. 2007;57:1358-62.