Evidence Blocks to Evaluate the Relative Value of Oncology Treatments

An interview with Robert Carlson, MD, Chief Executive Officer, and Joan McClure, MS, Senior Vice President, National Comprehensive Cancer Network

Robert Carlson, MD, and Joan McClureMany institutions and health care professionals have been aware of the staggering rise of health care costs. An IMS Health report published in May of 2015 confirmed that drug sales in the United States surpassed $100 billion in 2014.1 Findings from a recent 2014 study report that costs related to health care are the leading cause of personal bankruptcies in the United States.2

As a result, cost is becoming a key factor to consider when determining treatment plans. Patients, aware of the staggering financial toll treatment can have on themselves and their families, are asking for greater explanation about the basis for their treatment and if it is worth the economic impact. Additionally, many patients bring their own values to treatment discussions, including a greater emphasis on quality of life, control of side effects, and financial burden.

At the same time, 2014 also saw an increase in drug development, with 44 drugs being approved by the FDA.3 Of those 44 approvals, 8 were for the treatment of cancer.3 Between 2010 and 2014, a total of 34 cancer agents were launched in the United States.1 With so many treatments available to patients, there has been some concern over how to discern which options are best for different patients.

Over the past 20 years, the National Comprehensive Cancer Network® (NCCN®) has developed the NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®), a comprehensive set of guidelines detailing the sequential management decisions and interventions that currently apply to 97% of cancers affecting patients in the United States. The NCCN Guidelines document evidence-based, consensus-driven management to ensure that all patients receive preventive, diagnostic, treatment, and supportive services that are most likely to lead to optimal outcomes. In addition, separate guidelines provide recommendations for some of the key cancer prevention and screening topics as well as supportive care considerations. The intent of the NCCN Guidelines is to assist in the decision-making process of individuals involved in cancer care—including physicians, nurses, pharmacists, payers, patients and their families—with the ultimate goal of advancing patient care. However, some have argued that the NCCN Guidelines are too inclusive and do not go far enough to distinguish treatments that truly provide superior outcomes to patients from those that offer only marginal benefits. Furthermore, the guidelines do not provide information about the strength of the evidence supporting the use of particular treatments.

To address these concerns, NCCN announced on October 16th the release of its first new visual framework for assessing treatment options, called NCCN Evidence Blocks™. The aim of these tools is to help patients and providers better visualize the relative value of different treatments, taking into account the factors most important to the patient. The Journal of Clinical Pathways spoke with NCCN’s Robert Carlson, MD, Chief Executive Officer, and Joan McClure, MS, Senior Vice President of Clinical Information and Publications, about what factors led to the development of NCCN Evidence Blocks and their plans to develop Evidence Blocks for more disease states in the future.

What needs did you identify within the health care center that led you to create the Evidence Blocks?

RC: The Evidence Block concept really arose out of requests from a number of different stakeholders within cancer care. We had providers who were using our guidelines but requesting more information about why a specific therapy was recommended or preferred over other therapies. We had patients who wanted to better understand the trade-offs between different options of care, so that they could participate more in shared decision-making. The payers wanted that same information, but they also have asked us—as have many patients—to add more information about cost or affordability of different regimens to add transparency to health care decision-making. The industry has always been interested in differentiating their products, and I think they’re interested in some of the efficacy and safety dimensions; although, in general, they have not been as enthusiastic about us adding affordability or cost into the equation. So this was something the entire industry was interesting in having. 

What kind of benefit do you hope these Evidence Blocks will offer to decision-making among those different stakeholders?

RC: I think the biggest benefit that we envision happening here is that it adds transparency to the decision-making process, especially for the patient. So that patients, once they understand what the Evidence Blocks look like, can look at the different dimensions, and that can allow them to key in on those that are likely to be of most interest to them and then work with their health care providers in a shared decision to understand which of the available options is the best for them. The Evidence Blocks are not going to allow patients to make conclusions, but I think they may well allow patients to sort of restrict the choices and then work with their health care providers to decide among the choices they’re faced with and decide which is the best for them. 

JM: I would also add that we have 49 guideline panels that have been developing clinical practice guidelines for the last 20+ years. And the criteria for the Evidence Blocks, with the exception of affordability, are the same criteria that the panel members have been using for years when they have been deciding which treatments they feel are appropriate and necessary to include in the guidelines. So, they have been considering the efficacy, the safety, the quality and quantity of the evidence, and the consistency of the evidence in making their decision. But how they have weighted those various aspects has not been transparent to the community. So I think what this does is actually open the door to the deliberations of the panel so that the whole oncology community can understand how the decisions were made. 

Was there any concern involved with including “affordability” as a factor? 

JM: We have thought about affordability for a long time, but we have not added that as a criterion in the development of our guidelines—in fact we have still not. But as cancer therapies have become more expensive and patients have been exposed to more costs through their copays, their coinsurance, and their deductibles, our panel members have become concerned about people being aware of the different costs of therapies. So, I think, among the NCCN, there has been very little concern about adding affordability. I think that everyone in NCCN believes that this is information that patients and providers who are not disease sub-specialists need to have.

Is there a reason you chose the term “affordability” as opposed to “cost?”

JM: Oh, yes. When we piloted the concept, we used toxicity instead of safety and cost instead of affordability. And then we were going to shift as we built the blocks, so that a fuller block would be better than an emptier block. Because these are 5x5 graphs. So, the more it was filled in, the better an intervention rate on that particular criterion. And we realized that people were getting confused and scoring things that were of low toxicity—

RC: As a low score instead of a high score. So what ended up happening was that, if we have efficacy instead of toxicity, a full bar for efficacy makes sense—because it’s really good—a full bar for affordability then would equate to low cost, so high affordability. It ended up being counterintuitive that a high bar was low cost or that a high bar was low toxicity. So we ended up inverting them so that a high score was intuitive. So that’s how we ended up with safety, which we see as the inverse of toxicity, and affordability, which we see as the inverse of cost. This made it easier to visualize.

What are the reasons that each of these facets is so important in optimizing care? 

RC: Well, if you ever have an opportunity to be in the room when a panel meeting is going on, the single thing the panel emphasizes the most is efficacy. That’s really their first filter or the first thing they’re focused on. And then after they focus on efficacy, in terms of treatment options, then toxicity becomes the next most important consideration. And then as they’re looking at the various studies available—and these are actual clinicians, researchers, and clinical investigators—they focus very closely on the quality of the study that they are evaluating in order to come up with the efficacy and toxicity numbers. So, quality, quantity, and the consistency of data are something that the panels routinely and consistently have focused on throughout the years. And that’s why those four major facets were chosen, because that is actually where most of conversation resides. Occasionally there will be discussions—not so much in systemic therapies, although there are circumstances where that’s considered—about the expertise the therapy requires: whether the skillsets and resources needed to actually be able to administer the specific therapy are available. And then, of course, we added cost, not because our panel used that to put things on the guidelines, but just because our society is telling us that we have to start considering costs. And as Joan pointed out, more than two-thirds of personal bankruptcies in the United States today are health care bankruptcies. And so, you know, cost has to be something that our patients and our physicians have to pay attention to. So that’s why cost—or rather affordability—was added.

JM: If you look at it from the patient’s perspective, the quality and quantity of the data actually is very relevant to the patient, too. Because there are some people who see relatively preliminary data that look very, very good in one study and they would like to go with something that looks promising, even though there is a distinct possibility that those data are not going to be replicated in other studies. Other people want to be sure of the quality of the data they’re basing their decision on and really are looking for more data and more consistency. Patients feel more comfortable with a treatment when they have an understanding that it has worked well in the past. 

How can the Evidence Blocks be used with consideration for different needs and priorities of individual patients in order to ensure that treatment decisions are personalized? 

RC: Joan just commented on some of that. It’s really trying to work with the patient to identify what is important to them. The example I always use is that a 25-year-old woman with two small children comes in with a very high-risk breast cancer and she wants to talk about adjuvant systemic chemotherapy. Almost all of her discussion is going to be about efficacy. She will not be terribly concerned about toxicity. She’ll just say the payback time is decades for me if we get this right. I’m not all that concerned about toxicity. And she is almost certainly not going to be concerned about cost. Now on the other hand, if you have a 85-year-old woman come into the clinic with an identical breast cancer, almost certainly the first thing she is going to talk about is quality of life and safety. And the second thing will be cost or affordability. The 85-year-old would be much less concerned about efficacy. She will have questions about efficacy, certainly, but it will not be her first concern. And we viewed both of those perspectives as valid and important, and by using Evidence Blocks, the individual patient will be able to look at the part of the Evidence Block most important to them as their initial filter for what options might be best for them. 

And it probably benefits the physician as well, just in terms of being able to understand what patients want and map out clearly what the pros and cons of different treatments will look like.

RC: Yes, exactly. One of the things that may well happen in the exam room when these are used with patients is that, just because of the fact that there is an affordability measure, it gives permission for everyone in the room to talk about affordability. I think some patients are very hesitant to talk with their doctors about affordability just out of the blue. They’re embarrassed, they may not have the financial resources, but they also don’t want to bias the physician into using something that is not actually efficacious perhaps because they’re concerned about cost. And I think physicians are hesitant to talk about cost because they don’t want the patient to think that they’re out for money or that money is what is most important to them and not the patient’s health. Consequently, I think there is a conspiracy of silence that often happens around cost and affordability within the health care environment. So then having this there, on the page in front of them where it really jumps out at you, will hopefully demystify that and get people to really talk about those issues.

JM: I think the other thing that it will do is to give a signal to the clinician and to the patient, when something is quite expensive, that it is time to talk with the person in the practice who helps patients figure out their insurance coverage and what their own share of the cost is going to be. Because cost is variable, depending on the plan and the copay and the various discounts or deals that have been made between the pharmacy and the drug company. 

How does the Evidence Block system differ from other tools that are out there, such as the Memorial Sloan Kettering Drug Abacus or the American Society of Clinical Oncology (ASCO) Value Framework?

RC: Well, first of all, I think the three systems are really complimentary, and they look at different issues and certainly use different methodologies. The Memorial Sloan Kettering Drug Abacus really is focused primarily on cost: is the cost of this agent appropriate given its efficacy, its toxicity profile, and other considerations? But really, it is all about what the cost of a drug should be, and it considers the drug in isolation. Our cost metric looks at the entire episode of care. So it looks at the drug cost, the administration costs, the toxicity monitoring cost, and so forth.

ASCO’s Value Framework requires that a randomized trial be available, because the tool only works when there are randomized trials available. And the comparisons have to be direct, so that if there are randomized trials available for treatments in a certain disease state, but, specifically, the two treatments you’re considering in that circumstance don’t have a randomized trial between just those two drugs, then you can make no decision or no comment based upon the ASCO initiative.

The other differences are that the ASCO system has a fixed formula for how to weigh efficacy and toxicity. Our system allows flexibility there; we don’t give an answer or a conclusion or a number, just different options. And so that’s different. Also, the ASCO tool calculates a number, and there’s no clear way to look at that number and sort of adjust it for the two patients I gave you earlier—the 25-year-old and the 85-year-old—it’s the same number for both of them. On the other hand, the ASCO tool is a very objective tool, because it has a fixed formula. You know, 10 different people can look at a study and all come up with the same number. So it does have the strength of being a solidly objective tool. The fact that there is the same number, though, is potentially problematic, because the calculation is a relative calculation between two different therapies. If you have two different therapies, for instance, that are both highly effective, you may come up with a low number, because they are both highly effective and while they are very effective, there is still subtle differences between the two. On the other hand, you may have a situation with two drugs that are just awful drugs, and yet the percentage difference may be a 1% versus a 4% outcome and that’s a 400% difference in the two. And so that actually may have the appearance of a very high number, where it looks like the therapy is very, very effective, when in fact it really is not. So, we do have some concerns about how the number is used in the ASCO tool.

At the moment, there are Evidence Blocks available for only two disease states: chronic myelogenous leukemia (CML) and multiple myeloma. What other Evidence Blocks are planned for the future?

JM: In progress, we have: colorectal cancer, melanoma, non-small cell lung cancer, breast cancer, kidney cancer, some subtypes of non-Hodgkin’s lymphoma that we’ll be following on, and prostate cancer. We’re planning on rolling these out for our whole library over the next couple of years, and we’re starting with cancers that are high incidence and high interest or where there is a range of therapies available.

Is that what led you to releasing these two first? That they were high in incidence, interest, and had multiple treatment options?  

RC: So, it was a pretty pragmatic decision. We released CML and multiple myeloma first, because when we released these, there was an educational activity that had to go on with the health care system of providers to understand how the Evidence Blocks are actually developed. And we released them at our Hematologic Malignancies Conference [on October 16-17, 2015, in San Francisco, CA] because that gave the opportunity to educate 600 providers at the same time that the blocks were actually released. So it was an educational opportunity we didn’t want to miss. 

JM: It also had to do with our season of guidelines updates; we start our so-called “season” earlier with our Hem-Malignancies, because of ASH being in the winter, and the solid tumor with ASCO’s Annual Meeting being in the summer time. So, it felt right for us. The stars were aligned.

Is there anything additional you would like Journal of Clinical Pathways readers to know about the Evidence Blocks?

RC: I think it is going to be very interesting how and when they start to be used in the clinic: how effective they are, and how we will need to modify them in the future to make them a better tool. And then, also, we will be expanding them ultimately to include radiation oncology, surgical oncology, diagnostics. As we start to look at those different types of therapy, then almost certainly we are going to need a different scale for those circumstances as well. So, you know, it is going to be fascinating to see the way everything unfolds and what sort of measures we use for those different types of therapy. 



1.    IMS Institute for Healthcare Informatics. Developments in cancer treatments, market dynamics, patient access and value: global oncology trend report 2015. http://bit.ly/1QKEIS6 Published May 2015. Accessed November 17, 2015. 

2.    LaMontagne C. NerdWallet Health finds medical bankruptcy accounts for majority of personal bankruptcies. NerdWallet website. http://nerd.me/1kb7wlk. Published March 26, 2014. Accessed November 17, 2015. 

3.    Munos B. 2014 New drug approvals hit 18-year high. Forbes. http://onforb.es/1422O5I Published January 2, 2015. Accessed November 17, 2015.

4.    Tefferi A, Kantarjian H, Rajkumar SV, et al. In support of a patient-driven initiative and petition to lower the high price of cancer drugs. Mayo Clinic Proceedings. 2015;90(8):996-1000.