Skip to main content

Pharma Insights

Open-Source Tools for Value Assessment: A Promising Approach

Authored by

Mark T Linthicum, MPP; Jennifer Bright, MPA; Jason Shafrin, PhD


Innovation and Value Initiative, Los Angeles, CA 


The Innovation and Value Initiative is a part of Precision Health Economics (PHE), a health care consultancy with clients in the pharmaceutical and insurance industries. Linthicum and Shafrin are employees of PHE, and Shafrin holds equity in PHE’s parent company, Precision Medicine Group. Bright is a consultant to PHE and Principal of Momentum Health StrategiesTM.


 J Clin Pathways. 2018;4(5):37-39. doi:10.25270/JCP.2018.06.00024

While many advancements have been made thus far in the progression toward value-based health care, there are persistent challenges in the measurement of value. In order to deliver value-based care, health care decision makers, eg, insurers and health system administrators, need value data at their fingertips—data that are relevant to their own context and reflect their own perspective on what costs and benefits matter. With ongoing input from all stakeholders and real-life application in settings such as clinical pathway design, these tools will evolve and improve.

When clinical pathways entered practice in the 1980s, they represented the leading edge of the value-based health care discussion. Since that time, new approaches for incentivizing the use of high-value care—including outcomes-based contracts, value-based reimbursements, indication-specific formularies, and others—have emerged in response to the growing need for decision makers to balance the benefits of health care against its costs.

While these innovations in care delivery and reimbursement represent significant progress, there are persistent challenges in measuring value in health care. Many of these challenges derive from one simple fact: individual health care decision-makers face unique circumstances and need personalized information about value to make the best choices for maximizing value in any given situation. To provide this kind of context-relevant information, we must vigorously pursue the creation of scientifically valid, flexible, and transparent quantitative tools to inform value assessments that are relevant to real-world experiences. 

In Value Assessment, One Size Does Not Fit All

There is widespread agreement that we need to credibly assess the value of health care and make decisions based on value. The question is, how?

Decades of work in economics, epidemiology, and other disciplines have built a solid foundation for value assessment. The leading and most commonly used approach to comparing relative value of therapies is cost-effectiveness analysis (CEA), which compares treatments or practices based on their incremental costs relative to the benefit generated.1,2 Benefits are generally measured in terms of quality-adjusted life years, a measure that aims to capture impact on both quantity (years of life lived) and quality of life. CEA is commonly used outside the United States for health care resource decisions and is the basis of work in the United States by nongovernmental organizations such as the Institute for Clinical and Economic Review.3

These conventional approaches have important limitations, however, especially in the context of the decentralized US health system. First, conventional value assessments based on population averages may fail to account for patient heterogeneity, even when they examine clinically relevant subgroups. Patients vary immensely along dimensions likely to impact the value of an intervention, eg, clinical biomarkers, demographics, previous treatment history, and comorbidities, to name a few. For decision makers trying to maximize value, information on value needs to reflect the diversity of clinical contexts faced by the patients who actually receive these treatments. Similarly, this information needs to be up-to-date; the conventional approach of generating reports on population cost-effectiveness provides a snapshot based on the evidence available when the study was conducted, but these analyses are seldom updated to reflect new evidence or changes in clinical practice.

Perhaps more importantly, conventional approaches to value assessment struggle to accommodate the fact that the meaning of “value”—the elements that fill the columns under benefits and costs, and their relative importance—changes based on a stakeholder’s role in health care. In addition to treating their diagnosed illness, patients may define value based on factors such as impact on mobility and daily activities, ability to return to work, or burden on caregivers. On the cost side of the equation, patients care not only about out-of-pocket costs, but also about missed work, transportation costs, and time cost of navigating an insurer’s often complex approval process. Employers may be concerned about effects on absenteeism and business productivity. Hospitals and health systems may prioritize avoidance of hospital readmissions and infections. 


CEA approaches often take the perspective of a single “payer” managing population health and costs; recently, the Second Panel on Cost Effectiveness in Health and Medicine recommended that CEA studies also include a broader, societal perspective.2 Including the societal perspective is a good first step, but to provide relevant, actionable information on value to decision makers, we need methods that incorporate these diverse perspectives and the various determinants of value for each.

Advancing the Science of Value Assessment

The challenges described above present difficulties in the science of assessing value in health care, as well as in the practice of incorporating value information into decision-making. Health care decision makers, eg, insurers and health system administrators, need value data at their fingertips—data that are relevant to their own context and reflect their own perspective on what costs and benefits matter. Decision makers need transparency into how value is calculated and how assumptions affect the final result of the value equation. Furthermore, stakeholders need visibility into how differences in priorities can affect the final result and the ability to include all of the relevant costs and benefits in the calculation. 

To meet this need, the science, tools, and evidence base for value assessment must evolve considerably. Such an evolution requires many actors working collaboratively to advance value research, and such collaboration requires transparency and reproducibility. 

The Innovation and Value Initiative (IVI), for example, is drawing inspiration from open-source approaches to software development in its Open-Source Value Project (OSVP).4 The OSVP develops disease-specific decision models designed to provide scientifically valid and flexible approaches to quantify value. These tools allow users, especially health plans, employers, and health system administrators, to evaluate different sequences of treatment based on unique factors of value as defined by their covered population. OSVP models are designed to provide maximum flexibility for decision makers, including control over the scientific assumptions that lead to simulated outcomes such as lifetime costs, number of hospitalizations, changes in quality and length of life, and more.

Decision makers are also able to determine how best to combine these outcomes in a value calculation—whether to use CEA vs a multicriteria decision analysis approach, for example, and whether to include factors such as effects on economic productivity or novel measures of value such as insurance value.5 By placing all OSVP models in the public domain, including source code, detailed documentation, public feedback, and plans for revision, the iterative development process remains fully transparent while facilitating collaboration. 

How Can Open-Source Value Models Inform Local Decision-Making?

As an example of how flexible, open-source models might support decision-making, consider the case of rheumatoid arthritis (RA). RA is a chronic, life-long, and progressive disease that requires treatment over the patient’s lifetime.6 Many newly diagnosed RA patients are initially treated with traditional disease-modifying antirheumatic drugs (DMARDs). They then move on, due to lack or loss of clinical response or intolerable side effects, to more expensive biologic DMARDs, of which there are numerous options with different mechanisms of action.7 While this initial course of treatment is common, the reality of RA treatment is that patient response is highly varied.8 As a result, RA management requires individualized treatment planning by rheumatologists, necessitating clinical flexibility. Meanwhile, RA therapies are a major driver of pharmacy spending for many health plans, so there is a need to balance clinical flexibility against cost considerations.

The first OSVP modeling platform focuses on biologic treatments for moderate to severe RA. The IVI-RA modeling platform allows users to examine the value of sequences of therapies through an individual patient simulation.6 To illustrate the use of a decision model like the IVI-RA platform, imagine an insurer grappling with coverage decisions for RA. Faced with the need to balance rheumatologists’ flexibility in treatment decisions against population costs and benefits, an insurer could first tailor the RA population parameters to match those of the covered population and set cost parameters equal to the exact costs faced. The insurer could then compare alternative sequential treatment strategies (where failure on one therapy leads to initiation of another specified therapy) over patients’ lifetimes or over a discrete time frame. This information would provide insight into the impact of sequencing and other assumptions on variability in value for a specific patient population. 

Furthermore, given information on what drives value for covered patients, the insurer could then identify patient-preferred treatment sequences and compare their relative value. Mode of administration, for instance, may be a determinant of value for many patients; others may place a high value on the amount of available real-world safety evidence. By examining different benefit designs through the lens of multicriteria decision analysis, which allows weighting of very different attributes, the insurer could access actionable data on which designs deliver the most value—not only to the average patient but to the majority of their covered patients in the real world. 

The current version of the IVI-RA modeling platform is a starting point for such context-specific explorations of value. As the ongoing open-source development process continues, so too will the potential opportunity for real-world applications. One example is linking to local health system data or incorporating patient preferences from a specific population. Ultimately, the IVI-RA platform and others like it may also inform context-specific pathway design by allowing for tailored comparison of different courses of therapy over time—using local real-world data to supplement clinical evidence.

Striving to Build Actionable Tools for Real-World Decision-Making

Balancing cost and clinical outcomes is an essential activity of achieving a value-based health care system, and decision makers need comprehensive, relevant, and up-to-date information to make value-based health care a reality. Static assessments of value and published reports on cost-effectiveness provide only a fraction of the information needed. IVI believes that the missing components in the current value assessment ecology are dynamic tools that allow decision makers to access objective, up-to-date, and customized information on value and the ability to adjust the calculations based on an explicitly stated decision-maker perspective, tailoring the output to a specific use. 

Development of such open-source tools is nascent. With ongoing input from all stakeholders and real-life application in settings such as clinical pathway design, these tools will evolve and improve. At the same time, IVI’s focus on advancing methods in value assessment aims to contribute fresh thinking and solutions to the public discussion about the important perspectives and determinants of value inherent in patient experience.


1. Robinson R. Cost-utility analysis. BMJ (Clinical research ed). 1993;307(6908):859-862.

2. Sanders GD, Neumann PJ, Basu A, et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: Second panel on cost-effectiveness in health and medicine. JAMA. 2016;316(10):1093-1103.

3. Institute for Clinical and Economic Review. Overview of the ICER value assessment framework and update for 2017-2019. Accessed May 14, 2018.

4. Jansen JP, Incerti D, Linthicum MT. An open-source consensus-based approach to value assessment. Health Affairs Blog. Published December 14, 2017. Accessed May 22, 2018.

5. Lakdawalla D, Malani A, Reif J. The insurance value of medical innovation. J Public Econ. 2017;145:94-102.

6. Incerti D, Jansen J. A description of the IVI-RA model v1.0. Published December 13, 2017. Accessed May 22, 2018.

7. Singh JA, Saag KG, Bridges SL, et al. 2015 American College of Rheumatology guideline for the treatment of rheumatoid arthritis. Arthritis Rheum. 2016;68(1):1-26.

8. Wijbrandts CA, Tak PP. Prediction of response to targeted treatment in rheumatoid arthritis. Mayo Clinic Proc. 2017;92(7):1129-1143.

Back to Top