Measures That Matter — But To Whom?

The health care and health policy communities are increasingly calling for “measures that matter.” Journal articles and policy reports have highlighted the proliferation of performance measures, and many conferences have focused on reducing the number of metrics to a small set. Overall, there are strong desires to rationalize the entire measurement approach, with leaders like Don Berwick advocating for an immediate reduction in measurement (perhaps by 50 percent).

One notable effort is the CMS-AHIP Core Quality Measures Collaborative, which seeks to align measures across public and private payers. The Collaborative, supported by technical assistance from the National Quality Forum (NQF), has identified core measures for several clinical specialty areas, and further efforts will be needed to identify such measures for additional specialties.

The Institute of Medicine (IOM)’s Vital Signs report took a different approach. The report identified the broad conceptual areas that span across the health care system and community health, and it outlined 15 areas where it is important to assess the nation’s progress, from life expectancy to healthy communities to patient safety to community engagement. The challenge now, as described by the IOM committee leaders, is to identify specific metrics in these areas that can be used for particular and very different situations, such as accountability for public health agencies or assessing a small clinical practice.

These initiatives share the goal of ensuring we can get the best information for the effort spent collecting measurement data. Yet defining the measures that matter is complicated because we often gloss over an important question — measures that matter to whom?

It’s no surprise that different groups have different thoughts about what’s important. If we think about air travel, passengers are less interested in whether a pilot consistently uses their checklist — but that measure is still very important for pilots to use and regulators to monitor. Passengers also may not care so much about fuel consumption per passenger mile, which is of keen interest to airlines. But passengers will care about their ticket price, experience, timeliness, and safety.

A similar situation is at play in health care. Different groups will want to know different information, and they will put that information to use in different ways. The following table highlights some of the varied perspectives on measures that matter, and what is important to the broad spectrum of stakeholder groups. Adding to the complexity, each stakeholder group is itself diverse, and there will be exceptions to the broad themes in the table. Given space constraints, the table also leaves out many important stakeholders (such as payers, population health, and government) who need and use health care measures.

Table: Stakeholder Perspectives On Measures That Matter

Different stakeholders will not only have different perspectives about what measures matter to them, but even have different views on what the same terms mean. For example, stakeholders generally look at health care costs differently. Patients will consider out-of-pocket costs, while employers look at premiums and impact of health on employee productivity. Insurers define costs in terms of claims, while hospitals and doctors see the emphasis on costs as a focus on squeezing their revenue and prices. And federal and state governments may consider the impact of health care on annual budgets.

How Can We Move Forward?

The biggest lesson is that we need all perspectives at the table to know which measures matter broadly. The multistakeholder process helps us know whether a metric assesses something important, whether it will be useful, and whether there are any unintended consequences to implementing it. This process isn’t easy (and it takes a lot of work), but it’s the only way to ensure measures are meaningful.

We are lacking measures that matter in many important areas — and these gaps are not being filled fast enough. To address this challenge, we have launched the NQF Measure Incubator to fill measurement gaps in those important areas, like patient-reported outcomes or measures for patients with multiple chronic conditions. The Measure Incubator aims to reduce measure development time by allowing for access to big data sets and clinical systems as test beds to more rapidly test and adjust measures.

We also need to clarify the intended use of measures or the goals that measures are seeking to accomplish. Rather than every measure being deployed for every use and user—which may lead to too many measures—it is more fruitful to focus on the specific goals for a measure. For example, the NQF Measure Applications Partnership (MAP) applies this philosophy when recommending measures for federal programs, with each measure being considered in light of the specific goals and needs of the program. Furthermore, growing availability of better data sources will make it possible to personalize measures to better meet the disparate needs of diverse stakeholders (without adding to the data collection burden) and account for factors that influence health outcomes, such as homelessness and language barriers.

There will always be a tension between wanting a small set of measures (that reflect the nation’s top priorities) and patients’ and clinicians’ need for measures reflecting specific uses and conditions. While the recommendations of the MAP focus on measures for federal programs, the growing tension between “too many measures” and “not the right kinds of measures” will require collaboration between—and action by—federal, state, and private sector leaders.

The NQF Board, which includes leaders from various stakeholder groups, is in the final stages of developing a strategic plan to address this need — and this issue has emerged as one of the core challenges. Whatever solutions are identified will take discipline to implement and not everyone will be happy. After all, the only way to reduce burden is to have everyone give up sacred cows in measurement.

Regardless of where you sit, we should all agree that the measures that matter are the ones that can help improve health care and people’s health — no more and no less.