5
HARMONIZED PERFORMANCE MEASUREMENT FOR CONTINUOUS LEARNING
Performance measurement is a topic of great concern to health system leaders. In this session, panelists discussed uses of data for understanding performance, measuring performance, and creating the next generation of more meaningful performance measures. David Blumenthal, president and CEO of The Commonwealth Fund, discussed the design of infrastructures for data collection that are also useful for research. Christine Cassel, president and CEO of the National Quality Forum (NQF), described the potential of data infrastructures to serve as the measurement framework for accountability at the national level. Benjamin Chu, group president for Kaiser Permanente’s Southern California and Georgia regions, discussed his experiences putting data into action to achieve better outcomes through a systems-based approach. Highlights and main points are summarized in Box 5-1.
INFRASTRUCTURES FOR DATA COLLECTION
A challenge for health organizations is to create data infrastructures that are useful for measurement both at the national level and for the purposes of their own improvement. According to Blumenthal, the way to create such an infrastructure is to design it for research purposes. He acknowledged, however, that the available electronic systems are not designed for research purposes.
He noted that efforts to design EHR technology to meet Centers for Medicare & Medicaid Services (CMS) Meaningful Use requirements have considered what basic clinical elements might be important in laying the foundation for a National Patient-Centered Clinical Research Network (PCORnet)-style research infrastructure. As such, current certified information systems do have a core data component that supports clinical research and the comparison of data across institutions. The quality metrics that are specified under the Meaningful Use rule fit that criterion, for example. There are still many aspects of currently
available EHR systems that do not support research, and elements need to be added after the fact. One example is the need for open application programming interfaces that would allow researchers to develop software that could interface with multiple EHR data repositories.
Even if research is considered at the outset, he continued, it is impossible to design a system that anticipates the many evolving research needs of a country,
region, or individual system. Because it is not possible to predict the data that will be required to answer as yet unknown questions, EHR systems must be adaptable and reconfigurable to make them useful. Blumenthal suggested that the ability to redesign electronic information systems should be a core competency of health organizations, as essential as the ability to dispense medicines appropriately or ensure a hygienic environment. The capability for adaptation will not be evenly distributed across organizations. For an institution to be an active research participant using EHR data for measurement purposes, having developers capable of system redesign will be a requirement.
Another requirement is the ability to manage the burden of data entry. Entering standardized data in standardized fields enables the extraction of the data and comparison to data from other records and systems. The process of entering data in that way, however, is not intuitive or comfortable. Instead of jotting down shorthand notes and abbreviations on paper, frontline clinicians are now key participants in building a research infrastructure that does not benefit them or their patients in the near term, although it has significant potential societal benefits. A future challenge for the kind of work that PCORnet aspires to is how to build in that reward, whether intrinsic or extrinsic, for those who bear the often frustrating burden of data entry.
TAPPING NEW DATA SOURCES TO IMPROVE HEALTH QUALITY
Cassel discussed the potential of data infrastructures to serve as measurement frameworks, not only for research but for accountability at the national level. She reminded participants that the NQF was created 15 years ago to be the one organization where public and private stakeholders from every part of health care would come together to decide what is meaningful information, and how to conduct rigorous and accurate measurement using that information. The NQF was focused on getting information to the public with the idea that comparisons between providers would facilitate consumer-driven reductions in cost and increases in quality. However, the health care marketplace is not quite that simple and over the past decade there has been increasing focus on linking metrics to payment. It is not just the payers who are demanding these measures, she added, but also consumers who want to know, for example, which doctor or hospital is best.
The current measurement system does not serve any of its constituencies sufficiently. Cassel commented that there are too many measures, and it is too confusing, and there is now a push for developing a set of core measures. Several
sets of core measures have been proposed, including those described in the Institute of Medicine (IOM) report titled Vital Signs: Core Metrics for Health and Health Care Progress (IOM, 2015b). According to Cassel, while core measures as indicators of progress are essential, they will not necessarily meet the needs of consumers, who want disease- or provider-specific information, not public health measures. There is extensive information available to consumers on the Internet, with wide-ranging accuracy and value. There are also measures for value-based purchasing, which are systems-level measures (system-level accountability and system-level payment) rather than individual clinician measures. The NQF can help to meet the needs of payers, providers, and consumers, she suggested, if better measures can be agreed to.
There is also tension between the requirements by payers to ensure that their money is spent on value (i.e., accountability) and the burden of collecting the data for the many other reasons discussed, including improvement. Cassel relayed the case of a major health care organization that has 100 staff members dedicated solely to collecting the data that must be reported to Medicare. This is waste to the system as those measures are not clinically relevant, do not help the providers improve care, and are not meaningful to consumers because they are not made in real time and do not come out of the real experience of the data systems. Ideally, using the same data for quality and accountability metrics, and for system improvement, could reduce collection burdens and improve clinical relevance. Another need is real-time feedback to better understand the impact of the metrics that are part of accountability programs (e.g., more rapid information about impact on care, and unintended consequences).
This will be essential for improving the quality of the metrics and ensuring that the metrics are meaningful to both consumers and providers. Cassel noted that the NQF has been engaging specialty societies, including the American College of Physicians and the American College of Cardiology, on using their members as a means to obtain real-time feedback about their experience with NQF metrics.
In closing, Cassel described the NQF project called the Measure Incubator,9 aimed at addressing the fact that there is a market failure in developing good measures. While many organizations are developing measures, they are not brought through the national process to determine whether they could be used at a national level. There are also areas where there are few reliable measures (e.g., behavioral health, and care coordination for multiple chronic conditions) and
___________________
9 For more information see http://www.qualityforum.org/NQF_Measure_Incubator.aspx (accessed May 31, 2016).
cases where the development of measures has been inefficient (time consuming and costly). The Measure Incubator brings measure developers together with technical experts, funders, and data sources (e.g., large delivery systems, big data sources, crowdsourced data, and patient-reported outcomes). She suggested that as PCORnet moves forward, the network might also be used for developing meaningful quality measures.
ACCELERATING MEDICAL EVIDENCE GENERATION AND USE
A key concern for a CEO running a health system is how to operationalize knowledge to achieve better outcomes and ultimately improve population health. Benjamin Chu shared some of the lessons from Kaiser’s experience in putting data into action to improve outcomes over the past decade. According to the IOM report titled Best Care at Lower Cost: The Path to Continuously Learning Health Care in America (2013), one of the characteristics of a continuously learning health system is real-time access to knowledge. To achieve this, Chu explained that data need to be translated with intentionality into a systems-based approach. The process begins with defining a desired outcome, determining how to best obtain the real-time actionable data necessary to drive that outcome, and then developing the measures to obtain feedback and maintain an iterative approach to improvement. Transparency is a key element of the process. Aligned leadership direction is also essential, and leadership needs to galvanize the frontline staff, as they are the people collecting the data and implementing the change.
Chu briefly mentioned several specific examples of different approaches to achieving better outcomes (see Hudson et al., 2015; Kanter et al., 2010, 2013; Sim et al., 2014). One example included the creation of registries, but Chu cautioned that establishing a registry alone is not sufficient; there must be a system around the information in the registry that can drive better performance. Similarly, simply providing information about gaps in care to primary care doctors is not effective; there needs to be an intentional, systems-oriented approach that puts improvement strategies in place. A structural model is not going to drive improved care. Rather, it is the use of the information, and the feedback to push the systems. Diagnostic errors, or “diagnoses of omission,” are also a concern. Chu shared that health systems are receiving lawsuits from patients who were tested but for whom there was no timely follow-up on the test, and who 2 or 3 years later developed high-grade prostate cancer or colon cancer. In an integrated system, providers should be responsive;
however, patients are still falling through the cracks. To address this, Kaiser designed an electronic safety net system that uses an “if, then” hypothesis. If there was an abnormal result, then did a follow-up happen? Another example considered the use of high-cost imaging, including computed tomography and magnetic resonance imaging. Looking at comparative rates of diagnosis of diabetic retinopathy, Kaiser found wide variation in diagnosis via retinal screening across medical centers and discovered that interophthalmologist reliability was very poor. To address this, Kaiser implemented a teleophthalmology approach where trained technicians conduct centralized review of retinal screening images. As a result, diagnostic accuracy has increased and variability across centers has decreased.
DISCUSSION
During the open discussion that followed the presentations, participants discussed the burdens of data collection as well as the potential of having access to claims data and to bundling core measures data. Participants also discussed how to more effectively engage providers in implementation and the need to define measures for specific concerns, including measures of inequity within the system and measures of relative improvement.
Data Collection: Maximizing the Uses of Clinical and Claims Data
The challenges and burdens of data collection facing clinicians were discussed further. Cassel suggested that there is a significant burden on care systems to collect data that are used solely for payment purposes and that are not relevant to internal improvement and quality metrics. Many participants agreed. She added that payer and provider data complement each other and are a powerful combination. Chu suggested that an added value of having access to claims data in addition to clinical data is that care providers often cannot get the full picture of their patient’s care; they see only the hospital component, or the specialty care, or the emergency department visit. He proposed that PCORnet could look at all of the data sources and think about how to pool a population-level view for individuals in the population. This would be helpful to most health systems.
Bundling Core Measures
Participants raised the ideas of bundling core measures for the series of care events that a patient needs and integrating measurements across patients with multiple chronic conditions. The CMS Million Hearts Initiative was cited as an example of integrating data from EHRs with vital sign data and laboratory
data to estimate the risk of cardiovascular events and death across the management of hypertension, diabetes, and smoking. It will be important to develop measures that integrate the totality of care that patients are receiving and summarize their health in a meaningful way, including describing their risk for an event and the ability of a care system to influence that risk independent of what happens to the patient. Also mentioned was the potential of bundling for looking at the elements of care that cluster together from a payment perspective. As mentioned above, payer and care data together provide a richer data source, as each alone is a limited data set.
Engaging Providers in Implementing Change
It was mentioned earlier that one way to advance progress is to bring research much closer to clinical operations, embedded in the clinical delivery system. Implementation will be more timely and successful if clinical staff can be involved in the design and the interpretation of the interventions. In response to a question about dissemination and implementation, Chu stated that Kaiser faces challenges similar to those encountered by other systems. Practitioners want to verify that the strategy is right. One of the ways to get buy-in is to show providers the actual information and to have one of their colleagues present that information. For example, the implementation of centralized review of retinal screening images was guided by an ophthalmologist, and providers were shown the data on screening by each practice versus centralized screening.
Participants agreed with the need for transparency and noted that while comparative performance data may make providers feel uncomfortable initially, professionals care about patients and improving care and want to know how others have achieved better performance. Greater transparency could also lead to a learning network.
Another point was that during medical school and residency, not as much attention is given to the foundations of quality, system science, and safety science as is given to life sciences and social sciences. Participants suggested the need to define such additional competencies at the premedical and medical school levels and, because changing habits is difficult, incorporate change management courses in medical school as well.
Defining Specific Measures
Participants also suggested that measures not be thought of as discrete entities, but as more dynamic. For example, there is a need for a national conversation on a standardized way that systems should be measuring inequity. Cassel pointed out that the Medicare Access and CHIP Reauthorization Act (MACRA) of
2015 legislation includes a provision that rewards improvement (in addition to rewarding the attainment of certain levels of performance). There are ongoing internal discussions at CMS about what kind of improvement that would be, and how it would be measured. It is all specialty based, she added, and suggested it would make sense to think about it relative to clinical units or team-based models.
