I've often thought it would be helpful to see how I compare with other pediatricians on indicators of practice quality. So it was interesting to read this article in the New England Journal of Medicine. The author feels that the report card she receives does nothing but demoralize physicians as she had seen no improvement in her indicators over two years and felt that the same was true for many good physicians. Demoralization is likely if there was no input from clinicians as to what they would find useful. It would be doubly true if these report cards were publicly reported.
In contrast, the authors of this article in the same issue of the New England Journal take an opposing view. They feel that you can't manage what you can't measure. They have shown an overall improvement in process measures since reporting became a requirement of the American Joint Commission.
They go on to suggest criteria for process measures including:
- There is a strong evidence base showing that the care process leads to improved outcomes
- The measure accurately captures whether the evidence based care process was provided
- The measure addresses a process that has few interventing care processes that must occur before the improved outcome is realized
- Implementing the measure has little or no chance of inducing unintended adverse consequences.
It's hard to argue with these statements. There is no doubt that outcomes are a function of organizational structure and processes combined with individual competence. The literature suggests that the first two factors are the most important.
But let's consider the difference between the two articles. The first refers to reports on an individual physician's practice, the second on hospitals. It's fairly obvious that the hospital has much better control over patient behaviour/compliance than a physician looking after patients in a stand-alone office. There are a number of confounding variables including socio-economic status, culture, education, responsibility for children and so forth that an individual physician who is not gaming the system can not control. If we want to improve practice and not discourage physicians from taking high risk patients we must consider these factors.
Let's also consider the patient perspective. A diabetic patient probably doesn't care how often his or her hemoglobin A1c has been measured. Patients care about real life issues such as how healthy are they and is their health improving or declining. Those issues are not measured by indicators of process.
Going back to the issue of performance indicators for individual physicians, the problem in the first paper was not that there were performance indicators, but that they were the wrong ones. A managed care organization was trying to align the indicators used in hospital with those used in the ambulatory office.
So, I'm going to suggest some criteria for indicators that can be used successfully in physician offices:
- The indicators should measure outcomes, not processes
- The outcome measures should be risk adjusted by postal code
- There should be strong evidence suggesting that the outcomes can be influenced by active provider input
- The measures should be discipline specific
- The indicators should be easy to report - they should not involve review of paper records
- Physicians should be involved in selecting those indicators
- We should not report individual statistics publicly, but we should consider reporting the outcomes of networks of physicians.
Tackling these one by one, let's start with the suggestion that we measure outcomes. Why should we measure the number of times in a year that a patient's hemoglobin A1c is measured if that has no perceived value to a diabetic patient? Wouldn't it make more sense to measure the percentage of a physician's diabetic patients that progressed to renal dialysis? That's a number that the healthcare organization can readily access without increasing the burden on individual doctors.
Continuing with that item, there's no doubt that most of us will serve our patients better if we follow protocols and guidelines. It's quite possible however that there are other ways of achieving the same outcomes. Should we be discouraging our physicians from developing their own evidence and processes to improve outcomes? This is obviously a rhetorical question.
We don't want our physicians selecting their patients by risk, and we don't want to penalize physicians who work with high risk patients. Most health care organizations have the ability to stratify risk by postal code. Doing that would ensure that physicians looking after the riskiest patients would not be penalized and might actually be rewarded for their efforts.
Clearly there's no point in measuring an outcome that's inevitable. Cancer of the pancreas for example has a very low cure/remission rate. Most chronic diseases however can be influenced by care; we should be measuring those outcomes.
There's a difference on what a general surgeon can influence compared to what a general practitioner can influence - the GP has much more health outcome influence overall. So it makes sense to ensure that the indicators are discipline specific.
Toyota refers to the waste of human talent as one of the wastes that must be eliminated. It wastes a physician's time to review records for process measures. Outcome measures can and should be trackable through administrative data.
If you want commitment to a practice, you need engagement and participation. For this reason alone, physicians should be involved in selecting outcome measures. They are a highly educated and professionally motivated group. Treat them that way, and be surprised by the outcome.
To the last point, we are trying to improve overall performance, not punish physicians. Individual physicians should know how their numbers compare with those of the discipline's average. The physician network quality council should also know. On the whole however, it takes teams to achieve outcomes, and to promote teamwork, only the team's indicators should be made public.
As always, I look forward to your thoughts and comments.