Pediatric Congenital Cardiac Surgery in New York State, 1997-1999
RISK-ADJUSTMENT FOR ASSESSING PROVIDER PERFORMANCE
Hospital performance is an important factor that directly relates to patient outcomes. Whether patients recover quickly, experience complications, or die following a procedure is, in part, a result of the kind of medical care they receive. It is diffcult, however, to compare outcomes among hospitals when assessing performance because different hospitals treat different types of patients. Hospitals with sicker patients may have higher rates of complications and death than other hospitals in the state. The following describes how the New York State Department of Health adjusts for patient risk in assessing outcomes of care in different hospitals.
Data Collection, Data Validation and Identifying In-Hospital Deaths
As part of the risk-adjustment process, hospitals in New York State where pediatric cardiac surgery is performed provide information to the Department of Health for each patient undergoing those procedures. Each hospital's cardiac surgery department collects data concerning patients'demographic and clinical characteristics. Approximately 22 of these characteristics (or risk factors) are collected for each patient. These data are entered into a computer, and sent to the Department of Health for analysis, along with information about the hospital, physician, patient diagnosis on admission, procedure performed, and the patient's status at discharge. Data are verifed through the review of unusual reporting frequencies, cross-matching of pediatric cardiac surgery data with other Department of Health databases, and a review of medical records for a selected sample of cases. These activities are extremely helpful in ensuring consistent interpretation of data elements across hospitals.
The analysis is based on deaths occurring during the same hospital stay in which the patient underwent pediatric congenital cardiac surgery. In this report, an in-hospital death is defned as a patient who died subsequent to cardiac surgery during the same acute care admission.
Assessing Patient Risk
Each person who has a congenital heart defect has a unique history. A cardiac profle system has been developed to evaluate the risk of treatment for each individual patient based on his or her history, weighing the important health factors for that person based on the experiences of patients who have had similar health histories in recent years. All of the important risk factors for each patient are combined to create his or her risk profle.
Another important factor in the patient's risk profle is the diagnosis at admission. There are approximately 55 different diagnoses that are collected for pediatric patients, all of which have a varying degree of risk associated with them. To take this relative risk into account, a reference group consisting of lower risk diagnoses was created. This reference group (listed in Appendix 2) consists of all diagnoses that have a post-surgical mortality rate less than or equal to 2.00% . The remaining diagnoses are then compared individually to the reference group and their relative risk is used in addition to demographic and clinical factors to determine the patient's risk profle. For purposes of this report each patient has had only one of the listed diagnoses reported.
The statistical analyses conducted by the Department of Health consist of determining which of the risk factors and diagnoses collected are signifcantly related to in-hospital death. The signifcant risk factors and diagnoses are weighted and used to predict the chance that each patient will have of dying in the hospital given his or her specifc characteristics.
Predicting Patient Mortality Rates for Hospitals
The statistical methods used to predict mortality on the basis of the signifcant risk factors and diagnoses are tested to determine whether they are suffciently accurate in predicting mortality for patients who are extremely ill prior to admission as well as for patients who are relatively healthy. These tests have confrmed that the models are reasonably accurate in predicting how patients at all different risk levels will fare when undergoing pediatric congenital cardiac surgery.
The resulting rate is the predicted or expected mortality rate (EMR) and is an estimate of what the hospital's mortality rate would have been if the hospital's performance was identical to the State performance. EMR is therefore an indicator of patient severity of illness. A hospital's expected mortality rate is contrasted with its observed mortality rate (OMR), which is the number of pediatric congenital cardiac surgery patients who died in that hospital divided by the total number of pediatric congenital cardiac surgery cases in that hospital.
Computing the Risk-Adjusted Mortality Rate
The risk-adjusted mortality rate (RAMR) represents the best estimate, based on the associated statistical model, of what the hospital's mortality rate would have been if the hospital had a mix of patients identical to the statewide mix. Thus, the risk-adjusted mortality rate has, to the extent possible, ironed out differences among hospitals in patient severity of illness, since it arrives at a mortality rate for each hospital based on an identical group of patients.
To calculate the risk-adjusted mortality rate, the observed mortality rate is divided by the hospital's expected mortality rate. If the resulting ratio is larger than one, the hospital has a higher mortality rate than expected on the basis of the patient mix; if it is smaller than one, the hospital has a lower mortality rate than expected from its patient mix. The ratio is then multiplied by the overall statewide mortality rate (5.35 for 1997-1999) to obtain the hospital's risk-adjusted rate.
Interpreting the Risk-Adjusted Mortality Rate
If the risk-adjusted mortality rate is lower than the statewide mortality rate, the hospital has a better performance than the State as a whole; if the risk-adjusted mortality rate is higher than the statewide mortality rate, the hospital's performance is worse than the State as a whole. Signifcant differences, higher and lower, are identifed in Table 1 with one or two asterisk, respectively.
The risk-adjusted mortality rate is used in this report as a measure of the quality of care provided by hospitals. There are reasons that a provider's risk-adjusted rate may not be indicative of its true quality. However, we have developed mechanisms for limiting the impact of these issues.
For example, extreme outcome rates may occur due to chance alone. This is particularly true for low-volume providers, for whom very high or very low rates are more likely to occur than for high-volume providers. Expected ranges or confdence intervals are included as part of the reported results in an attempt to prevent misinterpretation of differences caused by chance variation.
Differences in hospital coding of risk factors could be an additional reason that a hospital's risk-adjusted mortality rate may not be refective of their quality of care. The Department of Health monitors the quality of coded data by reviewing patients'medical records to confrm the presence of key risk factors.
Some commentators have suggested that patient severity of illness may not be accurately estimated because some risk factors are not included in the data system, and this could lead to misleading risk-adjusted rates. This is not likely because the New York State data system has been reviewed by practicing physicians in the feld and is updated continually.
How This Contributes to Quality Improvement
The goal of the Department of Health and the Cardiac Advisory Committee is to improve the quality of care for pediatric patients with congenital cardiac anomalies in New York State. Providing hospitals in New York State with data about their own outcomes for patients with specifc congenital diagnoses allows them to examine the quality of the care provided for these patients and to identify opportunities to improve care.
The information collected and analyzed in this program is also given to the Cardiac Advisory Committee, which assists with interpretation and advises the Department of Health regarding hospitals that may need special attention. Committee members have also conducted site visits to particular hospitals, provided recommendations for improved care and, in some cases, have recommended that hospitals obtain expertise from outside consultants to design improvements for their programs.
