Using publicly reported hospital data to predict obstetric quality.

Abstract Purpose: To determine the association between obstetric outcomes and publicly reported hospital data on patient satisfaction, surgical quality measures and medical outcomes. Materials and methods: Hospitals in the Nationwide Inpatient Sample in 2011 were linked to Hospital Compare, a source of hospital data on patient satisfaction, quality and mortality for medical conditions. The risk-adjusted hospital-level rates of obstetric morbidity, episiotomy and lacerations were compared across the hospitals and reported as the absolute reduction in risk (ARR). Results: We identified 528 708 women. There was no association between any of the metrics and risk-adjusted obstetric morbidity (range −0.15% to 0.03% difference). Hospitals with a high mortality rate for pneumonia had a 0.38% (95% CI: 0.13% to 0.64%) higher rate of risk-adjusted third- and fourth-degree lacerations, while hospitals with a higher death rate for myocardial infarction had a −0.74% (95% CI, −1.34% to −0.14%) lower risk-adjusted episiotomy rate. The differences in the remainder of the publicly reported metrics and the risk adjusted rates of third- and fourth-degree lacerations and episiotomy were small and not statistically significant (p  > 0.05). Conclusion: There is little association between currently available, publically reported hospital data and obstetric quality. Obstetric-specific hospital measures of quality and satisfaction are needed.


Introduction
Medical consumers now routinely use publicly reported hospital data to guide decisions as to where to receive care [1,2]. Hospital-level data describing quality of care, patient satisfaction, structural measures of a hospital and actual patient outcomes are now widely available [2]. While there are a number of sources of hospital data, hospital compare, a web-based data source that reports hospital quality is one of the most frequently utilized consumer-oriented sites [2]. Hospital compare was created as collaboration between the Centers for Medicare and Medicaid Services and the Hospital Quality Alliance and provides a range of hospital quality metrics [2].
To date, most publicly reported hospital data has focused on process and outcome measures for common medical conditions and surgical procedures [3]. Currently, there is little publicly available data specific to obstetric quality and safety. Obstetric care is unique in that the outcomes of both the mother and child must be considered and the vast majority of parturients are healthy and have good outcomes [4]. While currently available measures may help to define overall hospital quality, it is unknown whether these measures can be used as a surrogate for high-quality obstetric care.
Despite the increased interest in improving maternal and neonatal outcomes, publicly available data on the quality of obstetric care within hospitals is largely lacking. We performed a population-based analysis to determine whether there were associations between obstetric outcomes and publicly reported, hospital-level patient satisfaction and quality and outcomes data for common medical and surgical procedures.

Data sources
We linked hospitals that provided obstetric care in the Nationwide Inpatient Sample in 2011 to Hospital Compare, a website that reports hospital-level patient satisfaction, perioperative surgical quality measures and mortality for medical conditions. Patient satisfaction and perioperative quality data collected from July, 2009 to June, 2010 was utilized, while the ascertainment period for mortality for medical conditions spans three years and included July 2007 to June 2010. Outcomes of obstetric patients reported in NIS were compared to hospital-level data reported in Hospital Compare. NIS is a representative dataset developed by the Agency for Healthcare Research and Quality that captures a representative sample of 20% of acute care hospitalizations in the United States. NIS collected data from eight million hospital stays from facilities across 45 states in 2010 [5]. The Columbia University Institutional Review Board deemed the study exempt.
Hospital Compare is a website maintained by the Centers for Medicare and Medicaid Services that publicly reports hospital data. Reported measures include hospital structural characteristics, process measures, outcomes and patient satisfaction [2]. Approximately 98% of hospitals report data to Hospital Compare [6]. We utilized hospital-level data on patient satisfaction, surgical quality and outcomes for medical conditions [6].
Patient satisfaction is most commonly measured using Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS). HCAHPS is a validated survey administered to patients to examine attitudes about their hospital experience [7]. The survey is administered to a random sample of patients within 2 days to 6 weeks after discharge and includes questions about patient demographics to perform risk adjustment [7,8]. The survey includes 27 questions on communication, responsiveness of the hospital staff, the hospital environment, management of pain and discharge planning. Two global satisfaction questions are included that evaluate the overall hospital experience (scored worst to best from 0 to 10) and whether the respondent would recommend the hospital (scored on 3 levels). The patient satisfaction component of our analysis utilized the two global satisfaction questions and we ranked hospitals based on their patient experience score and how often patients would recommend the facility to others [7].
Perioperative surgical quality was assessed using Surgical Care Improvement Project (SCIP) scores [9]. SCIP reports hospital-level compliance with perioperative process measures for at-risk surgical patients and is a Joint Commission core measure [9]. SCIP captures a number of measures, but we specifically focused on use of perioperative antibiotics (appropriate administration, appropriate discontinuation and correct drug choice) and venous thromboembolism (VTE) prophylaxis (ordered and used).
Lastly, we examined hospital-level mortality for three common medical conditions: acute myocardial infarction (AMI), heart failure (HF) and pneumonia (PN). The hospital-level rate of death within 30 days for patients admitted for each of these conditions is reported after riskstandardization [10].

Patients and hospitals
Using a previously described algorithm, we identified all women aged 18-50 who underwent a vaginal delivery, operative vaginal delivery or cesarean delivery (Supplemental Table 1) [11]. We included hospitals that performed a minimum of 50 deliveries in 2011. Patients were classified based on age (520, 20-24, 25-29, 30-34, !35 years), race (white, black, Hispanic, other/unknown), ZIP code income (low, medium low, medium high, high, unknown) and insurance status (Medicare, Medicaid, private, self-pay, other/unknown). Research literature was reviewed and conditions associated with increased morbidity were included in the analysis [12,13].
Each hospital included in the analysis was classified based on size (small, medium, large), location (urban, rural), region of the country (northeast, midwest, south, west), teaching status (teaching, nonteaching), ownership (government, private not for profit, private for profit, other) and delivery volume in 2011. In addition, the American Hospital Association (AHA) Annual Survey Database was used to capture a number of additional hospital characteristics including the presence of a neonatal intensive care unit (no, yes, unknown), number of obstetric beds (quintiles) and use of an electronic health record (no, partially implemented, fully implemented, unknown) [14]. Linkage was performed using the AHA identification number. As some states do not report AHA ID, linkage was only performed in those states providing the identification number for linkage.

Outcomes
Obstetric outcomes for this analysis included hospital-level rates of severe obstetric morbidity, episiotomy, and third and fourth degree lacerations, identified based on International Classification of Diseases, Ninth revision (ICD-9) coding. Severe obstetric morbidity was analyzed using criteria from the Centers for Disease Control and Prevention [15]. Episiotomy and third-and fourth-degree lacerations were captured using previously described methodology (Supplemental Table 1) [16,17]. Women who underwent cesarean delivery or had a diagnosis of shoulder dystocia or fetal distress were excluded from the episiotomy analyses given that these clinical scenarios are accepted indications for the procedure. Women who underwent cesarean delivery, had a multiple gestation, stillbirth or malpresentation were excluded from the analysis of third-and fourth-degree lacerations [16,17]. The risk-adjusted rate of each of the three outcomes was calculated for each of the hospitals included in the analysis. For each of three outcomes, we fit a separate mixed-effects logistic regression model. These models included all patient and hospital characteristics described earlier and utilize a hospital-specific ID as a random intercept term to account for hospital-level clustering. The hospital-level outcomes were estimated as the mean of the predicted patient-level probabilities for each outcome at a given hospital. The riskadjusted outcome rate was estimated by multiplying the observed-to-expected ratio of the outcome by the rate of the outcome for the entire cohort as previously described [3].

Statistical analysis
Data from Hospital Compare are displayed as hospital-level medians with interquartile ranges. Hospitals were grouped into quartiles based on their scores for overall patient satisfaction and the willingness of respondents to recommend the hospital to others. The clinical and demographic characteristics of patients in these hospitals were then compared across the quartiles using 2 tests and Kruskal-Wallis tests. The correlation between each hospital'.s global HCAHPS scores and hospital-level obstetric morbidity, episiotomy rate and rate of third-and fourth-degree lacerations were plotted and analyzed using Pearson'.s correlation coefficient.
Hospitals were ordered based on their scores for each of the patient satisfaction measures, SCIP metrics and mortality rates for the three medical conditions. The obstetric outcomes (obstetric morbidity, episiotomy rate and third-and fourthdegree laceration rates) were then compared between highand low-performing hospitals. Specifically, the obstetric outcomes were compared between hospitals that performed at the highest quartile (75th percentile) and those at the lowest quartile (25th percentile) for each measure of patient satisfaction (HCAHPS), perioperative quality (SCIP) and mortality for the medical conditions. These data are reported as the absolute risk reduction (ARR) and reflects the percentage difference in performance between high-and low-performing hospitals. A sensitivity analysis was performed in which hospitals at the lowest (10th percentile) and highest (90th percentile) decile were compared. All analyses were performed with SAS version 9.4 (SAS Institute Inc, Cary, NC).
All statistical tests were two-sided. A p value of 50.05 was considered statistically significant.

Results
In 2011, we identified 773 736 women age 18-50 who delivered at 613 hospitals with !50 deliveries in the United States. After exclusion of 230 (37.5%) hospitals that did not provide AHA identifiers and nine (1.5%) that could not be linked, a total of cohort of 528 708 women from 374 hospitals were included in the final cohort.
Hospitals were then classified based on their median overall satisfaction scores into quartiles ( Table 2). Patients treated at hospitals with the highest overall satisfaction ratings were more often older, white, privately insured and residents of neighborhoods with higher median income levels. Similarly, hospitals in the Midwest were more likely to receive the higher overall satisfaction ratings (p ¼ 0.053). High patient satisfaction hospitals were more likely to have a neonatal intensive care unit (p ¼ 0.01) and a higher number of obstetric beds (p ¼ 0.01). There was not a statistically significant association between hospital delivery volume and satisfaction ratings (p ¼ 0.36). Similar findings were noted when this analysis was repeated based on hospital rates of whether patients would definitely recommend a hospital (Supplemental Table 2).
The correlation plot between risk adjusted obstetric morbidity and high overall patient satisfaction is displayed in Figure 1(A). There was no correlation between obstetric morbidity and patient satisfaction (r¼ À0.01, p ¼ 0.85). Similar findings were noted when the correlation between patient satisfaction and hospital-level third-and fourth-degree laceration rates (r¼ À0.06, p ¼ 0.24) and episiotomy rates (r¼ À0.10, p ¼ 0.06) (Figures 1B and 1C).
Risk-adjusted obstetric outcomes (morbidity, third-and fourth-degree lacerations and episiotomy) were then compared between high (upper quartile) and low (lowest quartile) performing hospitals for each surgical process measure (Table 3). There was no association between high performance on any of the process metrics and risk-adjusted obstetric morbidity (range À0.15% to 0.03% difference) (p40.05 for all). Hospitals with a high mortality rate for pneumonia had a 0.38% (95% CI, 0.13% to 0.64%) higher rate of risk-adjusted third-and fourth-degree lacerations, while hospitals with a higher death rate for patients with myocardial infarction had a À0.74% (95% CI, À1.34% to À0.14%) lower risk-adjusted episiotomy rate. The differences in the remainder of the publicly reported metrics and the risk adjusted rates of thirdand fourth-degree lacerations and episiotomy were small and  (C) Correlation between hospital-level satisfaction and risk-adjusted episiotomy rates. Table 3. Absolute risk reduction in outcomes (morbidity, laceration, and episiotomy) between lowest performing (25th percentile and below) and highest performing (75th percentile and above) hospitals for hospital measures (HCAHPS, SCIP, and mortality measures).

Risk-adjusted morbidity
Risk-adjusted laceration not statistically significant (p40.05 for all). A sensitivity analysis in which hospitals in the highest and lowest deciles were compared revealed similar results (Supplemental Table  3).

Discussion
Our findings suggest that there is little association between currently available, publically reported hospital data and obstetric quality. Hospital-level patient satisfaction, metrics of surgical quality and mortality rates of common medical conditions are poor surrogates for obstetric quality. While the last decade has seen increased interest in the measurement of quality in obstetrics, ideal metrics for obstetrics remain uncertain [4]. The Joint Commission has developed five perinatal core measures including rates of elective delivery before 39 weeks, cesarean delivery, use of antenatal steroids, bloodstream infections in newborns and exclusive breast feeding, while the rates of cesarean delivery and of vaginal birth after cesarean delivery are both considered Inpatient Quality Indicators by the Agency for Healthcare Research and Quality [18,19] Although there is interest in collecting hospital-level obstetric quality data, to date, these measures are not widely reported to the public. 4 As a surrogate for hospital-level quality, we used widely reported metrics for common medical and surgical conditions and examined the association between these measures and three adverse outcomes. Overall, there was a minimal association between available hospital-level measures and obstetric outcomes. Similar findings were noted in a prior study that examined the association between medical and surgical quality indicators and outcomes for patients undergoing high-risk oncologic surgery [3]. Importantly, several studies have reported mixed results on whether hospital-level process measure adherence even correlates with the outcomes of the associated diseases [20][21][22][23][24]. One analysis of Hospital Compare data on process measure compliance for myocardial infarction, heart failure and pneumonia, noted only a modest association between the process metrics and risk-adjusted mortality for the diseases [20].
Along with the measurement of quality, evaluation of patient experience has become increasingly important [25,26]. Patient satisfaction is typically measured through patient survey after hospital discharge. These surveys measure the general hospital population and are not specific to obstetric discharges. However, unlike most medical and surgical patients, obstetric populations are unique in that the outcomes of the majority of women who deliver are excellent and a positive environment and experience are of great importance to mothers and their families [4]. Given these limitations, currently available patient satisfaction tools, such as HCAHPS, are unlikely to be an accurate reflection of the experience of obstetric patients.
We found no association between patient satisfaction and obstetric outcomes. Studies examining the association between patient satisfaction and quality and outcomes have reported variable findings [1,3,[27][28][29][30][31][32]. In a study that included over 2400 hospitals that reported patient satisfaction, Jha et al. noted that patient satisfaction scores were modestly higher in hospitals with high-quality ratings [26]. In contrast, some studies have found no association between patient satisfaction and outcomes after myocardial infarction and cancer-directed surgery [3,31]. Quality and satisfaction are both unique domains that are important to patients and the lack of association between the two in many studies is not necessarily surprising [33]. Importantly, not only is patient satisfaction publicly reported, it is also now tied to reimbursement through the value-based purchasing initiative [25].
We acknowledge a number of important limitations. First, the Nationwide Inpatient Sample only reports hospital identifiers for facilities in select states. As such, we were unable to link all hospitals to Hospital Quality and our findings cannot be generalized to all hospitals in the United States. Second, while we examined a number of obstetric outcomes that signify high-quality care, the ideal metrics of obstetric quality are evolving [4] A priori, we chose measures that could be reliably operationalized from administrative data. Some measures, such as laceration rates, may not be ideal metrics of quality. While we considered evaluating maternal mortality, the outcome is fortunately rare and difficult to model with adequate statistical powering. We acknowledge that our findings may differ for different obstetric outcomes. Lastly, while we adjusted for known confounders, as with any study of administrative data, we are unable to perform complete risk adjustment.
These findings have a number of important implications for patients. First, given the lack of association between hospital quality measures and obstetric outcomes, the public data, that is, currently available will be of limited value to women choosing a hospital for delivery. These findings strongly support the need for the development and reporting of quality measures unique to obstetrics. Second, similar to data on quality, patient satisfaction scores have limited applicability to young women seeking obstetric care. In addition to not specifically measuring obstetric populations, these scores are not associated with quality of care for pregnant women. Like the measurement of quality, specific measures of obstetric satisfaction would be of great value to medical consumers.