Predictive performance of different bleeding risk scores in patients with atrial fibrillation and acute coronary syndrome or undergoing percutaneous coronary intervention

Abstract This study aims to evaluate the predictive values of the HAS-BLED, ORBIT, ATRIA, REACH, PARIS, and PRECISE-DAPT scores in patients with atrial fibrillation (AF) and acute coronary syndrome (ACS) or undergoing percutaneous coronary intervention (PCI) who received both anticoagulant and antiplatelet therapy. 930 patients were consecutively recruited and followed up for 1 year. The primary endpoints were BARC class ≥3 bleeding and BARC class ≥2 bleeding. BARC class ≥3 bleeding occurred in 36 patients(3.9%), while BARC class ≥2 bleeding was seen in 134 patients (14.4%). The predictive performance of the HAS-BLED score for BARC class ≥3 bleeding was unsatisfactory (c-statistic = 0.575). The discrimination of the ATRIA, ORBIT, PARIS, and PRECISE-DAPT scores was also low-to-moderate. The REACH score was useless in bleeding risk stratification for this population. Multivariable logistic regression indicated that previous bleeding events and hemoglobin were two independent predictors of BARC class ≥3 bleeding. Compared to the HAS-BLED score, the model constructed by previous bleeding events and hemoglobin displayed a significant improvement in bleeding risk prediction [c-statistics: 0.704 vs. 0.575 (p = .008), NRI = 0.662,IDI = 0.049]. In patients with AF and ACS or undergoing PCI who received anticoagulant+antiplatelet therapy, the HAS-BLED, ORBIT, ATRIA, REACH, PARIS, and PRECISE-DAPT scores displayed only low-to-moderate performance in predicting BARC class≥3 bleeding. Future studies are required to develop more reliable scoring systems for bleeding risk evaluation in this population.


Introduction
Coronary heart disease (CAD) and atrial fibrillation (AF) often coexist due to multiple common risk factors. It's estimated that AF occurs in 6%-21% of patients with CAD [1], while approximately 20%-30% of patients with AF are complicated by CAD [2,3]. Antiplatelet therapy (APT) is essential for patients with CAD to reduce ischemic events [4], while oral anticoagulants (OAC) should be indicated for patients with AF at high risk of thromboembolism [5]. In patients with AF and acute coronary syndrome (ACS) or undergoing percutaneous coronary intervention (PCI), the combination of OAC and APT could effectively reduce ischemic and thromboembolic events, but result in a significantly increased risk of bleeding complications [6][7][8]. Evaluation of bleeding risk is recommended by clinical guidelines to guide individualized antithrombotic regimens [6][7][8]. Numerous prediction scores have been proposed for assessing bleeding risk in AF patients receiving OAC (e.g. the HAS-BLED [9], ATRIA [10], and ORBIT scores [11]) and CAD patients receiving APT (e.g. the REACH [12], PARIS [13], and PRECISE-DAPT scores [14]) respectively. Nevertheless, no standardized tool has been developed for bleeding risk evaluation in patients with AF and ACS or undergoing PCI who received both OAC and APT. Based on expert consensus, present guidelines recommend the HAS-BLED score for bleeding risk evaluation in this population [6][7][8]. However, the predictive performance of the various aforementioned scores has not been validated in this population. Therefore, we undertook a single-central, observational, cohort study in China to validate and compare the predictive performance of the HAS-BLED, ATRIA, ORBIT, REACH, PARIS, and PRECISE-DAPT scores in patients with AF and ACS or undergoing PCI who received both OAC and APT. or antiplatelet agents, and life expectancy less than 12 months. The study was approved by the ethics committee of Fuwai Hospital and conformed to the Declaration of Helsinki. All patients have signed the informed consent for participation.

Baseline
Baseline information about demographics, medical histories, physical examinations, laboratory tests, and therapeutic regimens was obtained by interviewing the patients, consulting their physicians, and reviewing medical records. Based on the above data, bleeding risk scores were calculated for every patient according to their definitions [9][10][11][12][13][14]. All included patients received an oral anticoagulant (warfarin or direct oral anticoagulant) and at least one antiplatelet agent (P2Y12 receptor inhibitor and/or aspirin). The antithrombotic regimen, type of anticoagulant/antiplatelet agents, and duration of antiplatelet therapy were determined individually by the treating physicians according to clinical guidelines [6][7][8].

Follow-up and outcomes
Follow-up with a duration of 12 months was carried out by trained research personnel via telephone interview, outpatient visit, or delivery of medical records. When an adverse event is suspected, the project coordinators would try the best to obtain the relevant source documents. Any additional information needed, would be acquired by contacting the patient's physicians or relatives. The primary endpoints were defined according to the bleeding academic research consortium (BARC) criteria [15] as major bleeding (BARC 3a, 3b, 3c, or 5) and any bleeding (BARC 2, 3a, 3b, 3c, or 5). The secondary endpoints included the International Society on Thrombosis and Hemostasis criteria (ISTH) major bleeding and ISTH major or clinically relevant nonmajor bleeding [16], the Thrombolysis in Myocardial Infarction criteria (TIMI) major bleeding and TIMI major or minor bleeding [15,17], as well as major adverse cardiovascular events (MACE) during 1-year follow-up. MACE referred to a composite of all-cause death, stroke, non-central nervous system embolism, myocardial infarction, definite or probable stent thrombosis, and target vessel revascularization [18]. All outcomes were adjudicated according to standardized principles by an independent committee blinded to patients' clinical characteristics. Information on all clinical endpoints would be recorded in the source documentation and in the case report forms. All relevant medical information was forwarded to the adjudication committee for formal review. The adjudication committee was composed of experienced cardiologists in Fuwai Hospital who did not participate in this study as investigators. All endpoints were adjudicated by the consensus of the adjudication committee.

Statistical analysis
Continuous variables are presented as medians with 25th to 75th interquartile ranges and compared by Mann-Whitney U test for the data are not normally distributed. Categorical variables are presented as frequencies with percentages and compared by Pearson's χ2 test or Fisher's exact test. Bar graphs were drawn to visualize the distribution of each risk score in the patients. Bleeding risk scores were entered into the logistic regressions both as continuous and categorical variables to evaluate the association between the risk scores and 1-year outcomes. Receiver operating characteristic (ROC) curves were constructed to assess the ability of the risk scores in predicting bleeding complications. Univariable and multivariable logistic regression analyses were performed to identify independent determinants of bleeding events, while odd ratio (OR) and 95% confidence interval (CI) were calculated. Variables with a p-value <0.10 in the univariable models or clinically relevant with outcomes were entered into the multivariable analysis with the backward LR (likelihood ratio) method [19]. Discrimination of the risk models was evaluated by the c-statistics, net reclassification improvement (NRI), and integrated discrimination improvement (IDI). The Hosmer-Lemeshow test and calibration plots were utilized to assess the calibration of different scores. A p-value of <0.05 was defined as statistically significant. All statistical analyses were performed by R statistic for Windows 3.6.2 (R Core Team, Vienna, Austria) and MedCalc 18.2.1.0 (MedCalc Software, Belgium).

Results
From September 2016 to March 2020, a total of 930 patients with AF and ACS or undergoing PCI who received both OAC and APT were included in this study. The flow chart for subject selection is shown in Supplementary Figure S1. The baseline characteristics and treatments of patients with AF and ACS or undergoing PCI are summarized in Table I (13.4%). Compared to patients without bleeding events, patients with bleeding events tended to have lower body mass index (BMI) and hemoglobin levels (all p < .05). They were more likely to have previous bleeding events and OAC+APT discontinuation (p < .05). Besides, patients with BARC class ≥2 bleeding were more likely to have heart failure and increased N-terminal pro-B type natriuretic peptide (all p < .05). The ATRIA, ORBIT, PARIS, and PRECISE-DAPT scores were significantly higher in patients with bleeding endpoints (all p < .05). However, the HAS-BLED and REACH scores were comparable between patients with and without BARC class ≥3 bleeding (p > .05).
The overall distribution of each bleeding risk score is displayed in Figure 1. The distributions of the HAS-BLED score and the REACH score in our cohort were shifted to the higherrisk group, which were significantly different from the original derivation cohorts ( Figure 1A and D). On the other hand, the distributions of the ATRIA score and the ORBIT score in the present study were comparable to those in the original derivation cohorts, except for the absence of score 0 in the ORBIT score ( Figure 1B and C). As to the PARIS score and the PRECISE-DAPT score, the histograms in our cohort were slightly deviated upward, compared to those in the original studies ( Figure 1E and F).
Logistic regressions demonstrated that the ATRIA, ORBIT, PARIS, and PRECISE-DAPT scores were significantly associated with the incidences of BARC class ≥3 bleeding, BARC class ≥2 bleeding, ISTH major bleeding, ISTH major or clinically relevant nonmajor bleeding, and TIMI major or minor bleeding (Table  III). The HAS-BLED score was a predictor of BARC class ≥3 bleeding, BARC class ≥2 bleeding, TIMI major or minor bleeding, and MACE, as a continuous variable but not as a categorical variable. In the meantime, no significant relation existed between the REACH score and 1-year bleeding endpoints (Table III). The predictive values of different bleeding risk scores were displayed in Table IV. According to the ROC analysis (shown in Figure 2), the c-statistics of the HAS-BLED for predicting the 1-year incidences of BARC class ≥3 bleeding and BARC class ≥2 bleeding were only 0.575 (95%CI: 0.542-0.607, p = .175) and 0.556 (95% CI: 0.524-0.589, p = .024). The predictive values of the ATRIA,

Laboratory tests, median (IQR)
White blood cell count (*10 9  <0.001 Abbreviations: AF atrial fibrillation, ACS acute coronary syndrome, PCI percutaneous coronary intervention, IQR interquartile ranges, GFR glomerular filtration rate, NT-proBNP N-terminal pro-B type natriuretic peptide, OAC oral anticoagulants, APT antiplatelet therapy. ORBIT, PARIS, and PRECISE-DAPT scores for BARC class ≥3 bleeding were relatively superior to that of the HAS-BLED score but remained low-to-moderate. The performance of these scores for predicting BARC class ≥2 bleeding, ISTH major bleeding, ISTH major or clinically relevant nonmajor bleeding, and TIMI major or minor bleeding was even worse (  Figure S3). Univariable and multivariable logistic regression analyses were performed to identify independent predictors of BARC class ≥3 bleeding and BARC class ≥2 bleeding during 1-year follow-up. Age, sex, body mass index, smoking history, drinking history, hypertension, diabetes mellitus, heart failure, prior stroke/transient ischemic attack, peripheral arterial disease, previous thromboembolic events, previous bleeding events, estimated glomerular filtration rate (eGFR) <30 ml/min/1.73 m 2 , left ventricular ejection fraction, hemoglobin, N-terminal pro-B type natriuretic peptide, type of oral anticoagulants, triple antithrombotic therapy, and proton pump inhibitors were entered into the multivariable logistic regressions with backward LR method. Results of the final multivariable logistic regression models for the primary endpoints were presented in Table V

Discussion
In the present study, the HAS-BLED score was significantly associated with 1-year incidences of BARC class ≥3 bleeding and BARC class ≥2 bleeding, as a continuous variable but not as a categorical variable. However, the predictive performance of the HAS-BLED score was quite unsatisfactory. The ATRIA, ORBIT, PARIS, and PRECISE-DAPT scores were also predictors for BARC class ≥3 bleeding and BARC class ≥2 bleeding. But their discriminative capacities were only low-to-moderate. Previous bleeding events and hemoglobin levels were identified to be independent predictors of BARC class ≥3 bleeding during 1-year follow-up. The model constructed by the two predictors displayed better discrimination than the HAS-BLED score. The coexistence of CAD and AF is common and complicates the selection of antithrombotic therapy [1][2][3]. For patients with AF and ACS or undergoing PCI, the combination of OAC and APT is recommended by clinical guidelines, which results in a significantly increased risk of bleeding complications [6][7][8]. In the present study, the incidences of bleeding events were higher than that in the derivation cohorts of aforementioned risk scores [9][10][11][12][13][14], but similar to those in other studies enrolling patients with AF and ACS or undergoing PCI [20][21][22][23]. This discrepancy could be comprehensible considering the complicated conditions and multiple antithrombotic therapies in this population.
Evaluation of bleeding risk according to risk scores could assist in individualized antithrombotic regimen selection [6][7][8]. Despite abundant bleeding risk scores proposed for patients with AF [9][10][11] or CAD [12][13][14], no standardized tool for bleeding risk evaluation has been developed for patients with AF and ACS or undergoing PCI yet. Present guidelines recommended the HAS-  BLED score for bleeding risk assessment in this population, based on the experience in patients with AF [6][7][8]. However, the predictive performance of the HAS-BLED score and other bleeding risk scores has not been validated in patients with AF and ACS or undergoing PCI who were treated with both OAC and APT. Limited studies have evaluated the predictive values of the existing risk scores in this population but have come to controversial conclusions. Kiviniemi, et al. tried to compare the predictive performance of the HAS-BLED, ATRIA, mOBRI, and REACH scores in a cohort of European patients with AF undergoing PCI and found that these bleeding risk scores were all useless in these patients [24]. However, the antithrombotic regimens in this cohort were diverse and a proportion of patients only received APT without OAC. In addition, no patient has been treated with direct OAC, since the study was undertaken a decade ago. All of these factors limited the generalizability of its conclusions in current clinical practice. Another retrospective study consisting of 302 Japanese patients treated with both OAC and APT demonstrated that the HAS-BLED, ORBIT, and PRECISE-DAPT scores could predict TIMI major or minor bleeding better than the PARIS score [25]. But it should be noted that this study recruited not only patients with AF but also patients with other indications for OAC (such as post-cardiac surgery, apical aneurysm, and pulmonary embolism). Due to its diverse subject resources and small sample size, prospective studies with large sample sizes are required to provide more evidence for these patients. To the best of our knowledge, our study evaluated the predictive performance of different bleeding risk scores in Chinese patients with AF and ACS or undergoing PCI for the first time. The present study was a post hoc analysis of a prospective cohort study, which focused on patients with AF and ACS or undergoing PCI who received both OAC and APT. Meanwhile, given that nearly two-thirds of patients have been prescribed direct OAC, our study could provide more valuable evidence for bleeding risk evaluation in the current era. In this study, the distributions of most risk scores deviated upward compared to the original derivation cohorts, which indicated that our cohort was at relatively higher bleeding risk. Yoshida, et al. have tried to evaluate the predictive performance of the HAS-BLED, ORBIT, PRECISE-DAPT, and PARIS score in 302 patients taking an anticoagulant undergoing PCI and have also found that the distribution of all risk scores among this study population was much higher than those in the original articles [25]. The discrepancies in the distribution of bleeding risk scores might be multifactorial. The concurrence of AF and CAD is associated with elevated incidences of cardiovascular comorbidities, such as hypertension, heart failure, and stroke [6][7][8]. These comorbidities are important components of these risk scores. On the other hand, the HAS-BLED and REACH scores assign some points for multiple antithrombotic therapies, which were much more common in our cohort. These factors could inevitably result in higher bleeding risk scores. In addition, ethnic discrepancies might partially account for different distributions of the bleeding risk scores in our study [26,27]. The present study was undertaken in the Chinese population, while the original derivation cohorts of the aforementioned risk scores mainly recruited Western patients. The relatively higher bleeding risk in East Asians compared to Caucasians could give reasons for the distinctive distributions of bleeding risk scores in our cohort. Several studies aimed to validate the existing risk scores externally in the East Asian population have revealed similar trends in the distribution of the risk scores as well [24,25,28,29].
In the present study, the discriminative capacity of the HAS-BLED for BARC class ≥3 bleeding was relatively low. The predictive abilities of other bleeding scores were not ideal either. There might be several explanations for the unsatisfactory performance of these bleeding risk scores. First and foremost, the distinctive study population and the relatively higher incidence of bleeding events in the present study might be the key reasons. Our study was designed to recruit Chinese patients with AF and ACS or undergoing PCI who received both OAC and APT. Compared to OAC or APT alone, the combination of OAC and APT has been indicated to increase the bleeding risk significantly [6][7][8]. The available bleeding risk scores were initially developed for AF patients treated with OAC [9][10][11] or CAD patients treated with APT [12][13][14], who were distinct from the present cohort. In the meantime, compared to the original derivation cohorts, our cohort tended to have relatively higher rates of hypertension, stroke, and ACS. These differences indicated that patients in the present study had a higher bleeding risk profile. Costa, et al. have tried to validate the PRECISE-DAPT score in the PLATO cohort and found that the score might underestimate the bleeding risk [14,30]. In the PLATO study, only patients with ACS were recruited, leading to a relatively higher bleeding risk [30]. The differences in the study population and incidences of bleeding endpoints might partially account for the weakened predictive values of previously-established risk scores in external cohorts. Secondly, different definitions of bleeding endpoints adopted between our study and the original derivation studies could have an impact on the discriminative capacities of the bleeding risk scores. While the original studies of the HAS-BLED [9], ATRIA [10] and REACH scores [12] used their own definitions of bleeding, the bleeding endpoint in the ORBIT study were defined as ISTH major bleeding [11] and that in the PRECISE-DAPT trial was TIMI major or minor bleeding [14]. On the other hand, the PARIS study adopted BARC class 3 or 5 bleeding as the primary endpoint [13], which was identical with the present study. The heterogeneity of bleeding definitions could result in discrepancy and uncertainty in the assessment of bleeding outcomes even in the same population [15]. Diverse definitions of bleeding events between derivation and validation cohorts could cause inconsistent adjudication of bleeding endpoints, which inevitably lead to impaired predictive abilities of the risk scores in validation cohorts. In fact, the PARIS score, which adopted the same definition of bleeding endpoints with our cohort, has also displayed a slightly better predictive value compared to other risk scores in the present study. Last but not the least, there existed ethnic discrepancies in risk profiles between the East Asian and Caucasian populations. Contrary to the ischemic risk, the bleeding risk in East Asians was relatively higher than that in Caucasians, which was known as the "East Asian Paradox" [26,27]. Differences in demographics (e.g. lower BMI), comorbidities, and responses to antithrombotic therapy might account for this phenomenon [26,27]. Compared to Caucasians, the incidences of Helicobacter pylori infection, intracranial atherosclerosis, and post-stroke hemorrhagic transformation were relatively higher in East Asians, resulting in increased risks of gastrointestinal and intracranial hemorrhage [26,27,31]. The original studies of available bleeding risk scores were mainly undertaken in Western countries, while our study only recruited Chinese patients. The ethnic differences between the present cohort and the derivation cohorts would unavoidably influence the predictive performance of the existing bleeding risk scores. Given the unsatisfactory predictive performance of the existing scores in patients with AF and ACS or undergoing PCI, multivariable logistic regressions were undertaken to explore predictors of bleeding events in the present study. Several studies have identified a couple of variables significantly associated with bleeding complications in this population, such as previous bleeding, anemia, and renal insufficiency [32][33][34][35]. Our study has yielded similar results in regard to bleeding risk factors. According to previous studies, the relationship between BMI and bleeding events remains controversial. Elevated bleeding risk was detected not only in overweight patients, but also in underweight patients [36]. Accumulative studies have demonstrated that there might exist a U-shape association between BMI and bleeding risk [36]. Our study only enrolled Chinese patients, who were generally thinner than the Western population. Only 11.6% of the patients in the present cohort had BMI ≥30 kg/m 2 (obesity). The relatively lower BMI and homogeneous recommended dosage of antithrombotic drugs in our cohort might account for the significant association between lower BMI and increased BARC class ≥2 bleeding in this study [26,27]. In the present cohort, previous bleeding events and lower hemoglobin were indicated to be the only two independent predictors of BARC class ≥3 bleeding during 1-year follow-up. The model composed of the two variables displayed a significantly better discriminative capacity than the guidelinerecommended HAS-BLED score. However, due to the limited sample size and low event rates, practical scoring schemes have not been constructed in the present study. Considering the relatively high bleeding risk and the dilemma of antithrombotic therapy selection in patients with AF and ACS or undergoing PCI, standardized risk scores should be constructed by prospective studies with large sample sizes and rational design in the future.
Several limitations should be noted in this study. Firstly, the present study was observational and had its inherent defects. The choices of antithrombotic therapy have been entirely at the discretion of treating physicians. Their pre-judgment about the patients' ischemic and bleeding risk would inevitably influence the antithrombotic regimen selection. In the meantime, the study population was limited to patients receiving both OAC and APT. However, patients at high risk of bleeding were usually less likely to be prescribed concurrent anticoagulants and antiplatelet agents in routine clinical practice. In addition, the treatment compliance of patients during the course of this study could not be regulated. All of these might have an impact on the result of our study. Secondly, due to the single-center nature of our study, the management pattern of patients is relatively homogeneous. This would limit the generalizability of our conclusions to other populations with different management strategies. Our study enrolled patients with AF and ACS or undergoing PCI from September 2016 to March 2020. During the past decade, guideline-recommended antithrombotic regimens for this population have changed significantly [6][7][8]. Therefore, half of the patients in our cohort were treated with triple antithrombotic therapy at discharge. However, current guidelines tended to shorten the duration of triple antithrombotic therapy, or recommend dual antithrombotic therapy [6][7][8]. Future prospective studies might provide more valid evidence for contemporary patients managed according to current guidelines. Finally, due to the limited number of patients and lack of validation cohorts, we did not construct a detailed scoring scheme for bleeding risk assessment. Further studies with larger sample sizes and longer follow-up periods are needed to develop reliable bleeding risk scores for patients with AF and ACS or undergoing PCI who received both OAC and APT.

Conclusion
In patients with AF and ACS or undergoing PCI who received both OAC and APT, the HAS-BLED score underperformed in stratifying bleeding risk. The predictive values of the ATRIA, ORBIT, REACH, PARIS, and PRECISE-DAPT scores were lowto-moderate as well. Previous bleeding events and hemoglobin levels were identified to be independent predictors of 1-year BARC class ≥3 bleeding. Future studies with larger sample sizes are necessary to develop standardized bleeding risk scores for this population.