The accuracy of clinical prediction of prognosis for patients admitted with sepsis to internal medicine departments.

Abstract Introduction Prognosis estimation offered by physicians for patients inflicted by sepsis on their admission to Internal Medicine (IM) departments is considered a challenge. Early prognosis estimation is critical and determines the intensity of treatment offered. The accuracy of prognosis estimation made by physicians has previously been investigated mainly among intensive care physicians and oncologists. Objective To ascertain the accuracy of prognosis prediction made by internists for septic patients on admission to IM departments. Methods Physicians were asked to estimate the prognosis of every patient identified to have sepsis on admission. Their intuitive assessment of prognosis was incorporated into the patients’ electronic medical record. Survival follow-up was recorded until death or for at least 2 years. Later we compared survival with physicians’ prognosis estimations. Results Prognosis estimation was recorded for 1,073 consecutive septic patients admitted throughout the years 2008–2009 to IM departments. The mean age of patients was 74.7 ± 16.1 years. A total of 42.4% were suspected to have pneumonia, and 65.4% died during a mean follow-up time of 661.1 ± 612.3 days. Almost half of the patients classified to have good prognosis survived compared to 14.9% and 4.9% of those with intermediate and bad prognosis estimation, respectively (P < 0.001). Conclusion Internists can discriminate well between septic patients with good, intermediate, and bad prognosis. Key Messages Many patients suffering from sepsis are treated at general medical departments. Good estimation of the severity of disease upon admission is critical for decision-making regarding treatment. Internists including junior staff could discriminate well between patients with different degrees of disease severity of sepsis.


Introduction
Many disease severity scoring systems now exist to help predict outcomes in various patient groups. These are mainly based on clinical parameters and are aimed to help stratify patients according to their risk for clinically significant outcomes. Prognostic models are primarily used for targeting specific populations in clinical trial settings or for decision-making related to health system policies and resource allocation (1,2).
The role of the physician's perspective in prognostication of a specific patient is of importance and has been investigated in several studies primarily focusing on seriously ill patients treated in intensive care units (ICU) (3,4) and on cancer patients (5,6). The physician's assessment provides valuable answers for patients and their families (7) and may determine the intensity of treatment or alternatively the decisions surrounding end-of-life care (8).
Survival prediction dilemmas are not exclusive for oncologists and ICU physicians. Internists face them commonly. In one study, the internist addressed the question, 'How long do I have to live?' an average of 10 times per year. It was also found that physicians feel poorly prepared for prognostication and find it stressful and difficult to assess. They also believe that patients expect too much certainty and might judge them adversely for prognostic errors (9).
Patients with the clinical syndrome of sepsis are commonly admitted to the Internal Medicine (IM) service. This group of patients is heterogeneous in many aspects including their outcomes. The mortality associated with sepsis differs with its severity stage (10). Early provision of prognostic information and shared decision-making, including clinician recommendations about appropriate treatments and goals of care, are recommended by the American College of Critical Care (11) and the Surviving Sepsis Campaign (12). Yet, it is difficult to make such estimations, particularly in the setting of the IM department admission, where less is known about the risks for poor outcomes. It is likely that accurate prediction in patients hospitalized with a common, serious, and treatable clinical condition such as infection could impact treatment choices and outcomes eventually.
In the present study, we investigated the accuracy of prognosis for septic patients admitted to IM departments, as determined by the admitting physician at the very early stage of physician-patient interaction in IM departments.

Methods
The study was conducted in a 110-bed division of IM in a 450-bed community-based university-affiliated hospital in Haifa, Israel, between 1 February 2008 and 30 April 2009. All enrolled patients were over 18, and had a presumed diagnosis compatible with sepsis at admission. No exclusion criteria were employed.

Data collection
The prospective collection of data through the electronic medical record (EMR) system has been previously described (13). In short, we developed a computerized database that was incorporated into our EMR system. The computerized system identified patients with presumed sepsis, based on the definition provided by the ACCP/SCCM Consensus Conference in 1991 (14), i.e. any patient admitted with suspected infection and at least two of the criteria of systemic inflammatory response syndrome (SIRS): 1) a temperature greater than 38 C or less than 36 C; 2) an elevated heart rate greater than 90 beats per minute; 3) tachypnea, manifested by a respiratory rate greater than 20 breaths per minute or hyperventilation, as indicated by a PaCO 2 of less than 32 mmHg; and 4) an alteration in the white blood cell count, such as a count greater than 12,000/mm 2 , a count less than 4000/mm 2 , or the presence of more than 10% immature neutrophils. In order to identify patients with suspected infection, the active diagnoses on admission were checked automatically against a previously defined list of infection-related diagnoses. This process was further accompanied by directed guidance to medical staff during the duration of study accrual and random manual check-up.
The system automatically collected demographic, clinical, and laboratory data. To complement these data, the physicians were prompted by the EMR system to fill a mandatory questionnaire that included structured input of supplementary data (available online). At the end of entering the data into the electronic questionnaire, the admitting physicians were asked for their overall assessment of prognosis for the patient (good, intermediate, or bad). The patient prognosis assessment was recorded while the physician did not use any scoring models for prognosis prediction, so it could only reflect the intuitive estimation made by the physician. Also, and in order to avoid any effect on treatment decision-making, the recorded estimation was saved in a hidden file. For the purpose of this analysis we retrospectively collected additional information on the physician's age and years of clinical experience (postgraduation) at the time of the encounter with the patient.

Follow-up
We recorded follow-up survival rates for at least 2 years or until the patient died if earlier. For patients who died, we recorded whether the death occurred during the index hospitalization or after discharge. For post-discharge death, data were extracted from our EMR system, which is supplied with death data from the Ministry of Interior records.

Statistical methods
We describe the characteristics of our study group by calculating mean, standard deviation, and range, and by presenting percentage of occurrences. The primary analysis was performed by comparing survival between patient cohorts grouped according to the physician prognosis using the log-rank statistics. Hazard ratios for survival, adjusted for significant patient covariates (based on a logistic regression model for survival) and physician's age and seniority, were populated. In-hospital, 30-day, and 60-day mortality rates of the different cohorts were compared with the Pearson chi-square test. All analyses were performed on IBM SPSS Statistics version 20 (IBM Corp., Armonk, NY, USA).
The study was approved by the Carmel Medical Center Institutional Review Board. The need for informed consent was waived.

Results
During a 15-month period 1073 of 7879 total admissions (13.6%) were of patients who met the criteria of sepsis on their admission to the general IM departments and were enrolled into the study. The mean age of our study group was 74.7 ± 16.2 years, with 49% of the study population being over 80 years of age and 11.7% over 90 years of age. Approximately 52% were males. Most patients (96.2%) were admitted through the emergency department, while the rest were transferred from other departments in the hospital. The suspected source of infection on admission was as follows: pneumonia (42.4%), urinary tract infection (26.7%), unknown source (15.0%), skin and soft tissue (6.2%), and other diagnoses in about 10%. At admission, 5.7% of the study cohort fulfilled the criteria for septic shock, and 10.8% for severe sepsis.  Figure 1). Survival according to sepsis severity also differed significantly (overall difference between the groups, P50.001), but the pairwise comparison between survival in patients with severe sepsis and septic shock was non-significant (P ¼ 0.239) (Table II).
We assessed the hazard ratios (HR) for death after adjustment for patient characteristics that were found to be significant predictors for survival (age, terminal disease, diabetes mellitus, dyspnea at admission, Glasgow coma scale, and systolic blood pressure), as well as physician's age and seniority level. The HRs for mortality were 2.89 (95% CI 2.17-3.85, P50.001) and 1.47 (95% CI 1.21-1.79, P50.001) for patients with bad or intermediate prognosis compared to patients with good prognosis, respectively. Figure 2 depicts the rates of in-hospital, 30-day, and 60-day mortality rates for patients according to their prognostic group. At all of these time points there was a significant difference between the prognostic categories.

Discussion
In this large, real-life, prospective cohort of patients admitted with sepsis to a community-based IM division, we have shown that the physician's rough prognostication into three major categories provides a robust differentiator between patient groups for overall longterm survival. The ability accurately to estimate the prognosis for septic patients in the context of the IM department at an early stage of treatment is crucial. It offers the physician an additional tool to support decision-making. At the same time, it serves as the basis for communication with patients and their families   (7), and may determine the intensity of treatment or alternatively the decisions surrounding end-of-life care (8).
Aging populations and scarce resources mean that many patients today are treated for serious acute conditions within IM departments. Despite being a host for the majority of septic patients, IM departments currently have no validated prediction tool specifically calibrated to allow accurate prognostication in IM patients with sepsis. In previous work we have shown that the areas under the receiver-operating curve (AUC) of other, more labor-intensive, clinical prediction scores do not perform very well in this setting. The AUC for the Simple Clinical Score (SCS), the Mortality in Emergency Department Sepsis (MEDS), the Rapid Emergency Medicine Score (REMS), and the Modified Early Warning Score (MEWS) were found to be 0.76-0.79, 0.73-0.75, 0.74-0.79, and 0.65-0.70, respectively (13). In the present study, comparing prognosis estimation made by physicians with survival predicted by the sepsis group classification that is widely used-i.e. sepsis, severe sepsis, and septic shock-we found physicians were more precise in discrimination between the three groups identified by each classification. Our study's rough prognosis estimation was not compared with the previously mentioned scoring systems as both represent scales with different resolutions for estimating survival. However, the intuitive clinical assessment of physicians may be used side by side with structured scoring systems as a complementary tool. The added value of integrating intuitive clinical assessment to existing scoring systems needs to be further determined.
Conflicting findings were seen regarding the physician's ability to predict outcomes in other clinical settings. It has been noted that oncologists seem to overestimate survival (3,4). Observational studies suggest, however, that ICU physicians discriminate between survivors and non-survivors more accurately than do scoring systems in the first 24 hours after admission (2). It has also been demonstrated that repeated assessment of prognosis 48 hours after admission does not alter the initial assessment significantly (15). Thus, despite being based on limited clinical, laboratory, and imaging data, prognostication during admission is likely to be constant. Our study shows that this initial assessment is valuable in predicting short-and long-term outcomes.
These findings are even more intriguing given the demographics of our cohort of patients, with a high percentage of elderly patients, and previous work that has shown that physicians are less accurate when predicting outcomes in this group (16). It is likely that a combination of clinical findings, physician's input, and factors associated with physician perceptions are all important to provide an accurate and robust tool to enhance decision-making at the bedside (17). However, given the findings of this analysis, in the setting of sepsis in IM departments, one might choose to follow the crude estimate of the treating physician in the face of limited resources and time. This is even further highlighted with the accuracy of the prognosis maintained after adjustment for patient characteristics that were found to be significant predictors for survival, as well as the physician's age and seniority level.
There are a few limitations to this study. Prognosis estimation is limited to only three classes of good, intermediate, and bad, without concrete and standardized definition. The prognosis of patients was assessed at admission to the IM department and not at the entrance to the hospital, which represents a time lag of several hours in which patients were likely stabilized and treated in the emergency department. Additionally, there was no way of controlling for the effect of the prognostication during admission on the initial choice of treatment. We believe that this limitation may have affected very early survival, but is unlikely to affect decisions made by the clinical teams later on during hospitalization (as the prognostication was not available to the teams), and its effect on longer-term outcomes is questionable. Finally, this study is limited in the fact that it was conducted in a single center. We believe, however, that its results can be generalized to similar settings of care.
The reported correlation between physicians' initial prognosis assessment and the patients' outcomes is important given the urgency for immediate decisionmaking which carries significant consequences. Our findings add a level of confidence in the clinical evaluation of patients with sepsis at admission.

Declaration of interest
The authors report no conflicts of interest.