External national validation of the Leicester Self‐Assessment score for Type 2 diabetes using data from the English Longitudinal Study of Ageing

To validate the Leicester Self‐Assessment score using a representative English dataset for detecting prevalent non‐diabetic hyperglycaemia or undiagnosed Type 2 diabetes (defined as HbA1c ≥6.0%) and for identifying those who may go on to develop Type 2 diabetes within 10 years.


Introduction
It is estimated that 366 000 000 individuals have Type 2 diabetes mellitus, a number predicted to grow, on average, by 2.7% a year [1]. Non-diabetic hyperglycaemia, also known as 'prediabetes' or impaired glucose regulation (which includes impaired glucose tolerance and impaired fasting glucose), is a condition in which an individual's blood glucose is raised above normal levels but remains lower than the diabetes range [2]. Recently, the use of glycated haemoglobin (HbA 1c ) to establish non-diabetic hyperglycaemia has been widely supported [3][4][5], with the International Expert Committee and the UK-based National Institute for Health and Care Excellence (NICE) recommending that HbA 1c values of 42-46 mmol/mol (6.0-6.4%) be classified as non-diabetic hyperglycaemia [4,5]. It has been estimated that, in 2013, 6.9% of the world's adult population had non-diabetic hyperglycaemia, a figure also expected to rise [6]. Individuals with non-diabetic hyperglycaemia are at greater risk of progressing to Type 2 diabetes compared with individuals with normal glucose levels [7,8].
NICE recommend that individuals aged 40-75 years with non-diabetic hyperglycaemia or undiagnosed Type 2 diabetes be identified using a two-stage screening approach involving a non-invasive risk tool, ideally using a computer-based risk assessment tool that uses routinely collected data, followed by a blood test [5]. If a computer-based risk assessment tool is not available or opportunistic screening is being undertaken, NICE recommend using a validated self-assessment questionnaire, such as the Leicester Self-Assessment score. Proven lifestyle interventions to prevent or delay the onset of Type 2 diabetes should then be provided to those with nondiabetic hyperglycaemia [5]. Risk assessment tools use an individual's risk factors to calculate their probability of having a particular health outcome or to calculate a risk score based on this probability [9]. Risk assessment tools help to optimize the resources required for detecting diseases by allowing screening to be targeted at those at highest risk [10]. Risk assessment tools can be developed for a crosssectional outcome, such as non-diabetic hyperglycaemia or undiagnosed Type 2 diabetes [11], or for a prospective outcome, for instance, the development of Type 2 diabetes within the next 10 years [12,13]. The Leicester Self-Assessment score contains seven risk factors and was designed to be completed by members of the public for opportunistic screening. The score was developed to detect those with non-diabetic hyperglycaemia or undiagnosed Type 2 diabetes using a multi-ethnic dataset of people aged 40-75 years [14].
To date, the Leicester Self-Assessment score has only been validated for cross-sectional outcomes, predominately using Leicester-based data [2,14]. The present external validation used the English Longitudinal Study of Ageing (ELSA) [15] to validate the Leicester Self-Assessment score for the crosssectional outcome of non-diabetic hyperglycaemia or undiagnosed Type 2 diabetes by HbA 1c , and prospectively for the outcome of 10-year self-reported doctor-diagnosed diabetes. Finally, we evaluated the use of the two-stage screening programme as recommended by NICE [5], with the Leicester Self-Assessment score as the first stage and HbA 1c as the second stage, in identifying the individuals who prospectively develop Type 2 diabetes.

Data source
Data were taken from ELSA, an ongoing panel study of a nationally representative cohort of people aged ≥50 years living in England [15]. ELSA collected data from participants every 2 years; each round of data collection is known as a wave. The first wave of the study (conducted in 2002 and 2003) recruited individuals living in private housing aged ≥50 years and their partners, irrespective of age, from households in which an individual participated in the Health Survey for England in 1998, 1999 or 2001 and agreed to follow-up [16]. Refreshment samples of people aged 50-53 years were added to the study cohort in waves 3, 4 and 6, to ensure the cohort continued to be representative of individuals of this age. The study collects an extensive array of data including demographic, economic, social, psychological, mental and physical factors, along with various blood assays [16]. The socio-demographics of the ELSA dataset have been found to be broadly reflective of the English population [16]. Ethical approval for the study was granted by the NHS Research Committee and participants gave fully informed consent in writing [17].
Participants were asked to complete a questionnaire every wave, with nurse visits being conducted every other wave (every 4 years) to collect further information, such as blood glucose levels. The analyses in the present study used data from wave 2 (conducted in 2004 and 2005) as the baseline, as this is the first wave that included nurse visits and therefore blood glucose measurements. A total of 9432 individuals participated in wave 2 of the study [16].
ELSA was purposefully designed with the ability to study the prevalence of non-diabetic hyperglycaemia and/or undiagnosed diabetes at waves 2, 4 and 6, as well as the incidence of diagnosed diabetes at every wave [18]. Individuals were asked to give one fasting blood glucose measurement and one HbA 1c measurement every 4 years, during nurse visits. In the present study we use the HbA 1c data collected during wave 2 to define non-diabetic hyperglycaemia/undiagnosed diabetes [HbA 1c ≥42 mmol/mol (6.0%)] at baseline. Participants were also asked at each wave whether they had ever been told they had diabetes by a doctor and if they were taking insulin or medication for diabetes [18]. These data, up to wave 7, were used to define self-reported doctor-diagnosed Type 2 diabetes for the 10-year incidence data. Given the way these data were collected, we do not know which method was used to diagnose Type 2 diabetes for individual participants. ELSA is a freely available dataset accessed through the UK Data Archive [19].
Leicester Self-Assessment score Table 1 shows the risk factors included in the Leicester Self-Assessment score, along with the number of points assigned for each of the categories and the variables from ELSA used for each risk factor. Because there was no information on family history of diabetes in wave 2, we used data for this risk factor from waves 6 (2012/13) and 7 (2014/15), which assumes constant family history between 2004 and 2012. Additionally, the family history variable recorded in waves 6 and 7 was defined as whether an individual has/had a parent who has/had diabetes; this is a slight variation from the risk factor question used in the Leicester Self-Assessment score, which asks whether an individual has/had a first-degree What's new?
• We have externally validated the Leicester Self-Assessment score for use across England to detect those at high risk of undiagnosed Type 2 diabetes and nondiabetic hyperglycaemia.
• Of those found to be at high risk using the Leicester Self-Assessment score, 89% will go on to have Type 2 diabetes diagnosed within the next 10 years.
• The Leicester Self-Assessment score can be reliably used across England either to detect risk of current undiagnosed disease or to identify those at risk of developing diabetes in the future.
family member who has/had diabetes. All other risk factors of the score were recorded in wave 2 and their actual value from baseline could be used in the calculation of the risk score.
The total score is used to place individuals into four risk groups: low risk (0 to 6 points); medium risk (7 to 15 points); high risk (16 to 24 points); and very high risk (≥25 points) [14]. The use of a threshold of ≥16 for the Leicester Self-Assessment score is recommended for identifying individuals who are at high risk and require a blood test [14].

Study population and subsets for different analyses
The score was calculated for individuals in the eligible study population with the required data available. To be in the eligible study population individuals had to be aged 50-75 years, free of diagnosed diabetes at baseline and have complete data for the seven risk factors used to calculate the Leicester Self-Assessment score. Figure 1 shows that 3983 individuals in the population of interest had complete data for the seven risk factors of the risk score. To be included in the analysis for non-diabetic hyperglycaemia/ undiagnosed Type 2 diabetes [HbA 1c ≥42 mmol/mol (6.0%)] individuals also required a baseline HbA 1c measurement; whilst individuals were required to also have self-reported Type 2 diabetes status at 10 years to be included in the analysis of 10-year diabetes incidence. Finally, only individuals who had both a baseline HbA 1c measurement and self-reported Type 2 diabetes status at 10 years, in addition to having complete data for the seven risk factors, were included in the subset used to analyse the utility of a two-stage screening (risk score followed by HbA 1c ) for detecting the prospective outcome of 10-year diabetes incidence.

Statistical methods
Complete-case analyses were used for the main analysis. The area under the receiver-operator curve (AUROC) of the score, as well as the sensitivity, specificity, proportion classified as high risk, proportion correctly classified, positive predictive value and negative predictive value for the score's recommended threshold (≥16), were calculated for the following binary outcomes: baseline HbA 1c ≥42 mmol/mol (6.0%); i.e. the score's ability to detect those with undiagnosed current non-diabetic hyperglycaemia and Type 2 diabetes; and self-reported, doctor-diagnosed diabetes within 10 years. The prevalence of both outcomes was calculated across the Leicester Self-Assessment score's four risk groupings. Two sensitivity analyses were conducted which assessed the robustness of the main results to the level of missing: (1) replacing missing values with the lowest possible value and (2) replacing missing values with the highest possible value. We also used a threshold of ≥25, which is currently used to define very high risk.
To assess the validity of the score when used as part of the two-stage screening programme, as recommended by NICE [5], the proportion of individuals diagnosed with diabetes within 10 years was calculated for the following three groups: individuals with a score <16; individuals with a score

Results
As shown in Fig. 1, of the 9432 individuals who participated in wave 2, 3983 met the inclusion criteria for this study. Baseline HbA 1c was measured in 3203 (80.4%) of the eligible study population; while 3550 (89.1%) had diabetes status recorded at 10 years. When considering the two-stage screening process, 2866 individuals (72.0%) had both baseline HbA 1c and 10-year diabetes data available. Table S1 shows the reduction in study population attributable to individuals of interest (those aged 50-75 years and free from diagnosed diabetes) having missing data for the seven risk factors used to calculate the Leicester Self-Assessment score. Table S2 shows the characteristics of the eligible study population and the three subsets used for the different analyses. The majority of characteristics across the three subsets were very similar to those seen for the whole study population. Additionally, Table S2 includes the characteristics of all individuals aged 50-75 years and free from diagnosed diabetes, to check whether using a complete-case approach may have biased the results. The only noticeable differences between the fuller dataset and the eligible study population were higher proportions of individuals who currently smoked (17.6% vs 14.6%) and had a history of high blood pressure (38.5% vs 36.2%). Table S3 shows the proportion of individuals with each of the outcomes across the datasets being used. The proportion of both prevalent HbA 1c ≥6.0% (42 mmol/mol) and 10-year diabetes incidence were higher before individuals were removed for having missing Leicester Self-Assessment score data; for example, 10.9% of those aged 50-75 years and free of diabetes in wave 2 developed diabetes within 10 years, compared with 9.1% of the population with Leicester Self-Assessment score and 10-year diabetes status recorded.
Performance of Leicester Self-Assessment score Table 2 shows the Leicester Self-Assessment score had good levels of discrimination for both outcomes assessed (See Tables S4 and S5 for 292 contingency tables). Using the score's recommended threshold (≥16) resulted in roughly 61% of individuals being classified as high risk and requiring a blood test across all data subsets. At baseline, 7.7% of the population analysed had non-diabetic hyperglycaemia or undiagnosed Type 2 diabetes. Of these, 82.2% were correctly identified. At 10 years, 9.1% had developed diabetes, 89.2% of whom were correctly identified. Both outcomes were associated with a high negative predictive value, suggesting that the majority of individuals with a score of <16 do not have or go on to develop the disease. Figure 2 shows that the Leicester Self-Assessment score risk groupings discriminated well for the outcome of HbA 1c ≥42 mmol/mol (6.0%); the very high risk category had more than double the prevalence compared with the high risk category, which in turn had approximately twice the prevalence of the medium risk category. Figure 3 shows that the Leicester Self-Assessment score risk groupings also differentiated well for the outcome of diabetes at 10-year follow-up. The incidence of diabetes within 10 years in the very high risk category was over twice the incidence observed in the high risk category, more than seven times the incidence observed in the medium risk category and in excess of 20 times the incidence of the low risk category.
The sensitivity analyses assessing the robustness of these results to the missing data assumptions are shown in Table S6. Generally the results seen were stable in terms of the interpretation. For example, for detecting HbA 1c ≥42 mmol/mol (6.0%), the percentage of those classified at high risk slightly decreased when imputing the lowest values (53.6% vs 60.5%) and increased when the highest values were imputed (68.0%).
Increasing the threshold used for the risk score to define high risk to ≥25 gave a much lower sensitivity and higher specificity for both outcomes (Table S7), showing that the threshold of ≥16 is more suitable for identifying those requiring a blood test.
Performance of two-stage screening (risk score followed by HbA 1c test) Approximately four out of every 10 individuals, 1157 of 2866 (40.4%) in whom the two-stage screening was assessed, had a risk score <16 and therefore would not have required an HbA 1c test (Table S8). Of these, 27 (2.3%) went on to develop Type 2 diabetes within the next 10 years. Of the individuals with a risk score ≥16, 89.5% had a baseline HbA 1c <42 mmol/mol (6.0%) and would therefore have been identified as low risk after the blood test. Two-stage screening would have deemed the 6.6% of individuals with risk score ≥16 and HbA 1c ≥6.0% (42 mmol/mol) to be at high risk of developing diabetes. Figure 4 shows that the incidence of diabetes diagnosis within 10 years is considerably higher in individuals with a risk score ≥16 and HbA 1c ≥ 42 mmol/mol (6.0%) at baseline, with 54.8% being diagnosed with diabetes within 10 years. This group contained 43.8% of the cases of diabetes diagnosed within 10 years, Table 2 Prevalence, AUROC of Leicester Self-Assessment score and other statistical metrics of using Leicester Self-Assessment score with a threshold of ≥16 for outcomes of baseline HbA 1c ≥42 mmol/mol (6.0%) and doctor-diagnosed diabetes within 10   despite only having been 6.6% of the population. One hundred and five (6.9%) of the 1521 individuals with a risk score of ≥16, but a low HbA 1c (<6.0%) went on to develop Type 2 diabetes over 10 years.

Discussion
We report the first prospective external validation using a nationally representative dataset of the Leicester Self-Assessment score. We have shown that this score discriminates well in this population for both baseline abnormal HbA 1c and diabetes diagnosis in the years that follow. The recommended threshold of ≥16 leads to very few people who will be diagnosed with diabetes in the future being missed; however, many individuals who were not diagnosed with diabetes within 10 years were identified as high risk. We believe this is acceptable because the risk score is intended to be the first stage of a two-stage screening programme, thus it should be viewed as reducing the number of individuals requiring a blood test. Although, given the cost of testing HbA 1c (currently approximately £7.70 within the NHS) a full cost-benefit analysis is warranted [20]. Two-stage screening with the Leicester Self-Assessment score as the first stage and HbA 1c as the second, results in a small proportion (6.6%) of the population with a substantially increased risk of developing diabetes in the years that follow being identified.
A key strength of this validation is that it used the ELSA cohort, which was collected with the aim of being nationally representative and thus provides a national external validation in 50-75-year-olds. Another strength is that, in addition to assessing the Leicester Self-Assessment score alone for detecting the outcomes, the Leicester Self-Assessment score is evaluated as part of the two-stage screening process in which it is intended to be used; none of the other validations to date have assessed this. This validation did not include analyses of  the calibration of the Leicester Self-Assessment score for different outcomes, as probabilities for the different scores have not been published and are not used in practice. This is because the Leicester Self-Assessment score was developed to be used as a paper-based tool, so a simple scoring system without probabilities was chosen to allow ease of completion for members of the public [14].
First-degree family history of diabetes was not collected at baseline (wave 2) in the ELSA data and therefore parents' history of diabetes was back imputed from the 8-year followup. Given the age range of the participants, we assumed that family history would be relatively stable over time and therefore we believe this imputation will not have overly adversely affected the results [21]. Family history data were also not available in the Health Survey for England (HSE) data, which were used for another external validation of the Leicester Self-Assessment score. In the HSE validation, data were not imputed and this therefore led to the risk of nondiabetic hyperglycaemia /Type 2 diabetes being underestimated for those with a family history of diabetes [2]. Another limitation is the amount of missing data; the subsets used for the analyses carried out were considerably smaller, approximately half the size, of the total number of individuals in the dataset who were aged 50-75 years and free from diagnosed diabetes at baseline. The small decreases in the outcomes for the subsets analysed indicate the individuals excluded for having missing data for at least one of the Leicester Self-Assessment score risk factors were slightly more likely to have each of the outcomes; however, the majority of characteristics were similar in the three subsets used for the analyses to those observed for all individuals aged 50-75 years and free from diagnosed diabetes at baseline. When we repeated the analysis imputing missing items, the results were robust to the assumptions made.
Numerous risk assessment tools have been developed for non-diabetic hyperglycaemia and/or diabetes outcomes [22,23]; however, many of these have not been externally validated and few are implemented in practice [9,11]. Implementation of the Leicester Self-Assessment score has been recommended by NICE and the score has been widely used; it is available across the UK in national pharmacy chains and online at the Diabetes UK website [24]. This is the first prospective validation of the Leicester Self-Assessment score for future diabetes diagnosis.
Three previous cross-sectional validations have been published (one internal, using the development dataset, and two external) [2,14]. This cross-sectional validation for HbA 1c ≥ 42 mmol/mol (6.0%) had similar levels of discrimination (AUROC 69%) to those seen in the development dataset for the outcome of non-diabetic hyperglycaemia or undiagnosed Type 2 diabetes (both defined using oral glucose tolerance test, AUROC 69%). Higher levels were seen in an external validation using the HSE dataset for the outcome of nondiabetic hyperglycaemia alone defined by HbA 1c (AUROC 78%) [2]. The HSE dataset included individuals aged >16 years, leading to a much lower percentage (28%) with a score ≥16 compared with that found in this study population (61%).
To date, four risk scores have been developed for use in the UK, but the Leicester Self-Assessment score is the only score developed for self-completion by members of the public. The three other scores were all developed for use in primary care using data from electronic medical records: the Leicester Practice risk score; the QDScore [13]; and the Cambridge risk score [26]. Of these, the most widely validated is the QDScore. This score estimates an individual's 10-year risk of developing Type 2 diabetes. The QDScore has been externally validated using the Health Improvement Network dataset, with an AUROC of >80% for both men and women [27]. Although this is higher than the 74.9% observed in the present study for the 10-year progression to diabetes, this is not surprising given the QDScore was developed for this outcome and has greater sensitivity as a result of the inclusion of continuous independent variables in its risk equation. Given the Leicester Self-Assessment score was developed to be completed by hand, all continuous risk factors have been collapsed into categorical variables, which leads to a loss of sensitivity [28]. A comparison of the four UK scores using the HSE dataset found that all scores gave an acceptable level of sensitivity and specificity for detecting those at high risk of non-diabetic hyperglycaemia (Leicester Self-Assessment score 77.9% and 66.1%, Leicester Practice risk score 79.7% and 66.8%, QDScore 77.6% and 65.6%, Cambridge risk score 70.3% and 68.9%, respectively) [2].
The prevalence of individuals with either non-diabetic hyperglycaemia or undiagnosed Type 2 diabetes at baseline, 7.7%, was considerably lower than in other cross-sectional validations of the Leicester Self-Assessment score [2,14]. The data used to develop the score had a non-diabetic hyperglycaemia or undiagnosed Type 2 diabetes prevalence of 19.5%, although non-diabetic hyperglycaemia was defined using fasting plasma glucose instead of HbA 1c . Furthermore, the development dataset had a higher proportion of people from black and ethnic minorities, who have been found to have increased prevalence of the outcome. Prevalence affects the positive predictive value, which means that the risk score will have a different positive predictive value depending on the population it is applied to. For example, in the present study, we observed a positive predictive value of 10.5%, with a prevalence of 7.7%. If the prevalence is doubled, but holding the sensitivity and specificity at the same level, the positive predictive value increases to 20.3%. This explains why a lower positive predictive value was observed in the present data compared with that observed in the development data for the recommended threshold of ≥16. This suggests that the biggest gains in using the score are seen as the baseline risk of the population increases.
In conclusion, the present study assessed the prospective validity of the Leicester Self-Assessment score in a nationally representative dataset of individuals aged 50-75 years. The Leicester Self-Assessment score is a useful tool for identifying individuals with prevalent non-diabetic hyperglycaemia or undiagnosed Type 2 diabetes, defined by HbA 1c . In addition, the score discriminated well the individuals who go on to develop diabetes. Using a two-stage screening process that classifies individuals with a risk score ≥16 and HbA 1c ≥6.0% (42 mmol/mol) as high risk is an effective way of identifying a small proportion of individuals in whom a large proportion will be diagnosed with diabetes in the near future. This twostage screening process could be implemented in the NHS Diabetes Prevention Programme to identify individuals to whom intensive lifestyle inventions should be offered.

Funding sources
S.R.B. was supported by a PhD stipend from the University of Leicester.

Competing interests
S.R.B. declares no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years, and no other relationships or activities that could appear to have influenced the submitted work. N.N.D. declares no support from any organization for the submitted work, no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years, and no other relationships or activities that could appear to have influenced the submitted work. L.J.G. declares no support from any organization for the submitted work and no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years. M.J.D. and K.K. declare no support from any organization for the submitted work and no financial relationships with any organizations that might have an interest in the submitted work in the previous three years. M.J.D. and K.K. (Chair) were members of the NICE PH 38 (Preventing type 2 diabetes: risk identification and interventions for individuals at high risk) Programme Development Group. M.J.D., K.K. and L.J.G. were authors on the paper which originally developed and validated the Leicester Self-Assessment score.

Table S1
Completeness of seven ELSA variables used to calculate LSA in all 50-75-year-olds free from diagnosed diabetes in wave 2. Values are numbers (percentages). Table S2 Baseline characteristics of all individuals of interest in the dataset, the study population and the subsets analysed. Values are mean (SD) unless stated. Table S3. Proportion of individuals with each outcome in all individuals of interest in the dataset, the study population and the subsets analysed. Values are % (95% CI). Table S4. Baseline glucose status compared to LSA screening status. Table S5. Diabetes status at ten years compared to LSA screening status at baseline. Table S6. Sensivity analyses of the Prevalence, AUROC of Leicester Self-Assessment score and other statistical metrics of using Leicester Self-Assessment score with cut-point ≥16 for outcomes of baseline HbA1c ≥ 6.0% (42 mmol/mol) and doctor diagnosed diabetes within 10 years in the ELSA dataset. Values are percentages (95% CI). Table S7. Statistical metrics of using Leicester Self-Assessment score with cut-point ≥25 for outcomes of baseline HbA1c ≥ 6.0% (42 mmol/mol) and doctor diagnosed diabetes within 10 years in the ELSA dataset. Values are percentages (95% CI). Table S8. Diabetes status at 10 years compared to two-stage screening status at baseline.