Manuscript preprint: Head-to-head benchmark evaluation of the sensitivity and specificity of five immunoassays for SARS-CoV-2 serology on >1500 samples

Background

SARS-CoV-2 has caused a global pandemic in 2020. Testing is critical to mitigating public health and economic impacts. Serology is considered key to population-level surveillance and potentially individual-level risk assessment; however, immunoassay performance has not been compared on large, identical sample sets.

Methods

We performed a head-to-head evaluation of four major commercial SARS-CoV-2 antibody immunoassays (Abbott’s SARS-CoV-2 Immunoassay, DiaSorin’s LIAISON® SARS-CoV-2 S1/S2 IgG, Roche’s Elecsys® Anti-SARS-CoV-2, Siemens’ SARS-CoV-2 Total [COV2T]), and a 384-well ELISA (the Oxford immunoassay [OIA]). We derived sensitivity and specificity from 976 pre-pandemic blood samples and 536 blood samples from laboratory-confirmed SARS-CoV-2 infections ≥20 days post-symptom onset. Receiver operating characteristic (ROC) curves were used to evaluate assay thresholds. We also investigated longitudinal changes in antibody titres and their association with disease severity.

Findings

Sensitivity and specificity (95% CI) at the manufacturers’ thresholds were: Abbott 92.7% (90.2-94.8%) and 99.9% (99.4-100%); DiaSorin 95.0% (92.8-96.7%) and 98.7% (97.7-99.3%); the OIA 99.1% (97.8-99.7%) and 99.0% (98.1-99.5%); Roche 97.2% (95.4-98.4%) and 99.8% (99.3-100%); and Siemens 98.1% (96.6-99.1%) and 99.9% (99.4-100%). Threshold adjustment could optimise sensitivity and specificity; all assays achieved sensitivity of ≥98% with thresholds optimised to achieve specificity ≥98% on samples taken ≥30 days post-symptom onset. However, differences in assay performance translate into thousands of additional incorrect ascertainments between the worst and best platforms if millions of tests are performed.

Interpretation

Four commercial, widely available assays and a scalable 384-well ELISA can be used for SARS-CoV-2 serological testing to achieve sensitivity and specificity ≥98%; the Siemens assay and OIA achieved these metrics without further optimisation. This benchmark study in immunoassay evaluation should enable refinements of testing strategies and the best use of serological testing resource to benefit individuals and population health.

The National SARS-CoV-2 Serology Assay Evaluation Group comprises the following (alphabetical listing):

Mark Ainsworth, Monique Andersson, Kathryn Auckland, J. Kenneth Baillie, Eleanor Barnes, Sally Beer, Amy Beveridge, Sagida Bibi, Luke Blackwell, Martyna Borak, Abbie Bown, Tim Brooks, Nicola A. Burgess-Brown, Susana Camara, Matthew Catton, Kevin K. Chau, Thomas Christott, Elizabeth Clutterbuck, Jesse Coker, Richard Cornall, Stuart Cox, David Crawford-Jones, Derrick W. Crook, Silvia D’Arcangelo, Wanwisa Dejnirattisai, Julie Dequaire, Stavros Dimitriadis, Kate E. Dingle, George Doherty, Christina Dold, Tao Dong, Susanna J. Dunachie, Daniel Ebner, Marc Emmenegger, Alexis Espinosa, David W. Eyre, Rory Fairhead, Shayan Fassih, Conor Feehily, Sally Felle, Alejandra Fernandez-Cid, Maria Fernandez Mendoza, Thomas H. Foord, Thomas Fordwoh, Deborah Fox McKee, John Frater, Veronica Gallardo Sanchez, Nick Gent, Dominique Georgiou, Christopher J. Groves, Bassam Hallis, Peter Hammond, Stephanie B. Hatch, Heli J. Harvala, Jennifer Hill, Sarah J. Hoosdally, Bryn Horsington, Alison Howarth, Tim James, Katie Jeffery, Elizabeth Jones, Anita Justice, Fredrik Karpe, James Kavanagh, David Kim, Richard Kirton, Paul Klenerman, Julian C. Knight, Leonidas Koukouflis, Andrew Kwok, Ullrich Leuschner, Robert Levin, Aline Linder, Teresa Lockett, Sheila F. Lumley, Brian D. Marsden, Jose Martinez, Lucas Martins Ferreira, Lara Mason, Philippa C. Matthews, Alex J. Mentzer, Alexander Mobbs, Juthathip Mongkolsapaya, Jordan Morrow, Matthew Neville, Sarah Oakley, Marta Oliveira, Ashley Otter, Kevin Paddon, Jordan Pascoe, Yanchun Peng, Elena Perez, Prem Kumar Perumal, Tim E. A. Peto, Hayleagh Pickford, Rutger Ploeg, Andrew J. Pollard, Anastasia Richardson, Thomas Ritter, David J. Roberts, Gillian Rodger, Christine S. Rollier, Cathy Rowe, Justine K. Rudkin, Gavin Screaton, Malcolm G. Semple, Alex Sienkiewicz, Laura Silva-Reyes, Donal Skelly, Alberto Sobrino Diaz, Marinou Spyridoula, Lizzie Stafford, Lisa Stockdale, Nicole Stoesser, Teresa Street, David I. Stuart, Angela Sweed, Adan Taylor, Hannah Thraves, HoiPat Tsang, Marije Verheul, Richard Vipond, Timothy M. Walker, Sue Wareing, Yolanda Warren, Charlie Wells, Clare Wilson, Kate Withycombe, Rebecca K. Young.