Efficacy of vulnerability detection techniques in non-traditional devices

The United States government is mandated to use a risk management framework to assess its computing systems for cyber security. Part of this framework calls for vulnerability assessments on all government assets. The federal government has a large and diverse set of assets: Desktops, Laptops and servers in office building to integrated, purpose-built hybrid systems for warfighting platforms and space travel. Many of these systems employ a hybrid of technology commonly referred to as Platform Information Technology (PIT).

These PIT systems may have elements of traditional Information Technology infrastructure, but are limited in functionality, similar to industrial control systems and IoT (Internet of things) devices. To address the challenge of cyber-attacks, vulnerability assessments are one of the methods to evaluate a system for risk. These assessments can be automated through software tools or manually performed or a combination of both techniques.

The goal of this research was to quantify the efficacy of several methods – namely vulnerability assessments performed with software tools and those performed manually against published searchable databases. This study was a comparative analysis of vulnerability detection on non-traditional IT devices. The results revealed which methods, or combination of methods, have an advantage and to what degree.