Robust tracking of respiratory rate in high-dynamic range scenes using mobile thermal imaging

The importance of monitoring respiration, one of the vital signs, has repeatedly been highlighted in medical treatments, healthcare and fitness sectors. Current ubiquitous measurement systems require to wear respiration belts or nasal probe to track respiration rates. At the same time, digital image sensor based photoplethysmography (PPG) requires support of ambient lighting sources, which does not work properly in dark places and under varied lighting conditions. Recent advancements in thermographic systems, shrinking their size, weight and cost, open new possibilities for creating smart-phone based respiration rate monitoring devices that do not suffer from lighting conditions. However, mobile thermal imaging is challenged in scenes with high thermal dynamic ranges (e.g. different ambient temperature distributions in inside and outside) and, as for PPG with noises amplified by combined motion artefacts and breathing dynamics, and with its low spatial resolution often leading to weak breathing signals. In this paper, we propose a novel robust respiration tracking method which compensates for the negative effects of variations of the ambient temperature and the artefacts and can accurately extract breathing rates from controlled respiration exercises in highly dynamic thermal scenes. The method introduces three main contributions. The first is a novel Optimal Quantization technique which adaptively constructs a color mapping of absolute temperature matrices. The second is Thermal Gradient Flow mainly based on the computation of thermal gradient magnitude maps in order to enhance accuracy of nostril region tracking. We also present a new concept of Thermal Voxel to amplify the quality of respiration signals compared to the traditional averaging method. We demonstrate the high robustness of our system in terms of nostril-region and respiration tracking by evaluating it during controlled respiration exercises in high thermal dynamic scenes (e.g. strong correlation (r=0.9983)). We also demonstrate how our algorithm outperformed standard algorithms in settings with different amount of human motion and thermal changes. Finally, we open the datasets collected for these studies (i.e. under both controlled and unconstrained real-world settings) to the community to foster work in this area.