S-Interface (Previously OTACS)
softwareposted on 30.12.2018 by Pradeep Buddharaju, Ashik Khatri, Ioannis Pavlidis
Code as a research output can either be uploaded directly from your computer or through the code management system GitHub. Versioning of code repositories is supported.
The S-Interface (previously OTACS) has been built as a modular system in C#. It is a communal software development project, where anybody can contribute modules with specific functionality. These modules are called plug-ins and need to conform to certain programming guidelines that are spelled out in the Plug-in Tutorial. Functionally speaking, the plug-ins are of two major varieties: (a) sensor channels, and (b) algorithm implementations. In its current plug-in library, the S-Interface features three sensor acquisition channels and four algorithms. Specifically:
C1 - Thermal Facial Channel: Thermal imaging is increasingly used to extract sympathetic signals from the face. The S-Interface can link-up with thermal imaging sensors, including the ThermoVision SC-6000 cooled mid-wave infrared (MWIR) camera and the TAU 2 uncooled long-wave infrared (LWIR) camera from FLIR Systems, Wilsonville, Oregon. It also supports peripheral devices that are often used in combination with these imaging sensors, including the Model 2004 Differential Blackbody by Santa Barbara Infrared, Santa Barbara, California and the QPT-50 pan-tilt from MOOG, Northbrook, Illinois.
C2 - Visual Facial Channel: Visual imaging sensors are often used for facial recording, especially in behavioral experiments. The S-Interface can link-up with visual imaging sensors, including a FireWire monochrome zoom camera from Imaging Source, Charlotte, North Carolina and a Pro Webcam from Logitech, Newark, California.
C3 - Operational Theater Channel: Visual imaging sensors are also used for the recording of actions in the operational theater of the experiment.
A1 - Tissue Tracking Algorithm: This plug-in implements the thermophysiological tissue tracker reported by Zhou et al. in . On the initial frame, the user initiates the tracking algorithm by selecting the upper orbicularis oris portion of the perinasal region for extracting theperspiration signal or the nostril region for extracting the breathing signal. The tracker estimates the best matching block in every next frame of the thermal clip via spatio-temporal smoothing.
A2 - Perspiration Signal Extraction Algorithm. This plug-in works collaboratively with the tissue tracking plug-in to extract perspiration signals from facial thermal imagery; these signals are equivalent to EDA signals . In facial thermal imagery, activated perspiration pores appear as small ‘cold’ (dark) spots, amidst substantial background clutter. The latter is the thermo-physiological manifestation of the metabolic processes in the surrounding tissue. The A2 plug-in quantifies this spatial frequency pattern by extracting an energy signal E(k;j), indicative of perspiration activity in the tracked area of subject k, for experimental session j. This tracked area is typically the upper orbicularis oris of the perinasal area. The signal computation is based on the clinically validated morphological method reported by Shastri et al. in .
A3 - Breathing Signal Extraction Algorithm. This plug-in works collaboratively with the tissue tracking plug-in to extract breathing signals from facial thermal imagery; these signals are equivalent to thermistor signals. The A3 plug-in extracts the breathing signal B(k;j) via wavelet analysis in the tracked area of subject k, for experimental session j. This tracked area is typically the nostril area of the subject. The plug-in implements the method reported by Fei and Pavlidis in .
A4 - Low-Pass Signal Filtering Algorithm. This plug-in implements an FFT low-pass filter, which is useful in reducing noise in thermophysiological signals, particularly perinasal EDA signals extracted via A2. The specific filter is reported by Tsiamyrtzis et al. in .
. Zhou, Y., Tsiamyrtzis, P., Lindner, P., Timofeyev, I. & Pavlidis, I. Spatio- temporal smoothing as a basis for facial tissue tracking in thermal imaging. IEEE Transactions on Biomedical Engineering 60, 1280–1289 (2013).
. Shastri, D., Papadakis, M., Tsiamyrtzis, P., Bass, B. & Pavlidis, I. Perinasal imaging of physiological stress and its affective potential. IEEE Transactions on Affective Computing 3, 366–378 (2012).
. Fei, J. & Pavlidis, I. Thermistor at a distance: Unobtrusive measurement of breathing. IEEE Transactions on Biomedical Engineering 57, 988–998 (2010).
. Tsiamyrtzis, P., Dowdall, J., Shastri, D., Pavlidis, I.T., Frank, M.G. & Ekman, P. Imaging facial physiology for the detection of deceit. International Journal of Computer Vision 71, 197–214 (2007).