Face Recognition and Recovery under Simultaneous Real-World Degradations

The usability of automated facial recognition techniques in the wild has been limited in scope due to the detrimental effects of acquisition quality of test data. Real-world
conditions which severely impede recognition performance include 3D pose variations caused by relative position and orientation of the acquisition sensor with respect to the
face, partial occlusions of the face caused by external objects or other facial components, illumination artifacts on the face caused by scene illuminants, and low-resolution images caused by distance of acquisition, sensor specification, and physical limitations. Multiples
of these degradations are often simultaneously observed, further degrading matching accuracy.
The focus of this thesis is to construct a single unified technique to address and overcome such cases and thereby enable robust face recognition. Our formulation demonstrates that each type of acquisition degradation can be reduced to a problem of missing
or corrupted data in a particular representation for the face. We propose a unique representation for faces which enables several of these real-world degradations to be interpreted
as a single unified weighted data problem. At the heart of our method is a spline-based multi-resolution technique for dense alignment of faces under a variety of conditions. The proposed model is consistent over these image variations, complete in its description of the face, and is constructed to deteriorate predictably in the presence of known degradations. We propose and evaluate two techniques to recover the underlying face information, and through them enable extraction of a compact discriminative feature vector, as well as synthesis of a complete 3D reconstruction of the face. These recovery techniques require the construction of linear face models from large amounts of high-dimensional, incomplete, corrupted training data. We address this model construction problem by suggesting two novel weighted variants to classical model learning techniques: we build on
Generalized Hebbian Analysis theory to construct a subspace from incomplete/weighted data, and adapt the popular K-SVD algorithm to construct a sparsity-inducing basis from the same. Both proposed methods are shown to be accurate, efficient, and suitable for several visual learning problems. We thoroughly analyze the performance of the proposed face representation and recovery
techniques by conducting several experiments on controlled as well as real-world datasets. Comparisons with previous work and popular commercial techniques demonstrate the superiority of our methods. We are also able to extend the capabilities of current commercial systems at certain challenging tasks by using our method as a facial
reconstruction pre-processing tool. The flexibility of our technique enables it to be implemented
in both fully- as well as semi-automated configurations.