Penalized Versus Constrained Generalized Eigenvalue Problems
Irina Gaynanova
James G. Booth
Martin T. Wells
10.6084/m9.figshare.3159700
https://tandf.figshare.com/articles/dataset/Penalized_versus_constrained_generalized_eigenvalue_problems/3159700
<p>We investigate the difference between using an ℓ<sub>1</sub> penalty versus an ℓ<sub>1</sub> constraint in generalized eigenvalue problems arising in multivariate analysis. Our main finding is that the ℓ<sub>1</sub> penalty may fail to provide very sparse solutions; a severe disadvantage for variable selection that can be remedied by using an ℓ<sub>1</sub> constraint. Our claims are supported both by empirical evidence and theoretical analysis. Finally, we illustrate the advantages of the ℓ<sub>1</sub> constraint in the context of discriminant analysis and principal component analysis. Supplementary materials for this article are available online.</p>
2017-04-25 05:40:45
Discriminant analysis
Duality gap
Nonconvex optimization
Variable selection