Supplement to “Personalized Prediction and Sparsity Pursuit in Latent Factor Models”

implying that the minimizer âj = 0. The same holds for b̂i if âj = 0. This completes the proof. Proof of Lemma PP-3: Consider SVD of Θ as follows: Θ = UDV T , where D is a diagonal matrix whose ith diagonal element is the ith singular value of Θ, and U and V are the left and right orthogonal matrices for the SVD. Note that ‖A‖2 is rotation-invariant under any orthogonal-transformation. Hence it suffices to consider the constraint that AB = D, equivalently, 〈aj, bi〉 = 0 when j 6= i and 〈ai, bi〉 = σi; i = 1, · · · , K. By the triangular inequality,


Yunzhang Zhu, Xiaotong Shen and Changqing Ye
We provide supplementary technical proofs for "Personalized Prediction and Sparsity Pursuit in Latent Factor Models".We use the prefix "PP" when referring to lemmas, equations etc., in the former paper, as in equation (PP-1) or Lemma PP-1.Proof of Lemma PP-1 Note that Â BT = Ã BT .For the L 1 -penalty function, Proof of Lemma PP-2: The proof for convergence of the algorithms follows the same argument as in Theorem 3.1 of [3], thus is omitted.To show that ( ÂL 1 , BL 1 ) satisfies (PP-3), note that if bi = 0 then it follows from (PP-11) that for any a j i∈R j• l(r ji , x T j α + βT implying that the minimizer âj = 0.The same holds for bi if âj = 0.This completes the proof.
Proof of Lemma PP-3: Consider SVD of Θ as follows: Θ = U DV T , where D is a diagonal matrix whose ith diagonal element is the ith singular value of Θ, and U and V are the left and right orthogonal matrices for the SVD.Note that A 2 is rotation-invariant under any orthogonal-transformation. Hence it suffices to consider the constraint that AB T = D, equivalently, a j , b i = 0 when j = i and a i , implying that the minimal of (PP-5) is no smaller than that of (PP-18), where the equality holds when A = U D 1/2 and B = V D 1/2 .This establishes the equivalence.Finally, uniqueness of the solution of (PP-18) follows from strict convexity of (PP-18) in Θ.This completes the proof.
Proof of Theorem PP-1: The proof uses a large deviation probability inequality of [4] to treat one-sided regularized log-likelihood ratios.Let f (R, {z ji }, Θ) be the probability density of (R, First we bound the bracketing u Hellinger metric entropy H(u, F 0 (s)) [6].To define this quantity, for any u > 0, call a finite set of pairs of functions The bracketing Hellinger metric entropy of F, denoted by the function H(•, F), is defined by H(u, F) = logarithm of the cardinality of the u-bracketing (of F) of the smallest size.For Θ = AB T with (A, B) ∈ F 0 (s), let where • F * is the Frobenius-norm whose jith element is taken over sup ( Ã, B)∈B δ (A,B) , and the fact that AB T 2 F * ≤ A 2 F * B 2 F * has been used.By Lemma 1 of [5], it suffices to bound the entropy of B δ (A, B).Note that there are s nonzero elements of (A, B) with where To apply Theorem 1 of [4], we verify the required entropy condition there for F: where ψ 1 (x, s) = x 1/2 x H 1/2 (u, F 0 (s))du/x with x = (c 1 ε 2 + λ(s − 1)).Using the entropy bound in (2), we obtain for 0 < x ≤ 1.Here we only focus our attention to the case of x ≤ 1 because ( 2) is met automatically when x > 1.Note that ψ 1 (x, s) is nonincreasing in s for any fixed x > 0. Then sup {s≥s 0 } ψ 1 (ε, s) = ψ 1 (ε, s 0 ).Solving (2) yields that c 1 ε 2 + λ(s 0 − 1) = ε 2 0,|Ω| , hence that ε 2 = 1 2c 1 ε 2 0,|Ω| and λ(s 0 − 1) ≤ 1 2 ε 2 0,|Ω| .The result then follows from Theorem 1 of [4].This yields (PP-19), thus the rate of convergence in P when letting |Ω|, M, U → ∞.For the corresponding risk result, note that h(•, •) ≤ 1, and The desired result then follows.This completes the proof.Proof of Lemma PP-4: Using the SVD of Θ 0 , we obtain Θ 0 = Ā BT , where Ā and B are U × r 0 and M × r 0 .Assume, without loss of generality, that ĀT = ( Ā1 , Ā2 ) where Ā1 is a r 0 × r 0 nonsingular matrix.Now define This completes the proof.Proof of Theorem PP-2: The proof is essentially the same except the entropy calculations.
For the L 2 -method, H(u, F 2 (s)) can be bounded similarly.Note that for some constant c > 0. The rest of the proof proceeds as that of Theorem PP-1.This completes the proof.
Proof of Lemma PP-5: Note that s 1 ≥ s0 c min , where s0 is the number of nonzero element for the best L 1 -factorization.The result then follows from the fact that s0 ≥ s 0 by definition.Moreover, minimization of A 1 + B 1 subject to Θ 0 = AB implies that the elements of its minimizer A and B are bounded away from zero provided that those of Θ 0 .Similarly, s 2 ≥ s 1 c min can be proved.This completes the proof.

is the 2 -
metric entropy F s,L and inequality n m ≤ e n m m has been used, c.f., Theorem 2.6 of [1].