Untrusted Predictions and Mean Estimation: Machine-Learning Primitives from Data-Dependent Perspectives

Machine learning has revolutionized the field of computer science in the recent years, yet its lack of rigorous, worst-case guarantees has raised various theoretical and practical concerns. The computer science community have thus shifted focus to data-dependent algorithm design and analysis, approaching the given algorithm problem with the specific instance in perspective, that can often times provide more fine-grained guarantees that better capture the non-worst-case patterns in real-world input that machine learning relies on. This thesis examines two classical problems from a data-dependent perspective: (1) Online optimization of covering and packing programs, augmented with untrusted predictions, and (2) instance-by-instance bounds on the hardness of mean estimation on the real line, accompanied by a novel beyond worst-case definition of optimality.

In the first part, we study learning-augmented algorithms in the context of online convex covering and concave packing optimization, utilizing untrusted data-dependent advice prudently to outperform classical counterparts reliably. We propose general-purpose frameworks for linear covering and concave packing, based on the simple idea of switching between candidate solutions from either the prediction or any classical online algorithm as a black-box subroutine. For convex covering where the switching strategy does not work, we extend the celebrated primal-dual framework, fine-tuning it to incorporate the external predictions. We show that our algorithmic frameworks beat classical impossibility results when the advice is accurate, while able to maintain robustness even if the advice is arbitrarily misleading.

In the second part, we examine the classical one-dimensional mean estimation problem, investigating whether it is possible to further our understanding of the estimation error landscape, beyond the worst-case sub-Gaussian rate. Our analyses show that in general the sub-Gaussian rate is in fact optimal on an instance-by-instance basis, and can only be outperformed if we make additional assumptions about the underlying distribution, such as symmetry. We formalize this notion of data-dependent optimality as neighborhood optimality, and provide tools to analyze estimators under this novel framework, establishing connections to robust mean estimation.