# Effective Dimension of Exp-concave Optimization.

We investigate the role of the effective (a.k.a. statistical) dimension in determining both the statistical and the computational costs associated with exp-concave stochastic minimization. We derive sample complexity bounds that scale with $\frac{d_{\lambda}}{\epsilon}$, where $d_{\lambda}$ is the effective dimension associated with the regularization parameter $\lambda$. These are the first fast rates in this setting that do not exhibit any explicit dependence either on the intrinsic dimension or the $\ell_{2}$-norm of the optimal classifier.

We also propose fast preconditioned methods that solve the ERM problem in time $\tilde{O} \left(nnz(X)+\min_{\lambda'\ge\lambda}\frac{\lambda'}{\lambda}~d_{\lambda'}^{2}d \right)$, where $nnz(X)$ is the number of nonzero entries in the data. Our analysis emphasizes interesting connections between leverage scores, algorithmic stability and regularization. In particular, our algorithm involves a novel technique for choosing a regularization parameter $\lambda'$ that minimizes the complexity bound $\frac{\lambda'}{\lambda}\,d_{\lambda'}^{2}d$, while avoiding the entire (approximate) computation of the effective dimension for each candidate $\lambda'$. All of our result extend to the kernel setting.

Publisher URL: http://arxiv.org/abs/1805.08268

DOI: arXiv:1805.08268v2

Keeping up-to-date with research can feel impossible, with papers being published faster than you'll ever be able to read them. That's where Researcher comes in: we're simplifying discovery and making important discussions happen. With over 19,000 sources, including peer-reviewed journals, preprints, blogs, universities, podcasts and Live events across 10 research areas, you'll never miss what's important to you. It's like social media, but better. Oh, and we should mention - it's free.

Researcher displays publicly available abstracts and doesnâ€™t host any full article content. If the content is open access, we will direct clicks from the abstracts to the publisher website and display the PDF copy on our platform. Clicks to view the full text will be directed to the publisher website, where only users with subscriptions or access through their institution are able to view the full article.