This tutorial surveys work at a new frontier of machine learning, where each point in the hypothesis space corresponds to an algorithm, such as a combinatorial optimization problem solver. Much of this work falls under the umbrella of so-called \emph{algorithm configuration}; it also draws on methods from bandits, Bayesian optimization, reinforcement learning, and more. The tutorial will begin by explaining the area, describing some recent success stories, and giving a broad overview of related work from across the machine learning community and beyond. Then, we will focus on the algorithm configuration problem and how to solve it based on extensions of Bayesian optimization and bandits. We will also survey a wide range of other methods based on stochastic local search, algorithm portfolios, and more. Throughout, we will emphasize big picture ideas, motivational case studies, and core methodological innovations. We will conclude by surveying important open problems and exciting initial results from the broader community that offer potential ways forward.