Bridging Spherical Black-Box Optimizers

When gradient information is unavailable, black-box optimization (BBO) methods provide a practical alternative. While Evolution Strategies (ES), Consensus-Based Optimization (CBO), Optimization via Integration (OVI), and related methods have each been studied independently, their connections remain underexplored. We unify these approaches within a common theoretical framework, revealing that they differ primarily in two design choices: fitness aggregation (controlling sharpness preference) and consensus scope (controlling modality). Leveraging these insights, we introduce hybrid optimizers that interpolate between existing methods. Our ES-OVI hybrid allows explicit control over the preference for flat minima, enabling a trade-off between performance and robustness in continuous control tasks. Our CBO-OVI hybrids combine the higher-dimensional efficiency of parametric methods with the multimodal capabilities of particle-based approaches, achieving competitive results on language model merging under limited evaluation budgets. We validate our methods on standard BBO benchmarks and higher-dimensional locomotion tasks, demonstrating that the hybrid methods can outperform their constituent algorithms.

Optimization is the general problem setting of trying to find the input to a function, for which this function is minimized. We consider a specific subfield of optimization called black-box optimization. In this setting, we can only access the value of a function at chosen points, but we do not have any other information such as the gradient of the function. There are many different such black box optimization methods, and found that some of them are connected to each other in unexpected ways. We report these connections and further show that they can be used to design new methods, which have desirable properties of either base method. We verify those properties in different experiments and show that the new methods are sometimes advantageous to existing methods. We hope that our findings are helpful for researchers to investigate the connections between existing methods further. We also hope that our methods can be used for practical applications.