Simultaneous Confidence Bounds for Aggregated Effects via Exact Subset Optimization

We study simultaneous confidence bounds for aggregated effects over downward-closed subset families of independent statistical tests. The bounds are obtained by bootstrap calibration of the maximum normalized aggregated effect over the relevant subset family, yielding valid post-hoc inference for data-selected subsets and tighter bounds than classical methods that protect all linear contrasts. A central challenge is that the required maximization is a nonlinear combinatorial optimization problem whose exact solution is essential for correct coverage. We address this challenge by casting the problem as a densest subgraph optimization and reformulating it as a linear program, or as a mixed-integer linear program when downward-closed linear constraints are imposed, enabling efficient and exact evaluation. We further characterize the growth regime of the number of tests for which the bootstrap calibration remains valid and illustrate the method on several machine learning applications.