We consider the problem of identifying any k out of the best m arms in an n-armed stochastic multi-armed bandit. Framed in the PAC setting, this particular problem
generalises both the problem of
best subset selection'' (Kalyanakrishnan & Stone, 2010) and that of
selectingone out of the best m'' arms (Roy Chaudhuri & Kalyanakrishnan, 2017). In applications such as crowd-sourcing and drug-designing, identifying a single good solution is often not sufficient. Moreover, finding the best subset might be hard due to the presence of many indistinguishably close solutions. Our generalisation of identifying exactly k arms out of the best m, where 1 ≤ k ≤ m, serves as a more effective alternative. We present a lower bound on the worst-case sample complexity for general k, and
a fully sequential PAC algorithm, LUCB-k-m, which is more sample-efficient on easy instances.
Also, extending our analysis to infinite-armed bandits, we present a PAC algorithm that is independent of n,
which identifies an arm from the best ρ fraction of arms using at most an additive poly-log number of samples than
compared to the lower bound,
thereby improving over (Roy Chaudhuri & Kalyanakrishnan, 2017) and Aziz et al. (2018).
The problem of identifying k > 1 distinct arms from the best ρ fraction is not always well-defined;
for a special class of this problem, we present lower and upper bounds. Finally, through a reduction, we establish a relation between upper bounds for the
one out of the best ρ'' problem for infinite instances and theone out of the best m'' problem for finite instances. We conjecture that it is more efficient to solve ``small" finite instances using the latter formulation, rather than going through the former.
Arghya Roy Chaudhuri (Indian Institute of Technology Bombay)
Shivaram Kalyanakrishnan (IIT Bombay)
Related Events (a corresponding poster, oral, or spotlight)
2019 Poster: PAC Identification of Many Good Arms in Stochastic Multi-Armed Bandits »
Fri Jun 14th 01:30 -- 04:00 AM Room Pacific Ballroom