We thank the reviewers for their thoughtful comments, questions and suggestions.$ To R1: When we implement AT-LUCB, we actually pull one arm at a time by pulling the chosen two arms one after the other. No extra samples are given to AT-LUCB. Thus, the comparison is fair. We will make that clear in the final version. Note that line 6 of Algorithm 1 should still be 'max'. The idea is that in general we increase s by one but sometimes we can pass one or more stages without pulling an arm. To R3: You are correct; line 190-193 has a typo, it should say that the first term dominates the second in most practical settings. In line 406, using \delta_1 = n is being generous for the argument therein since the maximum gets smaller for smaller \delta_1. Thus, the same argument holds for \delta_1 < n. To R4: Regarding line ~190, see our response to R2. Thank you for noticing the issue with the reference. We accidentally omitted the references, so the final version will have it correctly. The conclusions drawn from our main results are discussed in the introduction (lines 140-225). We chose LUCB for its good theoretical guarantees, its popularity, and good practical performance. Our theory itself is not general enough to cover any algorithm for the top-m arms identification, but we believe that one can apply the same technique to other algorithms like UGapE algorithm by Gabillon et al., which is very similar to LUCB. Our algorithm, AT-LUCB, adapts the confidence bound parameter in a data-dependent manner, and this is what we mean by “using confidence information”. We will give some thoughts to other wording for the title. As to advantages of using confidence information, we show that AT-LUCB is as good or better than doubling SAR in theory, and in practice we show that it behaves more smoothly (no severe fluctuations). These two advantages will be emphasized in the introduction. Finally, you have suggested comparing AT-LUCB to LUCB. Good suggestion! We can include the comparison in the final version. * References Gabillon, V., Ghavamzadeh, M., and Lazaric, A. Best arm identification: A unified approach to fixed bud- get and fixed confidence. In Advances in Neural Information Processing Systems 25, pp. 3221–3229, 2012.