Poster

Familywise Error Rate Control by Interactive Unmasking

Boyan Duan · Aaditya Ramdas · Larry Wasserman

Virtual

Keywords: [ Learning Theory ] [ Other ]

[ Abstract ]
[ Slides
Tue 14 Jul 7 a.m. PDT — 7:45 a.m. PDT
Tue 14 Jul 6 p.m. PDT — 6:45 p.m. PDT

Abstract:

We propose a method for multiple hypothesis testing with familywise error rate (FWER) control, called the i-FWER test. Most testing methods are predefined algorithms that do not allow modifications after observing the data. However, in practice, analysts tend to choose a promising algorithm after observing the data; unfortunately, this violates the validity of the conclusion. The i-FWER test allows much flexibility: a human (or a computer program acting on the human's behalf) may adaptively guide the algorithm in a data-dependent manner. We prove that our test controls FWER if the analysts adhere to a particular protocol of masking and unmasking. We demonstrate via numerical experiments the power of our test under structured non-nulls, and then explore new forms of masking.

Chat is not available.