While crowdsourcing has been a cost and time efficient method to label massive samples, one critical issue is quality control, for which the key challenge is to infer the ground truth from noisy or even adversarial data by various users. A large class of crowdsourcing problems, such as those involving age, grade, level, or stage, have an ordinal structure in their labels. Based on a technique of sampling estimated label from the posterior distribution, we define a novel separating width among the labeled observations to characterize the quality of sampled labels, and develop an efficient algorithm to optimize it through solving multiple linear decision boundaries and adjusting prior distributions. Our algorithm is empirically evaluated on several real world datasets, and demonstrates its supremacy over state-of-the-art methods.
Guangyong Chen (The Chinese University of Hong Kong)
Shengyu Zhang (CUHK)
Di Lin (Shenzhen University)
HUI Huang (Shenzhen University)
Pheng Ann Heng (Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences)
Dr. Heng is a professor at the Department of Computer Science and Engineering at The Chinese University of Hong Kong. He has served as the Director of Virtual Reality, Visualisation and Imaging Research Centre at CUHK since 1999 and as the Director of Centre for Human-Computer Interaction at Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences since 2006. His research interests include deep learning for medical image analysis, virtual reality applications in medicine, visualisation,and computer graphics.
Related Events (a corresponding poster, oral, or spotlight)
2017 Talk: Learning to Aggregate Ordinal Labels by Maximizing Separating Width »
Tue Aug 8th 06:06 -- 06:24 AM Room C4.9& C4.10