Timezone: »
While semi-supervised learning (SSL) has received tremendous attentions in many machine learning tasks due to its successful use of unlabeled data, existing SSL algorithms use either all unlabeled examples or the unlabeled examples with a fixed high-confidence prediction during the training progress. However, it is possible that too many correct/wrong pseudo labeled examples are eliminated/selected. In this work we develop a simple yet powerful framework, whose key idea is to select a subset of training examples from the unlabeled data when performing existing SSL methods so that only the unlabeled examples with pseudo labels related to the labeled data will be used to train models. The selection is performed at each updating iteration by only keeping the examples whose losses are smaller than a given threshold that is dynamically adjusted through the iteration. Our proposed approach, Dash, enjoys its adaptivity in terms of unlabeled data selection and its theoretical guarantee. Specifically, we theoretically establish the convergence rate of Dash from the view of non-convex optimization. Finally, we empirically demonstrate the effectiveness of the proposed method in comparison with state-of-the-art over benchmarks.
Author Information
Yi Xu (Alibaba Group (U.S.) Inc.)
Lei Shang (Alibaba Group)
Jinxing Ye (Alibaba)
Qi Qian (Alibaba Group)
Yu-Feng Li (Nanjing University)
Baigui Sun (Alibaba Group)
Hao Li (Alibaba Group)
rong jin (alibaba group)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Oral: Dash: Semi-Supervised Learning with Dynamic Thresholding »
Thu. Jul 22nd 01:00 -- 01:20 PM Room
More from the Same Authors
-
2023 Oral: ODS: Test-Time Adaptation in the Presence of Open-World Data Shift »
Zhi Zhou · Lan-Zhe Guo · Lin-Han Jia · Dingchu Zhang · Yu-Feng Li -
2023 Oral: Bidirectional Adaptation for Robust Semi-Supervised Learning with Inconsistent Data Distributions »
Lin-Han Jia · Lan-Zhe Guo · Zhi Zhou · Jie-Jing Shao · Yuke Xiang · Yu-Feng Li -
2023 Poster: Supported Trust Region Optimization for Offline Reinforcement Learning »
Yixiu Mao · Hongchang Zhang · Chen Chen · Yi Xu · Xiangyang Ji -
2023 Poster: Bidirectional Adaptation for Robust Semi-Supervised Learning with Inconsistent Data Distributions »
Lin-Han Jia · Lan-Zhe Guo · Zhi Zhou · Jie-Jing Shao · Yuke Xiang · Yu-Feng Li -
2023 Poster: mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video »
Haiyang Xu · Qinghao Ye · Ming Yan · Yaya Shi · Jiabo Ye · yuanhong xu · Chenliang Li · Bin Bi · Qi Qian · Wei Wang · Guohai Xu · Ji Zhang · Songfang Huang · Fei Huang · Jingren Zhou -
2023 Poster: Identifying Useful Learnwares for Heterogeneous Label Spaces »
Lan-Zhe Guo · Zhi Zhou · Yu-Feng Li · Zhi-Hua Zhou -
2023 Poster: ODS: Test-Time Adaptation in the Presence of Open-World Data Shift »
Zhi Zhou · Lan-Zhe Guo · Lin-Han Jia · Dingchu Zhang · Yu-Feng Li -
2022 Poster: MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection »
Zhenhong Sun · Ming Lin · Xiuyu Sun · Zhiyu Tan · Hao Li · rong jin -
2022 Spotlight: MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection »
Zhenhong Sun · Ming Lin · Xiuyu Sun · Zhiyu Tan · Hao Li · rong jin -
2022 Poster: Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding »
Lan-Zhe Guo · Yu-Feng Li -
2022 Poster: FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting »
Tian Zhou · Ziqing MA · Qingsong Wen · Xue Wang · Liang Sun · rong jin -
2022 Spotlight: FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting »
Tian Zhou · Ziqing MA · Qingsong Wen · Xue Wang · Liang Sun · rong jin -
2022 Spotlight: Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding »
Lan-Zhe Guo · Yu-Feng Li -
2020 Poster: Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data »
Lan-Zhe Guo · Zhen-Yu Zhang · Yuan Jiang · Yu-Feng Li · Zhi-Hua Zhou -
2020 Poster: Stochastic Optimization for Non-convex Inf-Projection Problems »
Yan Yan · Yi Xu · Lijun Zhang · Wang Xiaoyu · Tianbao Yang -
2019 Poster: On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization »
Hao Yu · rong jin · Sen Yang -
2019 Poster: On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimization »
Hao Yu · rong jin -
2019 Poster: Stochastic Optimization for DC Functions and Non-smooth Non-convex Regularizers with Non-asymptotic Convergence »
Yi Xu · Qi Qi · Qihang Lin · rong jin · Tianbao Yang -
2019 Oral: Stochastic Optimization for DC Functions and Non-smooth Non-convex Regularizers with Non-asymptotic Convergence »
Yi Xu · Qi Qi · Qihang Lin · rong jin · Tianbao Yang -
2019 Oral: On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimization »
Hao Yu · rong jin -
2019 Oral: On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization »
Hao Yu · rong jin · Sen Yang -
2019 Poster: Katalyst: Boosting Convex Katayusha for Non-Convex Problems with a Large Condition Number »
Zaiyi Chen · Yi Xu · Haoyuan Hu · Tianbao Yang -
2019 Oral: Katalyst: Boosting Convex Katayusha for Non-Convex Problems with a Large Condition Number »
Zaiyi Chen · Yi Xu · Haoyuan Hu · Tianbao Yang -
2018 Poster: Dynamic Regret of Strongly Adaptive Methods »
Lijun Zhang · Tianbao Yang · rong jin · Zhi-Hua Zhou -
2018 Poster: SADAGRAD: Strongly Adaptive Stochastic Gradient Methods »
Zaiyi Chen · Yi Xu · Enhong Chen · Tianbao Yang -
2018 Oral: SADAGRAD: Strongly Adaptive Stochastic Gradient Methods »
Zaiyi Chen · Yi Xu · Enhong Chen · Tianbao Yang -
2018 Oral: Dynamic Regret of Strongly Adaptive Methods »
Lijun Zhang · Tianbao Yang · rong jin · Zhi-Hua Zhou -
2017 Poster: Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence »
Yi Xu · Qihang Lin · Tianbao Yang -
2017 Talk: Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence »
Yi Xu · Qihang Lin · Tianbao Yang