Universal domain adaptation (UniDA) is a more challenging domain adaptation setting, which introduces category shift on top of domain shift. As UniDA lacks prior knowledge about the category overlap between the source and target domains, it needs to identify unknown category in the target domain that does not exist in the source domain and avoid misclassifying target samples into source private categories. To this end, this paper proposes a novel UniDA approach named Batch Singular value Polarization and Weighted Semantic Augmentation (BSP-WSA). Specifically, we adopt an adversarial classifier to identify target unknown category and align feature distributions between the two domains. Then, we propose a batch singular value polarization approach, which performs SVD on the classifier's outputs to maximize larger singular values while minimizing those smaller ones. This could prevent target samples from being wrongly assigned to source private classes, thereby achieving distribution alignment between common categories. To better bridge domain gap, we propose a weighted semantic augmentation approach for UniDA, aiming to generate data on common categories between the two domains. Extensive experiments on three benchmarks demonstrate that our proposed BSP-WSA could outperform existing state-of-the-art UniDA approaches.