BAT: Better Audio Transformer Guided by Convex Gated Probing

Probing is widely adopted in computer vision to faithfully evaluate self-supervised learning (SSL) embeddings, as finetuning may misrepresent their inherent quality. In contrast, audio SSL models still rely on finetuning because simple probing fails to unlock their full potential and alters their rankings when competing on AudioSet. Hence, a robust and efficient probing mechanism is required to guide the trajectory of audio SSL towards reliable and reproducible methods. We introduce Convex Gated Probing (CGP), a prototype-based method that significantly closes the gap between finetuning and probing in audio. CGP efficiently utilizes all frozen layers via a gating mechanism and exposes the location of latent task-relevant information. Guided by CGP as a reliable post-hoc evaluation probe, we rework the entire SSL pipeline of current best performing audio models that use legacy implementations of prior SSL methods. By refining data preprocessing, model architecture, and pretraining recipe, we introduce Better Audio Transformer (BAT), and establish new SOTA on audio benchmarks.

Modern audio AI models can learn useful patterns from millions of unlabeled recordings, but it is difficult to tell whether a new models is truly better than a previous one. In audio research, models are usually judged after they are heavily finetuned for a specific benchmark, which can hide whether improvements come from better pretraining algorithms and models or from extensive training. We introduce Convex Gated Probing, a lightweight evaluation method that keeps the pretrained audio model frozen and measures how useful its off-the-shelf features are. This method gives a clearer view of what the model has learned and where task-relevant information is stored during the training process. We redesign the training pipeline for audio self-supervised learning, improving the preprocessing, the transformer architecture, and the way the model learns from unlabeled audio. The resulting model, Better Audio Transformer, achieves stronger performance across audio benchmarks while using a more transparent and reproducible implementation. Our work helps make progress in audio models easier to measure, reproduce, and build upon.