Batch Normalization for Neural Networks on Complex Domains

Riemannian neural networks have proven effective in solving a variety of machine learning tasks. The key to their success lies in the development of principled Riemannian analogs of fundamental building blocks in deep neural networks (DNNs). Among those, Riemannian batch normalization (BN) layers have shown to enhance training stability and improve accuracy. In this paper, we propose BN layers for neural networks on complex domains. The proposed layers have close connections with existing Riemannian BN layers. We derive essential components for practical implementations of BN layers on some complex domains which are less studied in previous works, e.g., the Siegel disk domain. We conduct experiments on radar clutter classification, node classification, and action recognition demonstrating the efficacy of our method.

In many signal processing tasks, input data come in the form of complex values. Furthermore, they might have special geometric structures which cannot be leveraged by existing methods to enhance the performance of the target task. In this paper, we develop a novel method to deal with such data. By evaluating our method on different problems ranging from computer vision and natural language processing, we demonstrate that (1) the proposed method has the potential to improve existing methods, and (2) it opens up new perspectives for all applications involving complex-valued data.