In recent years, diffusion models have risen to prominence as the foremost technique for distribution learning. This paper focuses on structure-preserving diffusion models (SPDM), a subset of diffusion processes tailored to distributions with inherent structures, such as group symmetries. We complement existing sufficient conditions for constructing SPDM by proving complementary necessary ones. Additionally, we propose a new framework that considers the geometric structures affecting the diffusion process. Within this framework, we propose a method of preserving the alignment between endpoint couplings in bridge models to design a novel structure-preserving bridge model. We validate our findings over a variety of equivariant diffusion models by learning symmetric distributions and the transitions among them. Empirical studies on real-world medical images indicate that our models adhere to our theoretical framework, ensuring equivariance without compromising the quality of sampled images. Furthermore, we showcase the practical utility of our framework by achieving reliable equivariant image noise reduction and style transfer, irrespective of prior knowledge of image orientation, by implementing an equivariant denoising diffusion bridge model (DDBM).