Drop-in Circulant Structural Priors for Transformer Decoding of Cyclic Codes

While Transformer-based architectures have revolutionized neural decoding, existing models often treat codes as generic sequences, ignoring their inherent algebraic properties. In this paper, we take a step toward bridging these two domains by proposing a novel decoding approach that integrates the algebraic structure of cyclic codes into Transformer-based decoders. Leveraging the inherent cyclic properties, we introduce interpretable error correction patterns and inter-node relationship hypotheses that link the structural characteristics of the codes to the model parameters. Building on these insights, we design a plug-and-play, flexibly deployable decoding method tailored for cyclic codes. Experimental results show that our method achieves an average reduction in bit error rate (BER) by an order of magnitude, while also reducing the total number of parameters by approximately 97%. Additional comparative experiments validate our proposed conjectures and highlight a promising pathway for bridging classical coding theory and modern Transformer-based decoding architectures.

Digital communication systems add carefully designed redundancy to data so receivers can correct errors caused by noise. Recently, Transformer models have been used for this task, but they often treat an error-correcting code like an ordinary sequence and ignore the algebraic patterns that make the code work. This paper focuses on cyclic codes, a widely studied family whose code positions repeat in a circular pattern. We show how to expose this circular structure to Transformer-based decoders by using a circulant parity-check representation and by reusing model parameters according to the same cyclic pattern. In simple terms, the decoder no longer has to learn many similar correction rules separately; it can learn one rule and shift it around the code. This makes the decoder both more accurate and much smaller. Across several cyclic codes and Transformer decoders, the proposed plug-in method lowers the bit error rate by about one order of magnitude on average while reducing the number of parameters by about 97%. The work suggests that combining classical coding theory with modern neural networks can lead to communication systems that are more reliable, efficient, and easier to interpret.