BEAT: Tokenizing and Generating Symbolic Music by Uniform Temporal Steps

Tokenizing music to fit the general framework of language models is a compelling challenge, especially considering the diverse symbolic structures in which music can be represented (e.g., sequences, grids, and graphs). To date, most approaches tokenize symbolic music as sequences of musical events, such as onsets, pitches, time shifts, or compound note events. This strategy is intuitive and has proven effective in Transformer-based models, but it treats the regularity of musical time implicitly: individual tokens may span different durations, resulting in non-uniform time progression. In this paper, we instead consider whether an alternative tokenization is possible, where a uniform-length musical step (e.g., a beat) serves as the basic unit. Specifically, we encode all events within a single time step at the same pitch as one token, and group tokens explicitly by time step, which resembles a sparse encoding of a piano-roll representation. We evaluate the proposed tokenization on music continuation and accompaniment generation tasks, comparing it with mainstream event-based methods. Results show improved musical quality and structural coherence, while additional analyses confirm higher efficiency and more effective capture of long-range patterns with the proposed tokenization.

AI systems can now write convincing essays and generate striking images, yet getting them to compose coherent music remains surprisingly hard. The core difficulty is rhythm: music has a steady underlying pulse — beats and bars — that humans feel naturally, but most AI methods turn music into long lists of individual notes and timings, losing this pulse along the way. It is a bit like describing a song by reading out every note in order, never marking where the beats fall. We propose BEAT, a new way of presenting music to AI that keeps the beat as the basic unit. Instead of streaming notes as isolated events, BEAT groups everything happening within each beat into a single compact unit and stacks these units in order. This lets the AI learn directly from music's natural temporal regularity, rather than rediscover it from scratch. Models trained with BEAT generate pieces that listeners rate as more coherent and musical than current methods, and the same representation supports real-time generation — for example, an AI accompanist that follows you as you play. By aligning what the AI sees with how humans actually perceive music, BEAT brings practical, expressive AI music generation closer to everyday use.