Pianist Transformer: Towards Expressive Piano Performance Rendering via Scalable Self-Supervised Pre-Training

Existing methods for expressive music performance rendering rely on supervised learning over small labeled datasets, which limits scaling of both data volume and model size, despite the availability of vast unlabeled music, as in vision and language. To address this gap, we introduce Pianist Transformer, with three key contributions: 1) introducing large-scale self-supervised learning into expressive piano performance rendering through a unified Musical Instrument Digital Interface (MIDI) representation, enabling pre-training on 10B tokens of unlabeled MIDI data; 2) an efficient asymmetric Transformer with note-level compression, substantially improving training efficiency, memory usage, and inference speed for long-context music modeling; 3) a state-of-the-art rendering model with an editable workflow, achieving strong objective and subjective results and enabling integration into real-world music production workflows. Overall, Pianist Transformer outlines a scalable path toward human-like performance synthesis in the music domain. Code, audio samples, and model checkpoints are available on our project page: https://yhj137.github.io/pianist-transformer-demo/.

Many piano scores contain the notes to play, but not the subtle timing, dynamics, articulation, and pedaling that make a performance sound human. Existing AI systems for expressive piano performance usually depend on small paired datasets of scores and human performances, which limits how much they can learn. We propose Pianist Transformer, a model that first learns from a very large collection of unlabeled MIDI music and then fine-tunes on aligned score-performance data. To make this possible, we design a unified MIDI representation that treats scores and performances in the same format, allowing the model to learn broad musical patterns before learning the rendering task. We also introduce an efficient Transformer architecture that compresses note sequences, making long piano pieces easier and faster to process. Experiments show that Pianist Transformer produces performances closer to human recordings than previous methods, both in objective metrics and in human listening studies. The system can also output editable MIDI files, making it useful for composers, producers, and musicians who want expressive but controllable piano renderings.