PRAGMA: A Foundation Model for Banking Event Sequences

We present PRAGMA, a family of encoder-style foundation models for multi-source banking event sequences. PRAGMA is pre-trained with masked modelling on a large, heterogeneous corpus of user histories, using a key–value–time tokenisation and a two-branch encoder tailored to the discrete, variable-length nature of financial records. The pre-trained backbone transfers to a wide range of downstream tasks (fraud detection, product recommendation, lifetime value, and more), supporting both frozen-embedding probes and lightweight fine-tuning. Across six diverse banking benchmarks, PRAGMA matches or exceeds strong task-specific baselines from a single shared backbone, reducing the need for hand-crafted features. We report only relative improvements, as absolute metrics are commercially sensitive; all shown examples are synthetic.