Learned Sequence Representations over Raw Credit Events for Credit-Abuse Scoring

Credit abuse is a rare and costly outcome at consumer-credit platforms; industry evidence places its concentration in moderately-aged credit files. Tuned gradient-boosted decision trees (GBDTs) over hand-engineered features are the prevailing choice, and recent tabular foundation models are advertised to match them. We ask whether a learned representation over raw credit-event sequences, used in place of engineered aggregates of those events, is a better fit. To produce this representation, we pretrain a causal Transformer on credit-event sequences with self-supervised objectives and adapt the backbone to the downstream label via supervised finetuning. On an out-of-time evaluation, under both an XGBoost classifier and an in-context tabular foundation model, the sequence-based feature set beats the engineered-aggregate alternative on AUPR (XGBoost: 0.068 $\rightarrow$ 0.147; TabPFN: 0.097 $\rightarrow$ 0.141). Among single-factor upgrades from the (XGBoost, engineered) baseline, at production training scale the representation-only upgrade (XGBoost on the sequence-based vector) outperforms the classifier-only upgrade (TabPFN on the engineered alternative) on both AUPR (0.147 vs 0.097) and recall at a 0.5\% decline threshold (0.230 vs 0.184).