Does Temporal Encoding Matter? Evidence from GRU-based Clinical Foundation Models

Clinical foundation models for structured electronic health records handle temporal irregularity in varied ways, yet the impact of time encoding strategy on downstream performance remains poorly understood. While access to longer patient histories has been shown to improve predictive performance, there is no consensus on the consistent benefit of explicit time encoding in clinical foundation models. We train gated recurrent unit (GRU) models under five temporal encoding conditions, namely no encoding, discrete time tokens, learned projection, fixed sinusoidal encoding, and learned sinusoidal encoding on MIMIC-IV data, and evaluate downstream performance across six clinical prediction tasks using randomly sampled admission points spanning the full patient timeline. We find that the temporal encoding strategy has no significant impact on downstream performance, even for the patients with the longest sequence lengths and multiple historical visits. This implies that the ordering of data either sufficiently encodes temporal dependencies or that the dataset lacks the temporal diversity to realise the potential improvements from a deeper understanding of time. Our findings caution against assuming that richer time representations improve clinical prediction, and motivate evaluation on datasets with greater temporal heterogeneity.