Time-Conditioned Foreseeing: An EHR-Specific Foundation Model for Irregular Dynamics and Calendrical Time

Electronic Health Records (EHRs) possess unique characteristics distinct from natural language, yet existing EHR foundation models often rely on suboptimal NLP-based approaches. We propose a pretraining method tailored to EHRs' distinct features. First, we introduce Pathology-Focused Binning, a density-based quantization strategy that prioritizes clinically significant numerical ranges over usual values. Second, to jointly capture both the exact timing of clinical events and the relative intervals between them, we propose Dual-Calendar Rotary Positional Embedding (RoPE), which encodes absolute and relative temporal signals. Third, we introduce the Time-Conditioned Foreseeing (TCF) objective, aligning with clinical treatment planning to forecast events across multiple temporal horizons by explicitly modeling event timing. Our approach establishes a temporal generative EHR model that outperforms existing foundation models on eleven diverse downstream tasks—achieving up to a 54% improvement in AUPRC—and enables the generation of realistic, temporally consistent patient trajectories. Code is available at https://github.com/Pusheen-cat/TCF_PFM.

Hospitals record a constant stream of data about each patient—lab results, vital signs, treatments—each stamped with the exact time it happened. Artificial intelligence could learn from these records to help doctors anticipate what a patient needs next, but most existing systems borrow tools built for ordinary language. That mismatch causes them to blur the meaning of medical numbers, ignore when events actually occur, and only guess the very next step rather than plan ahead. We built an AI model designed specifically for medical records. It pays closer attention to unusual lab values that often signal danger, understands both the calendar timing of events and the gaps between them, and learns to forecast what may happen hours into the future—much like a physician planning care. Tested on two large public hospital datasets, our model outperformed leading alternatives across eleven clinical prediction tasks and could generate realistic, time-accurate patient histories that physicians judged more believable than those from competing systems. This work could help clinicians catch high-risk events earlier and supports safer medical research by creating lifelike synthetic patient data without exposing real records.