EveryQuery: Zero-Shot Clinical Prediction via Task-Conditioned Pretraining

Foundation models pretrained on electronic health records (EHR) have demonstrated zero-shot clinical prediction. They use an autoregressive procedure to generate synthetic patient futures and aggregate statistics over sampled trajectories, which is computationally expensive, statistically noisy for rare events, and not natively promptable. We introduce EveryQuery, an EHR foundation model that achieves zero-shot inference through task-conditioned pretraining over a structured query language. Rather than generating future trajectories, EveryQuery takes a patient's history and a query $q = (c, \Delta t)$ specifying a target code and a prediction horizon, and directly estimates the outcome probability via a single forward pass. On MIMIC-IV, EveryQuery outperforms an autoregressive baseline on 85% of 200 (code, duration) prediction tasks (mean $\Delta$AUC $+0.17$, 95\% CI $[0.14, 0.19]$). The advantage is most pronounced for rare events as autoregressive performance is strongly coupled to event prevalence, while EveryQuery's is prevalence-invariant. EveryQuery also generalizes to held-out codes and to held-out durations, including extrapolation beyond the longest training horizon. Together, these results suggest that task-conditioned pretraining is a viable paradigm for EHR foundation models.