Privately Fine-Tuned LLMs Preserve Temporal Dynamics in Tabular Data

Research on differentially private synthetic tabular data has largely focused on independent and identically distributed rows where each record corresponds to a unique individual. This perspective neglects the temporal complexity in longitudinal datasets, such as electronic health records, where a user contributes an entire (sub) table of sequential events. While practitioners might attempt to model such data by flattening user histories into high-dimensional vectors for use with standard marginal-based mechanisms, we demonstrate that this strategy is insufficient. Flattening fails to preserve temporal coherence even when it maintains valid marginal distributions. We introduce PATH, a novel generative framework that treats the full table as the unit of synthesis and leverages the autoregressive capabilities of privately fine-tuned large language models. Extensive evaluations show that PATH effectively captures long-range dependencies that traditional methods miss. Empirically, our method reduces the distributional distance to real trajectories by over 60% and reduces state transition errors by nearly 50% compared to leading marginal mechanisms while achieving similar marginal fidelity.

Sensitive data (like electronic health records or legal data) often tracks individuals over time, with each individual having a full history (e.g. vital signs, diagnoses, and treatments). Sharing this data is important (e.g. for medical research), but doing so risks exposing private information. Existing methods for generating privacy-preserving synthetic data assume each row in a table is independent, which works well for simple datasets but destroys the temporal patterns in longitudinal data (e.g., how a patient's heart rate evolves over the course of a hospital stay). We introduce PATH, a framework that treats each user's entire history as a single unit for privacy protection and uses large language models, fine-tuned with formal privacy guarantees, to generate realistic synthetic trajectories in data. Our approach learns to produce data that preserves how measurements change over time, rather than just matching aggregate statistics. PATH enables researchers and policymakers to work with synthetic versions of sensitive longitudinal datasets that maintain the temporal patterns necessary for meaningful analysis, while providing mathematical guarantees that no individual's private information is leaked.