Target-Driven Policy Optimization for Sequential Counterfactual Outcome Control

Identifying optimal intervention sequences from offline data to guide temporal systems toward target outcomes is a critical challenge with profound implications for fields like personalized medicine. While existing methods are mostly evaluated in offline settings, practical applications demand online, adaptive strategies that can respond in real-time. To address this, we propose \textbf{G}oal-conditioned \textbf{I}ntervention via \textbf{F}actual-\textbf{T}argeted Training (\textbf{GIFT}), a novel framework for learning sequential intervention policies from observational data. GIFT learns a goal-conditioned policy by rescaling rewards with clipped importance weights, stabilizing learning and steering toward the target. Under standard assumptions, the induced operator has a unique fixed point and our procedure converges to it. We also bound the bias from clipping and approximation via the gap to the policy’s true value. Experiments show GIFT significantly outperforms existing methods in creating goal-conditioned policies for online deployment.

Many real-world systems, such as healthcare and other personalized decision-making settings, require a sequence of actions to guide an individual toward a desired future outcome. For example, a patient may need a series of treatments that gradually move their health state toward a clinical target. Existing methods often first predict what may happen under different intervention sequences and then search for a fixed plan, but this can be slow and may fail when the patient’s condition changes over time. This paper proposes GIFT, a method that directly learns an adaptive policy from observational data. Instead of producing one fixed intervention sequence, GIFT learns how to choose the next intervention based on the current history and the desired target. To make learning easier from limited and imperfect data, GIFT reuses past trajectories with alternative goals and adjusts rewards to reduce the mismatch between historical treatments and the learned policy. Experiments show that GIFT reaches target outcomes more accurately and efficiently than existing methods. This suggests a promising direction for real-time personalized decision support, while real-world clinical use would still require careful validation and expert oversight.