Who Gets Credit or Blame? Attributing Accountability in Modern AI Systems

Modern AI systems are typically developed through multiple stages: pretraining, fine-tuning rounds, and subsequent adaptation or alignment, where each stage builds on the previous ones and updates the model in distinct ways. This raises a critical question of accountability: when a deployed model succeeds or fails, which stage is responsible, and to what extent? We pose the accountability attribution problem for tracing model behavior back to specific stages of the model development process. To address this challenge, we propose a general framework that answers counterfactual questions about stage effects: how would the model's behavior have changed if the updates from a particular stage had not occurred? Within this framework, we introduce estimators that efficiently quantify stage effects without retraining the model, accounting for both the data and key aspects of model optimization dynamics, including learning rate schedules, momentum, and weight decay. We demonstrate that our approach successfully quantifies the accountability of each stage to the model's behavior. Based on the attribution results, our method can identify stages associated with spurious correlations in image classification and text toxicity detection tasks and guide targeted follow-up interventions. Our approach provides a practical tool for model analysis and represents a significant step toward more accountable AI development.

Modern AI systems are typically developed through multiple stages, such as pretraining and fine-tuning, making it difficult to pinpoint which stage is responsible for a model’s success or failure. This work introduces the problem and framework of accountability attribution to trace a model’s behavior back to development stages. The framework estimates how the model would behave if a certain stage had not occurred without needing to retrain the model. Our approach efficiently captures the impact of both data and optimization details. This work provides a practical tool for understanding how AI systems evolve during training, promoting transparency and accountability in model development, especially as AI is deployed in increasingly high-stakes settings.