Constrained Multi-Objective Reinforcement Learning with Max-Min Criterion

Multi-Objective Reinforcement Learning (MORL) extends standard RL by optimizing policies with respect to multiple, often conflicting, objectives. While max-min MORL has emerged as an effective approach for promoting fairness, its applicability remains limited, particularly when constraints must be incorporated. In this paper, we propose a MORL framework that integrates the max-min criterion with explicit constraint satisfaction. We establish a theoretical foundation for the proposed framework and validate the resulting algorithm through convergence analysis and experiments in tabular settings. We further demonstrate the practical relevance of our approach in simulated building thermal control, multi-objective locomotion control, and greenhouse-gas-emission-aware traffic management. Across these domains, our method effectively balances fairness and constraint satisfaction in multi-objective decision-making.

Multi-objective reinforcement learning is a type of machine learning that trains an agent to make decisions while balancing several goals at the same time, which can often conflict with each other. In many real-world problems, it is also important to follow safety or resource limits while making these decisions. In this work, we propose a new learning framework that combines a fairness-oriented objective with explicit constraint handling. We test our approach in simulated environments such as building temperature control, robot locomotion, and traffic systems. Across these applications, our method consistently achieves a good balance between fairness across objectives and satisfying constraints.