Collusion of Reinforcement Learning-based Pricing Algorithms in Episodic Markets

Pricing algorithms have demonstrated the capability to learn tacit collusion that is largely unaddressed by current regulations. Their adoption in markets, including oligopolies with a history of collusion, necessitates further scrutiny by competition regulators. We extend the analysis of tacit collusion emerging through learning from simple pricing games to market domains that model goods with a sell-by date and fixed supply, such as airline tickets, perishables, or hotel rooms. We formalize collusion in this framework and define a measure based on the price levels under the competitive (Nash) and collusive (monopoly) equilibria. Since no analytical formulas for these prices exist, we illustrate an efficient computational method. Our experiments show that deep reinforcement learning agents learn to compete in both simple pricing games and our domain, while they show some evidence of learned collusion that warrants further analysis.