ICMAPE: In-Context Multiagent Pure Exploration

In some multi-agent systems, the primary objective is not reward maximization but efficient information acquisition to identify unknown properties of the environment as done in active sequential hypothesis testing (ASHT) problems. However, the ASHT literature tends to focus on finite single-agent problems with well-specified models, while there is currently a gap for practical multi-agent methods that can perform active sequential testing. In this work, we fill this gap and propose a Bayesian learning-based framework for decentralized multi-agent pure-exploration driven by inference objectives. Our method, ICMAPE, jointly learns a centralized neural inference network that estimates a posterior distribution over hypotheses from global trajectory data, and decentralized policies trained through reinforcement learning to select actions based only on local observation histories. Instead of relying on environment rewards, we use inference confidence as the training signal, encouraging agents to collect data that reduces uncertainty and enabling them to stop exploration once the inferred hypothesis reaches the target confidence level. On two synthetic benchmarks, ICMAPE-TD3 achieves target accuracy with fewer exploration steps, suggesting inference-driven training is a promising direction for decentralized multi-agent hypothesis testing.