Power and Limitations of Aggregation in Compound AI Systems

A common use of compound AI systems is querying multiple homogeneous copies of a model under different prompts and aggregating their outputs. We ask when this unlocks outputs that no single query can elicit. Within a stylized principal-agent framework that captures both prompt-engineering and model-capability limitations, we identify three natural mechanisms by which aggregation expands the set of elicitable outputs---feasibility expansion, support expansion, and binding-set contraction. We prove that strengthened versions of these mechanisms exactly characterize when aggregation adds power. We complement the characterization with an empirical illustration on a toy reference-generation task with LLMs.