Emergent Communication Under Misinformation

Social interactions are characterized by both adversarial and cooperative aspects. Communications between agents may also involve adversarially motivated actors. Messages can pass through intermediaries with malicious intent before reaching the intended receiver. These actors may modify the message to induce misunderstanding from the receiver while preserving the overall characteristics of the message. This form of misinformation is prevalent in real-world communications and may affect the dynamics under which communication protocols are developed. However, this aspect of social interaction is relatively underexplored in many emergent communication studies that aim to understand the environmental factors behind the emergence of languages' characteristics. This work explores how misinformation affects language emergence with a focus on compositionality. We design a communication game containing a malign intermediary between the sender and receiver. We find that risks of malign misrepresentation promote the emergence of compositional languages in simulations of communicative agents. Furthermore, we observe that adaptability of malign intermediaries is a crucial factor in forming a consistent pressure toward compositionality and that partial misinformation, in which the intermediary targets only a subset of attributes, can also induce compositionality.

Languages use rules that combine a finite number of words to express an almost unlimited number of concepts. This property, called compositionality, lets us express complex meanings by combining simpler parts. When communication channels are unreliable, small changes in a message may preserve the overall meaning, as when “email” is modified to “voicemail,” or destroy it, as when “blackmail” is changed to “voicemail.” Previous work has shown that random noise in communication can encourage agents to develop more compositional languages, because structured messages are easier to recover from corruption. We ask whether targeted misinformation can more effectively exploit such vulnerabilities and create stronger pressure toward compositionality. To test this, we simulate artificial agents that communicate about objects and introduce a malign intermediary that intercepts and modifies their messages to cause misunderstanding. We observe that misinformation can induce more structured languages than random noise, and that the resulting languages describe unseen objects better and are easier for new agents to learn. We find that intermediaries can be outmaneuvered and that introducing new generations of intermediaries during learning can create more consistent pressure. We also find that even when intermediaries target only some of the information contained in a message, misinformation can still promote compositional structure. The findings suggest that commonly observed acts of misinformation in social interactions can drive communication protocols to become more compositional.