PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training

Large language model (LLM) agents are increasingly deployed in personalized tasks involving sensitive, context-dependent information, where privacy violations may arise in agents' action due to the implicitness of contextual privacy. Existing approaches rely on external, inference-time interventions which are brittle, scenario-specific, and may expand the privacy attack surface. We propose PrivAct, a contextual privacy-aware multi-agent learning framework that internalizes contextual privacy preservation directly into models' generation behavior for privacy-compliant agentic actions. By embedding privacy preferences into each agent, PrivAct enhances system-wide contextual integrity while achieving a more favorable privacy-helpfulness tradeoff. Experiments across multiple LLM backbones and benchmarks demonstrate consistent improvements in contextual privacy preservation, reducing leakage rates by up to 12.32\% while maintaining comparable helpfulness, as well as zero-shot generalization and robustness across diverse multi-agent topologies. Code is available at https://github.com/chengyh23/PrivAct.

As AI agents are increasingly trusted with personal tasks like managing emails or schedules, they can accidentally leak private data. Current defenses rely on rigid rules or prompt engineering applied at inference time, which are brittle, complex to design, and can expand privacy attack surface. To fix this, we developed PrivAct, a framework that trains multi-LLM agents to naturally internalize privacy awareness during their initial development. Across various testing scenarios, PrivAct cut data leaks by up to 12.32% while keeping the AI agents effective and reliable.