PISA: Privacy-Preserving Split Adaptation with Model IP Protection

Fine-tuning Large Language Models (LLMs) enables data holders to construct proprietary, task-specific models by leveraging external high-performance computing infrastructure. However, existing paradigms typically address data privacy and model intellectual property (IP) in isolation, failing to simultaneously uphold both constraints. Privacy-prioritized methods compromise model IP by hosting parameters remotely, while IP-oriented collaborative schemes relying on end-to-end gradient flows inherently violate strict data privacy standards. To address these challenges, we present PISA (Privacy-preserving and IP-protected Split Adaptation), a split fine-tuning framework designed to preserve both data privacy and model IP while maintaining high utility. In PISA, we propose three methods: a Manifold Rectification Pre-training (MRP) method to equip the server-side model with intrinsic robustness against privacy-induced distribution shifts; a Dual-Stream Semantic Compensation (DSC) method to recover feature utility using local clean data as priors; and a Utility-Aware Gradient Rectification (UGR) method to adaptively maximize the performance of the parameter-constrained local model. Experiments on GLUE show that PISA ensures dual protection and delivers a substantial 23.0\% performance gain over the privacy-prioritized baseline under strict privacy budgets.

Training AI language models on sensitive data, such as medical records or financial information, risks exposing either the private data or the proprietary model to untrusted cloud providers. PISA is a framework that enables organizations to fine-tune large language models using private data while simultaneously protecting both data privacy and model ownership. By splitting the model between the server and the local device and applying novel compensation techniques, PISA achieves strong dual protection without sacrificing model accuracy.