INDEXGUARD: Index-only Backdoor Vetting for Secure Federated PEFT of Large Language Models

Federated parameter-efficient fine-tuning (PEFT) enables customizing large language models on private data, yet it is vulnerable to backdoor poisoning—especially when privacy constraints prevent inspection of per-client real-valued updates. We exploit the intuition that poisoning leaves a similar backdoor imprint in which adapter coordinates become salient, so overlap in salient-index supports remains informative even without values. We introduce IndexGuard, an unsupervised index-only vetting primitive in which clients send only Top-$K$ salient update indices and the server operates on the induced overlap geometry, clustering clients and filtering cohesion-outlier groups before aggregation. We analyze support stability under bounded rescaling and separability under shared-trigger poisoning under non-IID drift. Across attacks, backbones, and PEFT variants, IndexGuard provides end-to-end mitigation, preserving clean accuracy while achieving performance comparable to centralized methods.

Many organizations want to adapt large language models to their own private data, but sharing this data or exposing each participant’s model updates can create privacy risks. At the same time, malicious or compromised participants may try to secretly insert a “backdoor,” causing the model to behave normally most of the time but produce attacker-chosen outputs when a hidden trigger appears. This paper introduces IndexGuard, a lightweight way to check for such poisoned participants before their updates are combined into the shared model. Instead of inspecting full model updates, clients only report which parts of their small fine-tuning update changed the most. IndexGuard compares these compact reports across clients and filters groups that look suspicious. Experiments show that this approach can detect backdoored clients across different attacks, tasks, models, and fine-tuning methods while preserving normal model accuracy and remaining compatible with privacy-preserving federated learning.