FuseFSS: Efficient Secure LLM Inference with Function Secret Sharing

Two-server secure inference allows a client to query a hosted large language model (LLM) without revealing prompts or embeddings. Recent GPU systems based on function secret sharing (FSS) make linear layers efficient, but fixed-point nonlinearities and helper operations remain a bottleneck because each operator is typically implemented as a bespoke protocol with its own comparisons, wrap-around corrections, and preprocessing material. We present FuseFSS, a compiler that replaces per-operator protocol design with a single compilation pipeline. For each scalar fixed-point operator, a compact specification lists its interval partition, low-degree arithmetic pieces, and required predicate bits. The compiler emits two batched FSS evaluations on the public masked value: one packed comparison that returns all predicate bits, and one vector interval lookup that returns the active coefficients and constants. Compared to the current state-of-the-art FSS-based GPU secure inference, FuseFSS preserves accuracy while achieving a $1.24\times$–$1.50\times$ end-to-end speedup and reducing online communication by 9%–16% on BERT and GPT-style models; preprocessing is also lighter, with 14%–23% lower key-generation time and 20%–24% smaller keys.

Large language models are often used through online services, but users may not want to reveal sensitive inputs such as private prompts, medical notes, or business data. Secure inference protocols can protect these inputs by splitting the computation across two non-colluding servers, but this protection usually makes model inference much slower. This paper introduces FuseFSS, a compiler-based approach that makes a difficult part of secure language-model inference more efficient. Instead of designing a separate cryptographic protocol for each nonlinear operation in a model, FuseFSS describes these operations in a common format and automatically turns them into a small number of reusable secure-computation steps. This makes the system easier to extend and reduces repeated engineering effort. Experiments on BERT and GPT-style models show that FuseFSS preserves model accuracy while improving end-to-end inference speed and reducing communication and preprocessing costs. The work helps move privacy-preserving language-model services closer to practical deployment.