SLIM: Secure and Efficient Inference for Large Language Models on Untrusted Devices via TEEs

Deploying large language models (LLMs) on untrusted hardware entails a risk of weight extraction, which can lead to unauthorized replication and misuse of the model. A practical approach is to leverage Trusted Execution Environments (TEEs) and protect model security by obfuscating model weights. However, existing obfuscation schemes struggle to simultaneously provide strong security guarantees and high performance: schemes with security guarantees incur substantial overhead due to frequent TEE interactions, whereas schemes that achieve efficient inference are insecure. We propose SLIM, a secure inference framework that exploits the iterative structure of LLMs to let transformed representations cascade through consecutive obfuscated layers, thereby minimizing interactions with the TEE. SLIM introduces a T-Way Mixing algorithm that performs consecutive inter-vector covering using carefully constructed block-diagonal Householder matrices and combines it with successive random permutations, providing thorough weight obfuscation while keeping TEE-side computation lightweight. Evaluations demonstrate that SLIM provides robust security guarantees and significantly outperforms prior state-of-the-art obfuscation schemes in terms of performance, delivering up to a $13.80\times$ speedup while preserving fidelity.

Large language models, such as chatbots and coding assistants, are valuable because of the many internal settings they learn during training. When these models run on devices or servers that the model owner does not fully control, attackers may try to copy these settings and reuse the model without permission. One way to reduce this risk is to keep sensitive computations inside a protected part of the processor, but doing this too often can make the model much slower. We propose SLIM, a system that protects large language models while keeping inference efficient. Instead of repeatedly moving data in and out of the protected processor area, SLIM lets specially transformed data flow through many model layers on its own. It also scrambles the model’s internal settings in a structured way, making them difficult to recover while keeping the extra protected computation small. Our experiments show that SLIM can preserve the model’s outputs while running much faster than prior secure inference methods, with up to a $13.80\times$ speedup. This makes secure deployment of large language models on less trusted hardware more practical.