Recovering Selectivity with LTI State Space Operators for Portable Long-Context Inference

Mamba has established selective state space models as powerful building blocks for long-context foundation models by replacing linear time-invariant (LTI) recurrences with input-dependent selective recurrences. However, efficient training of such recurrences often relies on specialized fused kernels, limiting portability across diverse hardware. We propose SelectLTI, a simple but effective block that augments an LTI recurrence with input-dependent modulation to recover selective behavior. The proposed block equips any LTI state-space layer with sigmoid-gated modulators, making it trainable and deployable without specialized fused kernels. We evaluate SelectLTI on Selective Copying, a task designed to isolate selective-memory behavior in sequence models. SelectLTI matches or exceeds S6, the selective state-space operator of Mamba, showing that selective behavior can be recovered from an LTI recurrence augmented with lightweight modulation using only standard primitives. We position SelectLTI as a portable building block for efficient long-context foundation models deployed across diverse compute environments.