Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control

Controlling autonomous systems under real-world conditions often requires policies that can be evaluated with low latency and minimal energy consumption. Unfortunately, these conditions are at odds with the use of high-precision deep neural networks as controllers. In this work, we introduce Differentiable Weightless Controllers (DWCs), a symbolic-differentiable architecture that learns flexible, non-linear, yet highly efficient control policies. DWCs can be trained end-to-end via gradient-based techniques, yet compile directly into FPGA-compatible circuits with few- or even single-clock-cycle latency and nanojoule-level energy cost per action. Across five MuJoCo benchmarks, including high-dimensional Humanoid, DWCs achieve returns competitive with standard deep policies (full-precision or quantized neural networks). Furthermore, DWCs exhibit structurally sparse and interpretable connectivity patterns, enabling direct inspection of which input values influence control decisions.

Controlling robots in the real world often requires decisions to be made very quickly and with little energy use. Standard neural-network controllers can solve many control tasks, but they may require too much computation and power for small or embedded hardware. Instead, we train controllers that can be converted directly into efficient digital circuits. These circuits can make decisions with very low latency and low energy consumption, while achieving performance similar to standard neural-network controllers on several robot-control benchmarks. Because the resulting circuits are sparse and structured, they are also easier to inspect than typical black-box neural networks.