Causal representation learning tackles the problem of discovering high-level variables from low-level observations. In this talk, I will discuss how modular architectures such as Neural Interpreters and Neural Attentive Circuits implement inductive biases from the causal principle of independent mechanisms. Leveraging dynamic connectivity graphs and conditional computatations, I will showcase their scalability and interesting properties for robust recognition, efficient transfer, and reasoning.