Revisiting Non-Acyclic GFlowNets in Discrete Environments

Generative Flow Networks (GFlowNets) are a family of generative models that learn to sample objects from a given probability distribution, potentially known up to a normalizing constant. Instead of working in the object space, GFlowNets proceed by sampling trajectories in an appropriately constructed directed acyclic graph environment, greatly relying on the acyclicity of the graph. In our paper, we revisit the theory that relaxes the acyclicity assumption and present a simpler theoretical framework for non-acyclic GFlowNets in discrete environments. Moreover, we provide various novel theoretical insights related to training with fixed backward policies, the nature of flow functions, and connections between entropy-regularized RL and non-acyclic GFlowNets, which naturally generalize the respective concepts and theoretical results from the acyclic setting. In addition, we experimentally re-examine the concept of loss stability in non-acyclic GFlowNet training, as well as validate our own theoretical findings.

Imagine trying to guide the assembly of toy construction blocks to create the most impressive or functional structures. We work on smart AI helpers called GFlowNets that learn sequences of placing, removing, or rearranging blocks to achieve diverse successful builds. Traditionally, these helpers assume construction always moves forward linearly, never removing an already placed block or disassembling a section to try a different approach towards a specific design.Our research studies ways for these AI helpers to understand more complex assembly paths — ones where they might repeat beneficial construction techniques or take winding routes by disassembling and reconfiguring sections to discover even stronger, more creative, or more diverse finished structures. We revisit previous research on this topic, presenting a simpler and more intuitive theoretical framework for designing such models, as well as experimentally study the ways for these AI helpers to learn more efficiently. By allowing these models to explore more flexible building journeys, we're paving the way for AI that can find richer solutions to complex, real-world problems.