In this work we demonstrate different variations of counterfactual fairness that avoid one of the central sociological and normative tensions in counterfactual-based fairness notions: modularity. Building on recent developments in causal modeling formalisms, we introduce \emph{backtracking counterfactual fairness}, a novel definition of counterfactual fairness that avoids the modularity assumption by using backtracking rather than interventional counterfactuals. We also propose an alternate modeling strategy via causal relational modeling that instead provides a solution at the database schema-level: choosing to represent variables that violate the modularity assumption as database \emph{entities} rather than individual attributes. Both of these proposals allow for the consideration of counterfactual-based fairness notions even in the presence of non-modular variables.