End-to-end Auditing of Decision Pipelines

Many high-stakes policies can be modeled as a sequence of decisions along a \textit{pipeline}. We are interested in auditing such pipelines for both \textit{efficiency} and \textit{equity}. Using a dataset of over 100,000 crowdsourced resident requests for potentially hazardous tree maintenance in New York City, we observe a sequence of city government decisions about whether to inspect and work on a reported incident. At each decision in the pipeline, we define parity definitions and tests to identify inefficient, inequitable treatment. Disparities in resource allocation and scheduling across census tracts are reported as preliminary results.