Position: Peer Review in ML/AI Conferences Should Separate Publication from Presentation and Offer Non-Anonymous Review Tracks

In this position paper, we enumerate a number of problems with the current peer-review process based on extensive empirical evidence. We argue for two structural reforms: (1) separating publication from presentation via a four-step process that first evaluates correctness, publishes all sound papers, then uses community-based ratings to select presentations; and (2) offering parallel anonymous and non-anonymous review tracks, where the non-anonymous track releases all review data publicly to increase accountability and generate valuable research datasets. We argue how our proposed policies can mitigate these problems. We urge the community to leverage the learnings from the experiments conducted in peer-review processes and incorporate evidence-based policy design.