From Claim to Check: Auditing the Verifiability of Public AI Disclosures

Frontier AI labs frequently publish disclosures intended to validate the performance, safety, and reliability of their foundation models. These claims can shape governance, procurement, and external oversight because, absent rigorous independent validation, they are often one basis on which outside actors assess frontier AI systems. We ask whether the claims made in these disclosures can be validated using only the information released alongside them. As a proof of concept, we audit $61$ claims from $16$ public artifacts across $12$ model creators using a custom six-field rubric for claim reproducibility. Only $21$\% are directly verifiable on our rubric; the dominant failure modes are ambiguous wording, incomplete protocol specification, and missing access to the relevant evaluation context. We provide a full analysis of the rubric and results, and propose a disclosure standard targeted at the failures we observe.