NeurIPSはフロンティアAI安全性主張に再現性基準を課すべき NeurIPS Should Require Reproducibility Standards for Frontier AI Safety Claims

This position paper urges NeurIPS, one of the leading venues in machine learning, to impose stricter reproducibility requirements specifically on submissions that make empirical safety claims about frontier AI systems.

Over the past two years, developers of models such as GPT-4, Claude, and Gemini have published dangerous-capability evaluations, red-teaming results, and system cards alongside their releases. These artifacts frequently underpin claims that a model is safe to deploy, yet the underlying prompts, scoring rubrics, sampling parameters, and model access conditions are often withheld from outside researchers. The authors appear to argue that allowing such unverifiable claims to pass peer review lends them undue scientific authority and risks misinforming policymakers and downstream users.

The central proposal is to require, for any submission asserting safety properties, disclosure of evaluation code, full prompt sets, raw model outputs, decoding configurations, and enough methodological detail to allow independent replication. For closed-weight commercial systems, the authors likely envision intermediate mechanisms such as gated access for vetted reviewers, structured cooperation with third-party evaluators, or minimum disclosure protocols that preserve replicability without releasing hazardous content wholesale.

Context matters here. NeurIPS introduced its reproducibility checklist in 2019 and has progressively strengthened ethics review since 2022. The broader ecosystem has also moved in this direction: the UK AI Safety Institute (now AISI), the US AI Safety Institute at NIST, and independent evaluators like METR and Apollo Research have built dedicated capacity for external testing, while ICLR has run a long-standing reproducibility challenge. The paper can be read as extending these reproducibility norms into the specific and politically charged domain of frontier safety evaluation.

There are genuine tensions the authors must navigate. Frontier safety evaluations often probe capabilities related to bioweapon synthesis, cyberoffense, or autonomous replication, where full openness could itself be hazardous. A blanket mandate to publish all prompts and outputs would conflict with responsible-disclosure norms that AI labs and governments have been negotiating. The paper presumably advocates a middle path, possibly modeled on how security conferences handle vulnerability disclosure, where artifacts are shared under controlled conditions rather than posted publicly.

Another subtle issue is that safety claims are typically negative claims, asserting the absence of a capability. These are notoriously hard to verify, because elicitation methods improve over time and a model judged safe today may be coaxed into harmful behavior tomorrow with better prompting or fine-tuning. Reproducibility standards alone cannot solve this, but they at least provide a baseline against which future researchers can measure whether evaluations were thorough.

Whether NeurIPS area chairs would accept the operational burden of enforcing such standards remains an open question. Reviewers are volunteers, often without access to closed models, and adjudicating safety claims may require specialized expertise beyond typical ML peer review. One plausible compromise would be a dedicated track or supplementary review committee for frontier safety submissions, an idea that other venues such as ICML and COLM may also need to consider.

If adopted, the proposal could meaningfully shift incentives, pushing labs either to substantiate safety claims more rigorously or to refrain from publishing them in academic venues at all. The debate it opens, about how far peer review should police industrial safety claims, is likely to ripple through AI governance discussions well beyond NeurIPS itself.