Which action best supports reproducibility when reproducing a published AI safety experiment?

• A. Ignore data provenance
• B. Train a different model
• C. Use a different evaluation protocol
• D. Follow the exact evaluation protocol and data provenance

Answer: (D)

Reproducibility hinges on duplicating the exact conditions under which results were obtained. When you reproduce a published AI safety experiment, following the exact evaluation protocol and data provenance ensures you are measuring the same things in the same way and using the same inputs. Data provenance covers where data come from, how they were collected, any preprocessing or filtering, and how splits (train/validation/test) were made, including versioning. The evaluation protocol specifies the exact metrics, scoring rules, evaluation scripts, seeds, and any post-processing steps. Together, they pin down the conditions so differences in results reflect the phenomenon being studied rather than changes in data handling or measurement. If you diverge from these, such as using different data origins or a different evaluation method, you effectively alter what’s being tested and make exact replication impossible.