Which mentor is the AI Safety mentor focused on scalable oversight and evaluation?

• A. Sam Bowman
• B. Saffron Huang
• C. Nicholas Carlini
• D. Peter McCrory

Answer: (A)

Scalable oversight and evaluation is about building methods to supervise and judge increasingly capable AI systems without needing constant one-on-one human input. It emphasizes creating scalable benchmarks, metrics, and monitoring processes that reveal when a model behaves unexpectedly and how to compare models fairly as they scale.

Sam Bowman is the mentor whose work centers on developing scalable evaluation frameworks, designing robust benchmarks, and studying how model performance generalizes across tasks and distributions. This focus provides the practical tools and methods needed to assess and compare AI systems at scale, detect failure modes, and guide safe deployment, which is why he best fits this focus.

The other mentors are known for different angles within AI safety, such as adversarial robustness or other safety topics, rather than the specific practice of scalable evaluation and oversight.