Which pair of evaluation methods correctly characterizes tests used to assess robustness to distribution shift?

• A. Only in-distribution accuracy tests.
• B. Hyperparameter tuning is the primary method.
• C. Out-of-distribution tests and stress tests are used to evaluate robustness under distribution shift.
• D. User surveys determine robustness.

Answer: (C)

Robustness to distribution shift means evaluating how a model behaves when the data it encounters in the real world looks different from the data it was trained on. The most informative tests for this are out-of-distribution evaluations, where you measure performance on data from a different distribution or domain than the training data, and stress tests, which push the model with extreme or corrupted inputs to see how it holds up. Together, these tests reveal how well the model generalizes beyond the familiar training conditions and how it degrades under challenging scenarios. In-distribution accuracy tests only assess performance on data similar to what was seen during training and can miss weaknesses that appear when the distribution changes. Hyperparameter tuning may improve performance, but it’s not a direct robustness assessment under shift, and user surveys measure perceptions rather than actual reliability under shift.