What does calibration mean in probability estimation, and why is it important for model safety?

• A. Calibration means predicted probabilities align with observed frequencies; well-calibrated models avoid overconfident unsafe outputs.
• B. Calibration means the model is fast.
• C. Calibration means data quality.
• D. Calibration means the model's outputs are deterministic.

Answer: (A)

Calibration means predicted probabilities align with observed frequencies. In a well-calibrated model, if you group all predictions that have a given probability p, the actual occurrence rate of the event should be close to p. This matters for model safety because trusted confidence levels drive safe decisions: when a risk is labeled as, say, 85% likely, that 85% should reflect real outcomes; otherwise the model becomes overconfident or miscalibrated, leading to unsafe actions or missed warnings. Calibration also lets you set decision thresholds with predictable trade-offs between false alarms and misses, which is essential for reliable safety performance. The other options describe properties that don’t guarantee properly estimated probabilities: speed, data quality, or determinism.