Calibration in machine learning refers to how well a model's predicted probabilities of outcomes reflect the true probabilities observed in reality. A classifier is well-calibrated if, for example, among all instances to which it assigns a 70% probability of being positive, about 70% of those instances are actually positive. This concept is important especially for probabilistic classifiers: beyond accuracy, we care that a predicted confidence of 0.9 truly signifies roughly a 90% chance of the event. Poor calibration can manifest as over-confidence or under-confidence, meaning the predicted probabilities are not aligned with outcomes. In practice, calibration is often evaluated with calibration curves or reliability diagrams and quantified by metrics like the Brier score or expected calibration error. Techniques such as Platt scaling (sigmoid calibration) or isotonic regression can be applied after training to adjust a model's output probabilities for better alignment with observed frequencies. These calibration methods learn a mapping from the classifier's raw scores to calibrated probabilities, usually using a hold-out validation set. Calibration is crucial in applications like medical diagnosis or weather forecasting where decision-making relies on trustworthy confidence estimates.
Data Selection & Data Viewer
Get data insights and find the perfect selection strategy
Learn MoreSelf-Supervised Pretraining
Leverage self-supervised learning to pretrain models
Learn MoreSmart Data Capturing on Device
Find only the most valuable data directly on devide
Learn More