intermediate
9 min read
How Bootstrap Resampling Builds an Interval for a Statistic
You have a thousand bootstrap replicates sitting in a NumPy array. You have a histogram. What you do not have is the two numbers you actually need to…
Read tutorialconcept
Uncertainty Estimation
Quantifying or communicating uncertainty around predictions, metrics, probabilities, or learned conclusions.
Tagged articles
Uncertainty Estimation articles
13 articles in this tag.
intermediate
12 min read
Bootstrap Resampling in Machine Learning: Estimate Uncertainty, Not Certainty
Your model scores 0.82 accuracy on the test set. That feels like a fact—something solid you can report and defend. Run the experiment again on a different…
Read tutorial
intermediate
9 min read
Estimate a Statistic’s Uncertainty With a Reproducible Bootstrap Experiment
You have one sample, one number, and no honest way to say how much that number would move if you collected the data again. The bootstrap does not create…
Read tutorial
intermediate
11 min read
Decompose the Brier Score Into Reliability, Resolution, and Uncertainty
Two forecasters can post the same Brier score and deserve opposite verdicts. One is honest but says almost nothing. The other is sharp but lies about its…
Read tutorial
advanced
9 min read
How Certain Is a Cross-Validation Score? Derive the Limits of Fold-Based Uncertainty
Two models. Mean cross-validation scores of 0.842 and 0.847. The instinct is immediate: the second one wins.
Read tutorial
intermediate
11 min read
Gaussian Mixture Models for Clustering: Soft Groups With Probabilities
Picture a point sitting exactly between two well-separated blobs of data. K-means will force it into one group or the other, even when the data could…
Read tutorial
intermediate
10 min read
Inspect Gaussian Mixture Assignments With a Scikit-Learn Experiment
A hard label is a decision. A posterior probability is a confession about how close that decision was.
Read tutorial
intermediate
8 min read
Prediction Intervals for Regression: Put Uncertainty Around a Number
A regression model hands you one number. The person acting on that number needs to know how much weight it can bear. A prediction interval is how you show…
Read tutorial
intermediate
10 min read
Probability Calibration in Machine Learning: Can You Trust the Score?
A model can rank every case perfectly and still hand you probabilities you cannot spend. Calibration is the difference between a score that orders risk and…
Read tutorial
intermediate
9 min read
Quantile Regression: Derive the Pinball-Loss Objective
Your model keeps under-predicting the values you actually care about, and averaging the errors away does not fix it.
Read tutorial
intermediate
9 min read
Reproducible Machine Learning Experiments: Make Results Comparable
You tune a model, retrain, and the score moves. The question is whether that movement came from your edit or from luck. Most beginners cannot tell the…
Read tutorial
intermediate
10 min read
Compare Raw and Calibrated Probabilities With Scikit-Learn
A model can rank every sample correctly and still lie about the number attached to each one. This experiment isolates that lie, then measures whether a…
Read tutorial
advanced
11 min read
Split Conformal Prediction: Derive the Rank-Based Coverage Guarantee
A prediction interval built from a model's own residuals has no guarantee at all. Split conformal prediction replaces that hope with a finite-sample…
Read tutorial