Go beyond one score. Measure the errors that matter, inspect where models fail, and reason about imbalance, thresholds, calibration, drift, and uncertainty.
Tutorials
Follow the learning path in order or jump directly to the concept or workflow you need.
Your model just aced its training data. Ninety-eight percent accuracy. Beautiful curves. Then you hand it new examples, and it stumbles like it never saw…
Read tutorial
A model that scores 90% accurate can still fail at the exact job it was built for. The headline number looks impressive, but it may be hiding a model that…
Read tutorial
A model that predicts "no fraud" for every transaction can score 98% accuracy while catching zero fraud. The number feels trustworthy. It is not. Accuracy…
Read tutorial
Two models can post nearly identical test scores and still fail for opposite reasons. One misses because it never learned the pattern. The other misses…
Read tutorial
Your spam filter just flagged an email with a score of 0.55. Is it spam? The model isn't telling you—it's asking you to decide. Somewhere between the…
Read tutorial
A model can rank every case perfectly and still hand you probabilities you cannot spend. Calibration is the difference between a score that orders risk and…
Read tutorial
"The model says education matters" sounds like a complete explanation. It is not. It is the beginning of a much narrower claim: the model leaned on…
Read tutorial
If you are like most beginners, you start guessing. Maybe more data will fix it. Maybe a bigger model. Maybe fancier features. You try one, see little…
Read tutorial
You train a fraud detector on a dataset where only 0.5% of transactions are fraudulent. The ROC AUC comes back at 0.95. You feel great. Then you look at…
Read tutorial
You've added features. You've tuned hyperparameters. You've tried a more flexible model. And still, validation performance sits at the same stubborn…
Read tutorial
A regression model reports a solid R-squared, so you call it done. Then you plot the predictions against the actual values and notice something…
Read tutorial
Your model aced the test set. Clean evaluation, strong metrics, confident you. Then you deploy it, and the real world quietly moves on without telling you.
Read tutorial
You improved the model twice. The validation score went up, then up again. Now every change you try makes it worse or does nothing. You are not out of…
Read tutorial
A regression model hands you one number. The person acting on that number needs to know how much weight it can bear. A prediction interval is how you show…
Read tutorial
Your model reports 0.91 accuracy. That number is probably correct, and it is probably useless for deciding what to fix next.
Read tutorial
A model can rank every sample correctly and still lie about the number attached to each one. This experiment isolates that lie, then measures whether a…
Read tutorial
A respectable R² can sit on top of a model that is wrong in a structured way. The score summarizes average fit; it says nothing about where the fit fails.…
Read tutorial
A model returns 0.4. The default cutoff turns that into "negative," and nobody asked whether that was the right call.
Read tutorial
Retrain the same model on a different random split and the predictions move. Nobody changed the code. Nobody changed the hyperparameters. The wobble is not…
Read tutorial
Your model is live. Labels arrive in three weeks. Right now, the only thing you can actually see is the input stream — and it looks different from what you…
Read tutorial
A prediction interval built from a model's own residuals has no guarantee at all. Split conformal prediction replaces that hope with a finite-sample…
Read tutorial
A model with a respectable AUC can still make the wrong decision every single day. The scores are fine. The cutoff is the problem.
Read tutorial
You have seen the classic two-line plot. One line starts high and drifts down. The other starts low and climbs. Somewhere on the right, they meet, and the…
Read tutorial
You already know noisy labels hurt models. Knowing it changes nothing. The moment you flip a known fraction of labels yourself, retrain, and watch the…
Read tutorial
You fit a model, call a feature-importance attribute, and get a tidy ranked bar chart. It looks like a verdict. It is not. That chart is a measurement of…
Read tutorial
You resample your data to fix the imbalance, rerun the same model, and watch ROC-AUC barely move while PR-AUC collapses. Nothing about the model changed.…
Read tutorial
A model can keep the same weights, the same features, and the same threshold, and still report precision 0.9 on your test set and precision 0.3 in…
Read tutorial
Two forecasters can post the same Brier score and deserve opposite verdicts. One is honest but says almost nothing. The other is sharp but lies about its…
Read tutorial
A model degrades in production. Three engineers offer three diagnoses: "the inputs drifted," "the class balance changed," "the meaning changed." All three…
Read tutorial
You fit a line, plot the residuals, and they look like a clean cloud. No funnel, no curve, no obvious outlier. It is tempting to read that plot as a…
Read tutorial
Your model passed validation. Then it went to production and started predicting one class far more often than it should. The features look normal. The…
Read tutorial