advanced
11 min read
Derive AdaBoost’s Weight Updates From Exponential Loss
AdaBoost's algorithm listing looks like three separate rules stitched together: fit a weak learner, compute a coefficient, multiply the sample weights by…
Read tutorialuse-case
Classification
Supervised prediction of discrete class labels or class-related scores, including binary, multiclass, multilabel, and ordinal formulations.
Tagged articles
Classification articles
46 articles in this tag.
intermediate
9 min read
Build and Evaluate a Stacking Classifier With Scikit-Learn
A stacking classifier can score beautifully on your test set for a reason that has nothing to do with skill: the meta-model was trained on predictions the…
Read tutorial
intermediate
9 min read
Choose a Classification Threshold With a Cost-Based Scikit-Learn Experiment
A model with a respectable AUC can still make the wrong decision every single day. The scores are fine. The cutoff is the problem.
Read tutorial
intermediate
9 min read
Classification Thresholds Explained: Turn Scores Into Decisions
Your spam filter just flagged an email with a score of 0.55. Is it spam? The model isn't telling you—it's asking you to decide. Somewhere between the…
Read tutorial
intermediate
7 min read
Compare Random Forests and Extra Trees With a Controlled Experiment
One score is a snapshot. A distribution is a measurement. If you fit Random Forest and Extra Trees once, watch Extra Trees edge ahead by 0.01 accuracy, and…
Read tutorial
intermediate
9 min read
Derive the Cost-Sensitive Classification Threshold
A model returns 0.4. The default cutoff turns that into "negative," and nobody asked whether that was the right call.
Read tutorial
intermediate
11 min read
How Cumulative-Link Models Represent Ordered Outcomes
A satisfaction rating of 4 when the truth was 5 is not the same kind of mistake as predicting 1. Most models cannot tell the difference. This one can — but…
Read tutorial
beginner
10 min read
A Decision Tree Experiment: Inspect Splits and Observe Overfitting
A decision tree can score 100% on the data it was trained on and still be wrong about almost everything else. That gap is not a bug. It is the lesson.
Read tutorial
intermediate
10 min read
How Decision Trees Score Classification Splits: Gini, Entropy, and Gain
You fit a tree, print it, and there it is: the root splits on worst radius, not mean texture. The library made a choice. You can't say why.
Read tutorial
intermediate
8 min read
Deriving the Logistic Regression Classification Objective
You can fit a logistic regression in one line and read its coefficients in the next. What almost nobody shows you is where the loss function comes from.…
Read tutorial
intermediate
11 min read
Compare Feature Hashing and a Vocabulary on Sparse Text
Two representations. One corpus. One split. One classifier. The only honest way to know what hashing costs you is to measure it against the vocabulary you…
Read tutorial
intermediate
8 min read
Generative vs Discriminative Models: Model the Data or the Decision
Naive Bayes and logistic regression can classify the same dataset with nearly the same accuracy, yet they behave very differently on small samples, missing…
Read tutorial
intermediate
10 min read
Experiment With Gradient Boosting: Learning Rate, Trees, and Validation
A single accuracy number tells you almost nothing. A controlled experiment tells you whether that number means anything at all.
Read tutorial
intermediate
11 min read
Imbalanced Classification: Why Accuracy Can Hide the Failure
A model that predicts "no fraud" for every transaction can score 98% accuracy while catching zero fraud. The number feels trustworthy. It is not. Accuracy…
Read tutorial
beginner
7 min read
Test KNN Neighbor Count and Feature Scaling With Scikit-Learn
Same data. Same code. Two accuracy scores that disagree, and nothing in the output explains why.
Read tutorial
beginner
12 min read
Measure the Effects of Label Noise With a Controlled Classifier Experiment
You already know noisy labels hurt models. Knowing it changes nothing. The moment you flip a known fraction of labels yourself, retrain, and watch the…
Read tutorial
intermediate
9 min read
Simulate Label Shift and Test Prior-Probability Correction
Your model passed validation. Then it went to production and started predicting one class far more often than it should. The features look normal. The…
Read tutorial
intermediate
10 min read
Linear Discriminant Analysis: Derive the Gaussian Classifier
You fit LinearDiscriminantAnalysis, plot the boundary, and get a straight line. Then someone asks why it is straight, and the honest answer is "because the…
Read tutorial
intermediate
12 min read
Logistic Regression for Classification: From Scores to Probabilities
A logistic regression prediction is not a label. It is a probability—and the label is a decision you make on top of that probability.
Read tutorial
beginner
7 min read
Fit and Inspect Logistic Regression With Scikit-Learn
You call predict(), and you get back a tidy row of zeros and ones. Clean. Decisive. And completely silent about the thing you actually wanted to see: the…
Read tutorial
beginner
9 min read
Machine Learning Baselines: The Simple Prediction Your Model Must Beat
You train your first classifier. It hits 85% accuracy on the test set. You feel great—until you realize that a rule which ignores your data entirely would…
Read tutorial
beginner
11 min read
Machine Learning Evaluation Metrics: Choose the Score That Matches the Cost
A model that scores 90% accurate can still fail at the exact job it was built for. The headline number looks impressive, but it may be hiding a model that…
Read tutorial
beginner
9 min read
Multiclass Classification Explained: Predict One of Many Classes
Most people assume multiclass classification is just binary classification with more labels. It is not. The real problem is that many classical models only…
Read tutorial
intermediate
9 min read
Derive Softmax Probabilities and the Multiclass Cross-Entropy Objective
A multiclass model hands you K raw scores. The obvious move — divide each by their sum — collapses the first time a score goes negative. Here is the fix,…
Read tutorial
intermediate
11 min read
Multilabel Classification Explained: When One Example Has Several Answers
Most classification tutorials teach you to pick one answer. A news article is politics or finance. A movie is action or comedy. A support ticket is billing…
Read tutorial
beginner
9 min read
Multioutput Prediction: When One Example Has Several Targets
You open a dataset, and the last few columns all look like answers. One column says the fruit is an apple. The next says it is red. A third says it is…
Read tutorial
intermediate
11 min read
Naive Bayes: Derive the Classification Score From Bayes' Rule
The intuition article told you Naive Bayes "combines prior and evidence." Then you tried to write $P(x_1, x_2, \dots, x_n \mid y)$ for real data, and the…
Read tutorial
beginner
11 min read
Naive Bayes Explained: How Simple Probability Can Classify Well
Most classifiers earn their keep by capturing relationships between features. Naive Bayes does the opposite: it assumes those relationships don't exist.…
Read tutorial
beginner
9 min read
Build a Sparse-Text Naive Bayes Classifier With Scikit-Learn
The most dangerous bug in a beginner text classifier is not in the model. It is in the order of operations. Fit your vectorizer on the whole dataset before…
Read tutorial
intermediate
10 min read
Why the Mean, Median, or Class Prevalence Is the Best Constant Baseline
You build a baseline, get a number, and have no idea whether that number is the best a constant can do or just a habit copied from a tutorial. Here is the…
Read tutorial
intermediate
11 min read
Ordinal Classification Explained: Predict Categories That Have an Order
A rating scale looks like it should be either a multiclass problem or a regression problem. Both instincts quietly discard the information that matters…
Read tutorial
intermediate
12 min read
How Class Prevalence Changes Precision and the PR Baseline
A model can keep the same weights, the same features, and the same threshold, and still report precision 0.9 on your test set and precision 0.3 in…
Read tutorial
intermediate
10 min read
Probability Calibration in Machine Learning: Can You Trust the Score?
A model can rank every case perfectly and still hand you probabilities you cannot spend. Calibration is the difference between a score that orders risk and…
Read tutorial
beginner
11 min read
Regression vs Classification: Identify the Prediction Problem First
You have data. You have a prediction goal. And you're stuck on a question that feels like it should be simple: should I use regression or classification?
Read tutorial
intermediate
10 min read
Compare ROC and Precision–Recall Curves as Prevalence Changes
You resample your data to fix the imbalance, rerun the same model, and watch ROC-AUC barely move while PR-AUC collapses. Nothing about the model changed.…
Read tutorial
intermediate
11 min read
ROC vs Precision–Recall Curves: Which Classifier Plot Should You Trust?
You train a fraud detector on a dataset where only 0.5% of transactions are fraudulent. The ROC AUC comes back at 0.95. You feel great. Then you look at…
Read tutorial
intermediate
10 min read
Run a Scikit-Learn Error Analysis: Find Slices and Choose a Next Experiment
Your model reports 0.91 accuracy. That number is probably correct, and it is probably useless for deciding what to fix next.
Read tutorial
beginner
10 min read
Build and Diagnose a Multiclass Classifier With Scikit-Learn
A model can score 94% accuracy and still be quietly failing one class. Here is how to see it.
Read tutorial
intermediate
10 min read
Compare Raw and Calibrated Probabilities With Scikit-Learn
A model can rank every sample correctly and still lie about the number attached to each one. This experiment isolates that lie, then measures whether a…
Read tutorial
beginner
10 min read
Multiclass vs Multilabel vs Multioutput: Frame the Prediction Problem
You have your features. You have your target. You are about to reach for a classifier — and then you freeze on a question that feels too basic to ask out…
Read tutorial
beginner
12 min read
Supervised vs Unsupervised Learning: How the Data Changes the Task
You have heard both terms. You can even recite rough definitions. Then you open your own dataset, stare at the columns, and freeze. Is this supervised or…
Read tutorial
intermediate
12 min read
Support Vector Machines: Derive the Maximum-Margin Classifier
Most explanations of support vector machines stop at the picture: two clouds of points, a line between them, a gap on either side. That picture is correct,…
Read tutorial
intermediate
8 min read
Test SVM Scaling, Kernels, and Regularization With Scikit-Learn
Same data. Same SVC call. One run scores 0.85, the next 0.55, and the only thing that changed was whether you scaled the features first.
Read tutorial
intermediate
10 min read
Support Vector Machines Explained: Margins, Kernels, and Tradeoffs
Most machine learning models are hoarders. They keep every training point around and let each one vote on the prediction. A support vector machine is the…
Read tutorial
advanced
10 min read
How the SVM Dual Makes Kernel Substitution Possible
The primal SVM asks for a weight vector in feature space. The trained model never hands you one. It hands you a sum over training points instead — and that…
Read tutorial
intermediate
10 min read
Voting and Stacking Ensembles: Combine Different Models Without Losing the Plot
More models do not automatically mean a better model. An ensemble only wins when its members make different mistakes—and the way you combine them…
Read tutorial