advanced
11 min read
Derive AdaBoost’s Weight Updates From Exponential Loss
AdaBoost's algorithm listing looks like three separate rules stitched together: fit a weak learner, compute a coefficient, multiply the sample weights by…
Read tutorialdifficulty
Advanced
Articles explicitly planned for readers at the advanced difficulty level.
Tagged articles
Advanced articles
8 articles in this tag.
advanced
9 min read
How Certain Is a Cross-Validation Score? Derive the Limits of Fold-Based Uncertainty
Two models. Mean cross-validation scores of 0.842 and 0.847. The instinct is immediate: the second one wins.
Read tutorial
advanced
13 min read
Derive the EM Algorithm for Gaussian Mixture Models
You can write the Gaussian mixture density in a single line. The trouble starts the moment you take the log.
Read tutorial
advanced
11 min read
Gradient Boosting as Additive Function Fitting: Derive the Pseudo-Residuals
Fit the next tree to the error. That instruction works beautifully for squared loss and then quietly falls apart the moment you switch to log loss. The…
Read tutorial
advanced
11 min read
When Is Least Squares Unbiased? Assumptions Behind OLS Guarantees
A coefficient can be unbiased on average and still be wrong in your one dataset. Worse, it can be unbiased and still predict badly, and unbiased and still…
Read tutorial
advanced
8 min read
Why Averaging Trees Helps: Derive the Random-Forest Variance Effect
Everyone repeats that a forest reduces variance. Almost nobody writes down the formula that governs the reduction — and that formula contains a term that…
Read tutorial
advanced
11 min read
Split Conformal Prediction: Derive the Rank-Based Coverage Guarantee
A prediction interval built from a model's own residuals has no guarantee at all. Split conformal prediction replaces that hope with a finite-sample…
Read tutorial
advanced
10 min read
How the SVM Dual Makes Kernel Substitution Possible
The primal SVM asks for a weight vector in feature space. The trained model never hands you one. It hands you a sum over training points instead — and that…
Read tutorial