Random Forest vs Extra Trees: Two Kinds of Tree Randomness

"Extra Trees" does not mean more trees. It means extra randomness — and that single distinction changes how the ensemble learns, how fast it trains, and when you should reach for it.

If you have worked with random forests, you already know the core trick: grow many decision trees, each one slightly different, then average their predictions. The differences come from bootstrap sampling and from limiting each split to a random subset of features. That mechanism reduces correlated errors between trees, which is what makes the forest stronger than any single tree.

Extra Trees — short for extremely randomized trees — keeps both of those ideas and adds a third source of randomness. The question this article answers is simple: what does that extra randomness buy you, what does it cost, and how do you decide which ensemble to try first?

The Name Lies: Extra Randomness, Not More Trees

The most common misconception about Extra Trees is baked into its name. Beginners hear "extra" and assume the algorithm grows more trees than a random forest. It does not. The name comes from extremely randomized, and the randomization is the entire story.

Both ensembles average many independent trees. Both randomly sample features at each split. The difference is what happens inside the split decision.

A random forest, at each node, considers a random subset of features and searches for the best threshold within each candidate feature. It evaluates many possible cut points and picks the one that most reduces impurity.

Extra Trees does something lazier and more aggressive: for each candidate feature, it draws a single split threshold at random, evaluates only that threshold, and then picks the best feature among those randomly generated candidates.

That is the whole difference. One ensemble searches for good splits. The other rolls dice on splits and then chooses among the rolls.

Where Randomness Enters Each Ensemble

Let us make the mechanism precise, because the practical differences all flow from here.

Random forest randomizes in two places:

1. Each tree trains on a bootstrap sample — rows drawn with replacement from the training data.
2. Each node considers only a random subset of features when searching for the best split.

Extra Trees randomizes in three places:

1. Each tree trains on a sample of the data (with an important implementation caveat below).
2. Each node considers only a random subset of features.
3. For each candidate feature, the split threshold is drawn randomly instead of chosen by exhaustive search.

Notice what Extra Trees does not do: it does not abandon optimization entirely. After generating random thresholds for each candidate feature, it still compares those candidates and selects the best one. The algorithm adds randomness but keeps a layer of selection on top.

Here is a concrete picture. Imagine a feature with a thousand possible threshold values. A random forest evaluates many of them and picks the cut that best separates the classes. Extra Trees draws one threshold at random, measures how good that single cut is, and compares it against the randomly drawn cuts from the other candidate features. It never sees the other 999 possibilities.

There is also a subtle implementation difference you need to know about. In scikit-learn, RandomForestClassifier defaults to bootstrap=True, meaning each tree sees a different resampled version of the data. ExtraTreesClassifier defaults to bootstrap=False, meaning every tree trains on the full dataset. The Extra Trees variant can still work well without bootstrap sampling because the random thresholds already decorrelate the trees — but if you compare the two models out of the box, you are comparing more than just the split mechanism.

What Extra Randomness Buys: A Bias–Variance Tradeoff

The bias–variance tradeoff gives you the vocabulary to predict what this mechanism does before you ever run an experiment. But you have to be careful about which object you are describing: an individual tree, or the averaged ensemble.

Start with individual trees. A random threshold is rarely the genuinely best cut for that feature. Each individual Extra Tree is therefore somewhat less optimized than a random forest tree would be. The bias of each tree goes up.

Now consider the ensemble. Random thresholds make trees less correlated with each other. When trees disagree more, the averaged forest becomes more stable across different training sets. This is the same logic that makes random forests better than single decision trees, pushed one step further — and it tends to reduce the variance of the complete ensemble.

So the tradeoff runs in opposite directions, and it is easy to mix the levels:

• Per tree: Extra Trees usually has higher bias, because each split is a random draw rather than a careful search.
• Per ensemble: Extra Trees often has lower variance, because the trees are less correlated and their errors average out more cleanly.

The net balance is dataset-dependent. On many problems, the variance reduction wins, which is why Extra Trees often matches or slightly beats random forest accuracy. On datasets where finding the exact right split matters — where the signal is concentrated in a narrow threshold range — the bias cost becomes visible.

One more consequence worth noting: because Extra Trees does not fixate on the single best split, it tends to be more robust to noisy or irrelevant features. A random forest can waste effort repeatedly selecting and splitting on a noisy feature that happens to offer a tempting impurity reduction. Extra Trees treats every feature more suspiciously, which dilutes the influence of any one misleading column.

I want to be honest about what is established and what is not. The direction of the per-tree and per-ensemble tendencies follows from the algorithm's design. Which model wins on your particular dataset is an empirical question. Anyone who tells you Extra Trees always beats random forest is selling folklore, not evidence.

The Practical Payoff: Speed and a First Experiment

The most visible practical difference is training speed. Searching for the optimal threshold is the expensive part of building a decision tree. Every candidate threshold requires evaluating a split and measuring impurity. Extra Trees avoids most of that search — one random draw per candidate feature per node — so training often runs noticeably faster.

Notice the qualifier: often. Extra Trees still evaluates the randomly drawn thresholds it generates, and it still computes impurity for each candidate feature. The realized speed gap depends on your data shape, tree depth, the implementation, and whether training is parallelized. Treat speed as a tendency to measure, not a guarantee.

My practical rule is simple: when you are starting a new tabular problem, run Extra Trees first. It gives you a fast baseline, and on many problems the accuracy difference from random forest is small enough that you will not care. Then run a random forest and ask whether the extra split search actually bought you anything on your data. Sometimes it does. Often the difference is within the noise of cross-validation.

A Comparison Table: Random Forest vs Extra Trees at a Glance

Both are bagging-style ensembles: they average many independent trees. That distinguishes them from boosting, where each new tree is built to correct the errors of the previous ones.

Common Mistakes When Comparing the Two

Mistake 1: Assuming Extra Trees always wins. The accuracy gap between the two is often small and dataset-dependent. Treat the choice as an experiment, not a verdict.

Mistake 2: Forgetting the bootstrap default. If you compare scikit-learn's defaults, you are comparing bootstrap sampling and split randomness at the same time. If you want to isolate the split mechanism, set bootstrap=True on the Extra Trees model.

Mistake 3: Treating the choice as a one-time winner. Run both with cross-validation on your data. The model that wins on someone else's benchmark may lose on yours.

Mistake 4: Confusing Extra Trees with boosting. It is not a boosting method. It builds trees independently and averages them, exactly like a random forest. Gradient boosting is a different family with a different learning strategy.

Each of these mistakes reveals something about the mechanism. Extra Trees is not magic — it is a specific randomization strategy with predictable costs and benefits.

Your First Experiment: Choose a Test, Not a Winner

Here is what I want you to do next. Take a dataset you care about and run a two-pass comparison.

Pass 1: Defaults, for a practical choice. Run both ensembles with their default settings and the same number of trees. Use the same cross-validation splits and the same metric. Record the validation score and the training time for each. This tells you which model is the better default for your problem.

Pass 2: Controlled, to isolate the mechanism. Set bootstrap=True on the Extra Trees model so both ensembles use the same row sampling. Match the tree-size controls and the feature-subset setting as closely as you can. Use a fixed random seed for both. Now the main remaining difference is how each model chooses split thresholds. This tells you whether the extra randomness itself helped or hurt.

Then ask yourself two questions. Did the random forest's extra split search buy enough accuracy to justify the extra training time? And does the answer hold across multiple folds, or is it noise?

The results will vary across datasets and machines. That is the point. Your data is the only reliable judge of which randomization level fits its noise and feature structure.

The durable mental model is this: random forest and Extra Trees are not competing brands. They are two settings on the same dial — how much randomness you inject into the tree-building process. Random forest searches for good splits. Extra Trees rolls dice on splits and selects among the rolls. The right setting depends on your data, and the only way to know is to run both.

Once you have that comparison under your belt, the natural next step is gradient boosting — the contrasting ensemble that builds trees sequentially instead of averaging independent ones. But first, run the experiment. Let your data cast the vote.