AarXiv2017

Cost-complexity pruning of random forests

Kiran Bangalore Ravi, Jean Serra

Abstract

Random forests perform boostrap-aggregation by sampling the training samples with replacement. This enables the evaluation of out-of-bag error which serves as a internal crossvalidation mechanism. Our motivation lies in using the unsampled training samples to improve each decision tree in the ensemble. We study the effect of using the out-of-bag samples to improve the generalization error first of the decision trees and second the random forest by post-pruning. A prelimiary empirical study on four UCI repository datasets show consistent decrease in the size of the forests without considerable loss in accuracy. 1

A figure from Cost-complexity pruning of random forests — fig. from the paper

§ The Valyu brief

The paper, in five minutes

Read from the full text via Valyu Contents

Post‑pruning random forests using out‑of‑bag samples reduces forest size with little accuracy loss.

What’s new

i.Propose using out‑of‑bag samples to post‑prune decision trees in a random forest
ii.Apply cost‑complexity pruning to random forests
iii.Empirical evaluation on four UCI datasets showing size reduction with minimal accuracy loss

How they did it

The authors use the out‑of‑bag samples—data not selected in the bootstrap step—to perform cost‑complexity pruning on each tree after the forest is built. They then assess the pruned forest’s size and accuracy on four UCI benchmark datasets. The approach leverages the built‑in out‑of‑bag error estimate as an internal cross‑validation.

What they found

Consistent decrease in forest size across all four UCI datasets
No considerable loss in classification accuracy after pruning

§ The Valyu brief

Reading the full paper and taking notes. This takes a few seconds…

§ Ask this paper

Ask a question about this paper

Valyu reads the full text and answers from what the paper actually says.

§ Related work in other archives

Similar papers in all four archives

The same idea searched in arXiv, PubMed, bioRxiv and chemRxiv at once, so a method can lead you to where it is being applied.

AarXiv4 found

PPubMed4 found

BbioRxiv4 found

CchemRxiv2 found

Searching the other archives…