From Random Forest to XGB, the History of Decision Tree Algorithm
The ID3(Iterative Dichotomizer 3) algorithm made by Ross Quinlan later develops
from C4.5 to C5.0. These algorithms use the degree of uncertainty in the set,
‘Entropy’, to classify sets.
This idea of entropy originated from Claude Shannon’s information entropy.
Other than entropy, these kind of classification exists throughout the field.
CHAID(Chi-squared Automatic Intersection Detection),
stated by J.A. Hartigan, used the chi-square,
Leo Breiman’s CART algorithm used the gini index.
This CART algorithm further developed to become the sensationally popular Random Forest (2001).
This algorithm was created by Leo Breiman and his student Adele Cutler.
The world famous AlphaGo used this algorithm.
Then, a new Boosting algorithm that enhanced accuracy by adapting sample data appeared in 1997.
This algorithm is called AdaBoost, made by Yoav Freund and Robert Schapire.
Today, XGBoost (eXtreme Gradient Boosting) of the Ensemble algorithm,
which combined several decision trees, is gaining popularity.
"1959 Dartmouth Summer Research Project"