[Machine Learning] Ensemble Learning | Random Forest

[ Ensemble Learning ]

Ensemble Learning : when we take multiple algorithms or the same algorithm multiple times and we put them together that results in a much more powerful version.

 

=> It helps to improve the performance and accuracy of machine learning algorithms. 

=> Putting multiple ML algorithms together to create one bigger ML algorithm that leverages many other ML algorithms.

 

Type of Ensemble Learning

1. Voting

Hard Voting : decide final class with majority vote from multiple classifier 

 

Soft Voting : decide final class with average of class probabilities from multiple classifier 

> Use predict_prob() for each class's avg 

 

==> In general, soft voing's prediction is more adapted than hard voings'

 

 

 

2. Bagging (Bootstrap aggregating) 

- Random Forest 

 

 

3. Boosting

- data sampling(randomness)

 

 

4. Stacking 

 

 

 

 

 

[ Random Forest ]

Random Forest : a version of Ensemble Learning  (Bagging), based on Decision Tree algorithm 

 

=> Each individual decision tree in a Random Forest is trained and performs predictions independently. The final prediction is determined by averaging the predictions or using majority voting from all the trees.

 

 

 

Steps of Random Forest

 

Step 1)

Pick at random K Data points from the training set

 

Step 2)

Build the Decision Tree associated to these K data points

 

Step 3)

Choose the number Ntree of trees want to build and repeat STEP 1&2

 

Step 4)

For a new data point, make each one of Ntree tree predict the value of Y to for the data point in question. and assign the new data point the average across all of the predicted Y values.