Thread by @kdpsinghlab

Since my TL is filled with love letters to regression, let's talk about the beauty of random forests. Now maybe you don't like random forests are or don't use them or are afraid of using them due to copyright infringement.

Let's play a game of: It's Just a Random Forest (IJARF).

Decision trees: random forest or not?

Definitely a single-tree random forest with mtry set to the number of predictors and pruning enabled.

IJARF.

Boosted decision trees: random forest or not?

Well, if you weight the random forest trees differently, keep their depth shallow, maximize mtry, and grow them sequentially to minimize residual error, then a GBDT is just a type of random forest.

IJARF.

Linear regression: random forest or not?

It's commonly assumed that you need to have constant values in the leaves of random forest decision trees. But why? Recursive partitioning can be used to fit local regression models in the tree leaves.

IJARF.

pdfs.semanticscholar.org/36e9/825201bc3…

Neural networks: random forest are not?

With the right loss function and backprop strategy, a neural network is essentially a random forest with a different splitting strategy, relying on a global minimum error as opposed to the Gini index.

IJARF.

cv-foundation.org/openaccess/con…

For more on the coolness of RFs, check out this @microsoft technical report.

microsoft.com/en-us/research…

Chapter titles:
- Human body tracking in Microsoft Kinect for XBox 360
- Specializing ... forest model for regression
- Generative models
- Transduction forests

IJARF!!! 🤯