Introduction to Boosting Using Decision Trees

TreeNet stochastic gradient boosting is Stanford University Professor Jerome Friedman's latest advance in data mining methodology. In TreeNet, classification and regression models are built up gradually through a potentially large collection of small trees, each of which improves on its predecessors through an error-correcting strategy. Although each tree may have only one split, the full model can be extraordinarily accurate. The final model takes the form of a series expansion in which every term is a (small) tree.

TreeNet improves over conventional boosting in that:

• It is relatively impervious to errors in the target, such as mislabeling
• It is strongly resistant to overfitting
• It generalized well to future data

In-Depth Discussion of TreeNet Theory

Understand the theory of TreeNet with discussions focused on:

• Least squares regression
• Robust regression
• Logistic regression
• Variable importance
• Classification
• Poisson regression and survival modeling

TreeNet in Action

Explore SPM's unique modeling automation capabilities while running multiple data sets on both GUI and Non-GUI interfaces, and the advantages and disadvantages to both. We'll introduce the CART component of SPM, and explain:

• Building models in SPM
• Setting control parameters
• Interpreting output
• Variable importance
• Introduction to battery shaving in SPM using TreeNet

Interpreting TreeNet Models and Interaction Detection

Interaction detection is the detection and reporting component of TreeNet using Interaction Control Language (ICL).

You will understand:

• How to shape the structure of interactions
• How to impose different interactions in TreeNet models
• Dependency plots and how they are utilized

Modern Approaches To Regularized Regression

Generalized Path Seeker (GPS) is the most recent advance in regularized regression. This technology offers high-speed LASSO-style regression for extreme data set configurations with upwards of 100,000 predictors and possibly very few rows. Such data sets are commonplace in gene research and it is both supremely fast and efficient.

• Application using examples in data sets
• How GPS is implemented in SPM
• Command line operation of SPM
• Comments on parallel processing

Linking Engines

Explore how to:

• Combine TreeNet’s power of transformation and variable selection with GPS
• Identify the most influential trees in GPS with ISLE (Importance Samples Learning Ensembles)
• Use Rulefit to identify the most influential nodes and rules in a TreeNet model

Loose Ends and Application

Q&A with the experts for further discussion and apply SPM to your own data sets.