This short lesson summarizes key takeaways from section 31

You will be able to:

• Understand and explain what was covered in this section
• Understand and explain why this section will help you become a data scientist

The key takeaways from this section include:

• Probably Approximately Correct (PAC) learning theory provides a mathematically rigorous definition of what machine learning is
• The PAC is a learning model which is characterized by learning from examples
• Decision trees can be used for both categorization and regression tasks
• They are a powerful and interpretable technique for many machine learning problems (especially when combined with ensemble methods)
• Decision trees are a form of Directed Acyclic Graph (DAG) - you traverse them in a specified direction, and there are no "loops" in the graphs to go backwards
• Algorithms for generating decision trees are designed to maximize the information gain from each split
• A popular algorithm for generating decision trees is ID3 - the Iterative Dichotomiser 3 algorithm
• There are a range of pruning hyperparameters for decision trees to reduce overfitting - including maximum depth, minimum samples leaf with split, minimum leaf sample size, maximum lead nodes and maximum features
• CART (Classification and Regression Trees) trees can be used for regression tasks