Chapter 1: Machine Learning for Predictive Analysis

Extracting insights from data

The art of building and using models that make predictions based on patterns extracted from historical data

price prediction (businesses), dosage prediction (doctors), risk assessment (organizations), propensity modeling (predicting the likelihood or propensity of individuals to take different actions), diagnosis (doctors, engineers and scientists), document classification

In DA a prediction is the assignment of a value to any unknown variable. In everyday usage it has a temporal aspect, we predict what will happen in the future

in each case, a model is used to make a prediction to help make a decision AND a model is trained to make predictions based on a set of historical examples (machine learning is used to train these models)

Machine learning is an automated process that extracts patterns from data.

We use supervised machine learning to build the models used in predictive data analytics applications - They have labels/ classes/ events that provide us with feedback while learning

They automatically learn a model of the relationship between a set of descriptive features and a target feature based on a set of historical examples (instances)

training instance

training dataset

when there are no instances in the dataset for which the model does not make a correct prediction

automate the process of learning a model that captures the relationship between the descriptive features and the target feature in a dataset.

• When dealing with large databases there will likely be noise • The training set represents only a small sample of the possible set of instances in the domain.

An ill-posed problem is a problem for which a unique solution cannot be determined using only the information that is available

A single consistent model cannot be found based on the sample training dataset alone

• The ability to make predictions for queries that are not present in the data • A prediction model that makes the correct predictions for these queries captures the underlying relationship between the descriptive and target features and is said to generalize well

Finding the predictive model that generalizes best

A set of assumptions that defines the model selection criteria of a machine learning algorithm

• Restriction bias -Preference bias

• It constrains the set of models that the algorithm will consider during the learning process • Similar to choosing your go-to study method • Tells us what our model is able to represent

-It guides the learning algorithm to prefer certain models over others - Choosing out convergence/satisfaction mechanism - I like group study but prefer to be the leader than weakest link - Algorithm’s belief about what makes a good hypothesis

• In multivariable linear regression with gradient descent we only consider models that produce description based on a linear combination of the descriptive features • In Iterative Dichotomizer 3 we only consider tree-like prediction models where each branch encodes a sequence of checks on individual descriptive features

• In MLR with GD we linearly combine the descriptive features using only weights that were found though our gradient descent approach • In ID3 we are preferring shallower (less complex) trees over larger/deeper trees

Without it we could only perform memorization of our training dataset without generalization capacity