Math & Statistics - Machine Learning

Task T: example - playing checkers Performance measure P: example - percentage of games won agains opponents Training experience E: playing practice games against itself.

Any hypothesis found to approximate the target function well over a sufficiently large set of training examples will also approximate the target function well over other unobserved examples.

Acquiring the definition of a general category by samples of positive and negative training examples of the category.

Concept learning can be viewed as the task of searching through a large space of hypotheses implicitly defined by the hypothesis representation. The goal of this search is to find the hypothesis that best fits the training examples.

• FIND-S 2. Candidate Elimination

non-parametric approaches completely avoid this danger, since essentially no assumption about the form of f is made.

x

Categorical

We tend to refer to problems with a quantitative response as regression problems

qualitative response are often referred to as classification problems

In order to evaluate the performance of a statistical learning method on a given data set, we need some way to measure how well its predictions actually match the observed data. That is, we need to quantify the extent to which the predicted response value for a given observation is close to the true response value for that observation. In the regression setting, the most commonly-used measure is the mean squared error (MSE).

Building a statistical model to predict, or estimating, an output based on one or more inputs.

Learn relationships and structure from data, not trying to predict an output number.

Quantitative problems output a number, while classification problems output a classification.

Século 19 - Regressão Linear - Legendre e Gauss - Method of Least Squares 1940 - Logistic Regression - vários autores 1970 - Generalized Linear Models - Nelder e Wedderburn 1985 - Classification and Regression Trees - Breiman, Friedman, Olshen and Stone 1986 - Generalized Additive Models (non-linear models) - Hastie e Tibshirani

n - represent the number of distinct points, or observations, in our sample. p - denote the number of variables that are available for use in making predictions. Color fonte (red) - variables i - index the samples of observations. (from 1 to n) j - index the variables (from 1 to p) X - denote a n x p matrix whose (i, j)th element is xij. Like a spreadsheet. Xt - transpose of matrix. Yi - denote the ith observation of the variable on which we wish to make predictions.

Random error term

• Reducible error and 2. Irreducible error

The quantity may contain unmeasured variables that are useful in predicting Y : since we don’t measure them, f cannot use them for its prediction.

• Prediction 2. Inference

Descobrir o relacionamento entre as diferentes variáveis e o resultado (Y).

Gera uma previsão (Y) baseado em input X.

Linear Regression Logistic Regression