Advanced Machine Learning

Deep learning is a subset of machine learning that has become increasingly popular in recent years due to its performance along with improved computer speed.

• Take our inputs and multiply each input by a weight. • Add the results from step 1 together. • Add a value (called bias) to the result from step 2 • Pass result from step 3 through an activation function. • The result from step 4 is then passed to the next layer, where the processed is repeated.

One unit that process incoming information and produces a single output number

A collection of nodes working in parallel

The first set of nodes that work on the features of the input sample

The last set of nodes that output a prediction

The processing and passing of information forward through all layers to produce an output prediction

The difference between the predictions of all samples and their true labels

The process of updating the weights of each node to reduce the cost function

The process of completing one forward Propagation step on each sample in the training set and updating the weights of each node with backward Propagation is called an epoch

They can create non-linear functions to predict class or values

The sigmoid function maps all output values of the node to a value between 0 and 1

Tanh maps the output of a node to a value between -1 and 1, as shown above on the y axis of the second plot

It maps all negative outputs from a node to 0 and returns all positive outputs as is (linear function)

The exception is the output layer. Specific activation functions are required for a model to produce specific kinds of outputs

If the model should have a continuous output that can produce positive or negative. A Linear activation function simply returns the value that was passed to it. g(z) =z.

If the model should only output 0 or a 1, such as in binary classification problems

If the model should return the most likely candidate from a finite list of options, such as a multiclass classification problem

The prediction of a model for all samples in X.

The combination of all of a model’s errors on all samples

The difference between the predictions of all samples and their true labels

Changing weights to reduce the cost function

Keras is an API that acts as an interface for tensorflow, a popular choice for building networks, especially as you are first learning

Is is related to underfitting