General

Gradient descent algorithm that increments the parameters using only single observations at a time. More efficient than batch gradient descent, especially with large datasets.

Gradient descent algorithm in which it is required to scan through the entire training set before taking a single step.

A variant of traditional linear regression that uses only local data points around Xi to predict Yi

Incorrectly rejecting the null hypothesis in favor of the alternative hypothesis when the null is true. Same as alpha, set at the beginning of the experiment.

Failing to reject the null hypothesis when it is false Also known as beta. Note that power is (1 - beta)

A/B testing, also known as split or bucket testing, is a user experience research method that compares two or more versions of content to determine which one performs best A/B testing involves randomly assigning visitors to see either a control (A) version or a variant (B) version of a page or content. The performance of each version is then measured based on key metrics, such as the number of conversions or visitors who took the desired action.

Simple: Machine learning model that uses a hyperplane to differentiate and classify different groups of data Detailed: SVM identified an appropriate hyperplane by attempting to maximize the margins between points between the closest points of each class to boundary. If data that cannot be separated linearly, use transformations to map data into high dimensions.

A mechanism that serves to reduce the overfitting of maximum margin classification by penalizing misclassifications

A tradeoff in machine learning models where you have the choice of reducing bias (how well a model fits a specific set of data) vs. reducing variance (how much performance of a mode varies across many datasets).

TP / (TP + FP) Measures the accuracy of positive predictions (but not necessarily identifying all of them).

TP / (TP + FN) Measures completeness of positive predictions

2 / ((1/Precision) + 1/Recall)) Harmonic mean of precision and recall, ranging between 0 and 100%

Plots true positive rate (recall) against false positive rate. A good ROC curve goes toward the top left of the chart. X = FPR Y = TPR

Proportion of negative instances that are incorrectly classified as positive (i.e. false positive) FP / (FP + TN)

Check this

Check this

A way of regularizing a model by stopping training once validation error reaches a minimum

Also known as multinomial logistic regression. Classification with multiple classes. For each instance x, assigns a score s(x) for each class k, then estimates probability by applying a soft max function. Soft max function is as followed: Exp(s(x)) / summation(exp(s(x)))

Loss function used to measure difference between predicted and true probability distributions. Penalizes low probability on true labels significantly. -1/m ∑ ∑y log (p(k)) Essentially the mean of the -log(estimated probabilities).

Number of correctly classified instances / number of all classified instances

Proportion of positive instances that are correctly classified as positive (i.e. true positive) TP / (TP + FN)

True Negative Rate Proportion of negative instances that are correctly classified as negative (i.e. true negative) TN / (TN + FP)

An algorithm that minimizes a particular function (in ML the loss function) by taking small steps in the direction of the steepest descent for that function. Step 1: Take the derivative of the loss function for each parameter (i.e. take the gradient of the loss function). Step 2: Initialize parameters with random values Step 3: Plug parameters into the partial derivatives (gradient) Step 4: Calculate step sizes (Calculated slope from step 3 * learning rate) Step 5: Calculate the new parameters (New = Old - Step Size) Step 6: Repeat 4-5 until convergence

Step 1: Shuffle data into equally sized blocks (folds) Step 2: For each fold k, train model on all data except fold i, and evaluate validation error using the remaining fold i. Step 3: Average the validation errors from step 2 to get estimate of the true error.