Sem2 Week 3

Assigning an input to one of a finite set of categories

y = 1 if x > t, y = 0 if x ≤ t, where t is a threshold value

In 2D it is a line; in 3D a plane; in N dimensions it becomes a hyperplane — an (N−1)-dimensional surface dividing the space

A simple binary classifier that computes a weighted sum of its inputs, adds a bias, and passes the result through a step function to produce a 0 or 1 output. i.e. given some input, it outputs either 0 or 1

Inputs x₁…xₙ each multiplied by weights w₁…wₙ, summed together with a bias b, then passed through an activation function (step function) to give output ŷ

ŷ = f(w·x + b), where f is the step function: f(z) = 1 if z > 0, else 0 (z = w₀ + w₁x₁ + w₂x₂ + … = Σ wᵢxᵢ)

η = learning rate (step size); y = true label; ŷ = predicted label; (y − ŷ) = error signal; xᵢ = input feature

When ŷ = y (prediction is correct) — the error term (y − ŷ) = 0, so weights are unchanged

It can only represent a linear decision boundary — it cannot classify data that is not linearly separable (e.g. XOR problem)

The point, line, or surface at which a model changes its output — e.g. in a spam filter, the exact boundary between ‘spam’ and ‘not spam’

The step function gives a hard 0 or 1 with no gradient; sigmoid gives a smooth curve between 0 and 1, making it differentiable and suitable for learning

By engineering new features — e.g. squaring inputs for circular clusters, taking products x₁x₂ for multiple features, or using log transforms for exponentially distributed data

Exactly one — a specific set of weights corresponds to exactly one decision boundary

Infinitely many DB exists when output = 0 Rearranging equation shows two pairs of linearly dependant weights give the same boundary e.g. w₀=4, w₁=2 and w₀=2, w₁=1 both place the boundary at x₁ > -2; any positive scalar multiple works

The decision boundary stays in the same position but the labels flip — what was class 1 becomes class 0 and vice versa

Given labelled observations of x and y, find a set of weights such that the perceptron correctly classifies all data points

w_new = w_old + Δw, where weights are adjusted iteratively until the perceptron classifies correctly

It only accounts for the sign of the error but ignores the sign and magnitude of the input xᵢ, so updates can move weights in the wrong direction

To account for the sign and magnitude of the input — if xᵢ is negative, decreasing wᵢ increases the weighted sum, which multiplying by xᵢ handles automatically

It is a constant that scales the size of each weight change, controlling how aggressively the perceptron updates its weights each step

Δwᵢ = η(y_true − y_predicted)xᵢ

When y_true = y_predicted — the error term is zero so Δwᵢ = 0 and the weight is unchanged

It is guaranteed to converge to a correct solution for any linearly separable n-dimensional dataset — but makes no guarantee if the data is not linearly separable

manipulate data, or get more perceptrons