Neural Networks 1
No description yet. This curated deck is ready for daily study.
Card preview (19)
What is the McCulloch-Pitts neuron?
Proposed in 1949 to perform logical operations Showing neurons can compute logical operations (functions) Weighted sum = x1w1 +x2w2+…+xnwn
What are these models for?
Modeling real neurons Modeling populations of neurons Solving engineering problems with artificial intelligence
What does modeling real neurons mean?
Needs to be biologically realistic Ex. all-or-none firing Want math to reflect what is happening in a human brain
What does modeling populations of neurons mean?
No need for all or none firing, but still has to respect other biological/cognitive constraints
What does solving engineering problems with artificial intelligence?
No need to be biologically or cognitively realistic Just trying to solve a problem
What is supervised learning?
A right output or label associated with every pattern of inputs, and we use that to set weights
What is a linear perceptron?
One of the simplest neural networks that can learn Outline Make an initial guess of the connection weights For every (input, target) pair, compute the output and compare it against the target Use the error term to update weights Loop until the error does not get smaller Uses delta learning rule Supervised learning
What is the delta learning rule?
Wi = Wi + &(yi - oi)x New weight = Old weight + learning rate(target output-actual output) input value
What is the error in delta learning rule?
Target output - actual output
What can perceptrons be used for?
Images Regression
What is XOR?
A truth function that represents the exclusive use of or. That is the ‘one of the other but not both’ sense Ex. You can eat cake, or you can eat ice cream (but not both)
What is a limit of perceptrons?
A perceptron cannot learn XOR
Why can’t a perceptron learn XOR?
A perceptron can correctly classify inputs if and only if they are linearly separable Yet, XOR is not linearly separable.
How can XOR be computed?
It can be computed by combining other truth functions A network with hidden units
What is an AI winter?
An pessimism towards AI
What is an AI summer?
Strong AI funding and downturns
What was the cause of the first AI winter?
The seeming inability to train multi-layer networks
What ended the first AI winter?
The discovery of the backpropagation algorithm ended this winter.
What is backpropagation?
The error signal is computed at the output layer. Each hidden unit then sums the error signals from all the output units it connects to. Since the error signal gets propagated backward through the net.