Connectionism

Consists of an input layer, hidden layers and an output layer Overall the network transforms any input pattern into a corresponding output pattern as dictated by the arrangement and strength of the many connections between neuro

Many Chinese room’s might solve the symbol grounding problem (but we need to many since 1 neuron = one person)

An optimization algorithm for finding a local minimum of a differentiable function Used for back-propagation

Rises from complexity. How we assign meaning lays beyond language and humans One must look at how neurons code and transform sensory signals the relations between weights and activations of the neurons of the trained model

How much one neuron is firing to another (the lines that connect the units)

If a neuron/unit is activated (or not)

On/off

Instead of having serial processing (one at a time) one has multiple at a time

Learning neural networks by gradient estimation (optimization algorithm) The strategy exploits the calculated error between the actual values of the processing units in the output layer and the desired values, which is provided by a training signal. The resulting error signal is propagated from the output layer backward to the input layer and used to adjust each weight in the network. The network learns, as the weights are changed, to minimize the mean squared error over the training set of words

Identify the undoubtedly complex function that governs the human pattern of response to the environment and then write the program (the set of recursively applicable rules) by which the SM machine will compute it.

It’s like a programmed robot following a strict set of human-made rules

Nervous system: Parallel machines The neurons are comparatively simple: Analog response - They are somewhat digital - it is firing or not - We look at the firing frequency and translate it into a value between 0 and 1 - This is a contrast to computers 0’s and 1’s In the brain, axons projecting from one neuronal population to another are often matched by axons returning from their target population

Any vector-to-vector transformation

Everything is physical

Distinction between body and mind

It converts English text to speech sounds 1. Receives as input letter in a word ○ Local representation 2. Performs the transformation 3. Yield the elementary speech sounds Has 309 processing units and 18,629 connection strengths (weights) that must be specified. The network does not have any initial or built-in organization for processing the input or (more exactly) mapping letters onto sounds. All the structure emerges during the training period. The values of the weights are determined by using the “back-propagation” learning algorithm developed by Rumelhart, Hinton, and Williams (1986)

Symbol manipulating machine

Every effective computable functions is recursive computable

There is a “rote” procedure for determining, in finite time, the output of the function for a given input

There is a finite set of operations that can be applied to a given input, and then applied again and again to the successive results of such applications, to yield the function’s output in finite time

Because something is valid it doesn’t need to be sound Maxwell and light (theoretical) Axiom 1. Electricity and magnetism are forces. Axiom 2. The essential property of light is luminance. Axiom 3. Forces by themselves are neither constitutive of nor sufficient for luminance. Conclusion 1. Electricity and magnetism are neither constitutive of nor sufficient for light. Problem: If you stand in a dark room with a magnet there will be no light Maxwell’s responses: The “luminous room” experiment is a misleading display of the phenomenon of luminance because the frequency of oscillation of the magnet is absurdly low, too low by a factor of 10^15. But the room already contains everything essential to light according to Maxwell’s own theory

What people can or cannot imagine often has nothing to do with what is or is not the case, even where the people involved are highly intelligent Axiom 3 is false

The functional architecture of classical SM machines is simply the wrong architecture for the very demanding jobs require

There are various procedures for adjusting the weights so as to yield a network that computes almost any function-that is, any vector-to-vector transformation-that one might desire. In fact, one can even impose on it a function one is unable to specify, so long as one can supply a set of examples of the desired input-output pairs.