Learning and Hopfield Networks - Essay Example

Learning and Hopfield Networks Learning and Hopfield Networks Learning and Hopfield network A Hopfield network is a type of repeated man made neural network that John Hopfield invented. The Hopfield nets act as memory structure, which are made up of binary threshold nodes. They are always certain to meet to the local minimum, but meeting to one of the stored systems is not certain. The Hopfield networks are known to provide a proper understanding of human memory. There is a recall pattern that is found in the Hopfield network which closely relates to the recall mechanism in humans. If there is any destruction of the neurons of the network, then there will be a degradation of performance, but some network ability may be maintained just like how the human’s brains function. Hopfield Type Networks The Hopfield network a combination of a learning matrix, which comprises of repeated links that have been implemented. The learning matrix comprises a weight matrix in which the relation between the goals and inputs are stored. The network gives general information on the dependency between an incomplete and training data that does not make any meaning, and in this case it makes it act as a learning matrix (Hopfield, 1999). This type of network can be referred to as a linear model as it can only work on data that can be arranged linearly. It can also be viewed as a computation system that comprises of several composition of the neural system. The neurons that are found in this system are linked to all other neurons excluding themselves, and that means that there is no self reaction in the network. The connection wij arises due to the combination of two neurons, which connect. When Neuron I, which is the output and neuron j which is the input connect, they possess the same power as connection wji which makes the weight matrix to be symmetric (Hopfield, 1999). The Hopfield network can either operate in a continuous or discrete manner. An example of a network which works with the discrete style in which the neurons functions in a discrete manner is as below, Figure. Diagram of a Hopfield type network (Haykin, 1999) There are two modes in which Hopfield networks can operate and this includes, synchronous mode in which the neurons fire simultaneously and asynchronous mode in which the neurons fire randomly. The Hopfield network is an excellent example of a network that uses associative memory in which, the network is able to store several patterns and get the nearby match to the original state (Haykin, 1999). The energy dynamics of Hopfield network can be described as each pair of nodes is considered to be joined using a spring in which some of them are lengthened while others fully compressed. The springs which are used in the joining push the nodes closer while at the same time pulling them away from each self’s space (Maurer Hersch and Billard, 2005). This is described as analogue due to the positive and negative effects of weights that connect the two nodes which have opposite values. The on and off nodes when used hand in hand in the firing of one node contributes to the activation of the other one reinforcing a stable state. In case all the nodes are off there will be no contribution by any node to the other, and this will create an unstable system. If one node is active while the other one is not, the active one inhibits the non active one making it impossible to fire thus the stable state is reinforced. In the case where both the nodes fire, each inhibits one another and the state is destabilized. The conditions where the weights must be symmetric is always used as it assures that energy decreases as the activation rules are adhered to, and the network may show abnormal behavior where non symmetric weights are applied. Despite all these, Hopfield discovered that abnormal behavior rarely occurred and did not have a great ability of the system to work as a memory system (Maurer Hersch and Billard, 2005). Associative memory is accounted for by the Hopfield model through the integration of memory vectors. The use of the memory vectors trigger the reclamation of several other same vectors in the network, and this may cause intrusions. There are two types of associations in the associative memory for this network which includes hetero association and auto association. Auto association is related to itself and so a vector while hetero association is where there is the association of two different vectors in storage (Levy and Steward, 1999). All the two types of associations can be stored in one memory matrix, but with an exception of the given matrix is not one of the other but a blend of the two associations. The Hopfield’s model uses similar learning rule as Hebbs which showed that learning happened due to the strength of the weights when the activity is happening. The neural network model can explain the recurrence on recall accuracy by applying a difficult learning algorithm, and this was approved by Rizzuto and Kahana (2001). There is no learning when the retrieval process is taking place, and as an outcome, the weights are constant which enables the model to switch to a recall stage from a learning stage. Rapid forgetting that may be seen in a Hopfield model when a cued recall task is being carried out is showed by the addition of contextual drift (Levy and Steward, 1999). The network of the Hopfield model is established by the amounts of the neuron and its connections over a certain network. The number of memories to be saved vastly depends on the connections and neurons. It is clearly evident that a lot of mistakes occur if a large sum of vectors is stored. When there are difficulties for Hopfield model to recall the correct pattern, there is a strong possibility that an intrusion has occurred as related item are mixed up and occurrence of wrong patterns (Levy and Steward, 1999). When testing on the running of the Hopfield model, pick a node randomly at every step so that it may fit to be used in the system. The behavior of the node is deterministic in that as it moves to another state, its energy and the neighbor’s energy are minimized. During the training of a Hopfield net, energy of states is lowered to a level that the net can remember. This factor enables the net to act as a content addressable memory system, and this means that the network is able to converge to remembrance when given part of the state. The net can be most useful in the recovery of a destroyed input this is mostly referred to as associative memory as memory is recovered based on similarity. References Haykin, S., (1999). Neural Networks. A Comprehensive Foundation, Second Edition, Prentice-Hall, Inc., New Jersey, 1999, pp.680-696. Hopfield, J., (1999). Neural networks and physical systems with emergent collective computational properties. Proceedings of the National Academy of Sciences of the USA, 79:2554 - 2588, Levy, W & Steward, O. (1999). Synapses as associative memory elemnst in the hippocampal formation. Brain Research, 175, 233-245 Maurer A., Hersch M. and Billard A. (2005). Extended Hopfied Newtowork for Sequence Learning: Application to Gesture Recognistion, International Conference on Artificial Neural Network Warsaw. 493-498 Read More