Support for the Rescorla-Wagner Model - Coursework Example

Does neuroscientific research provide support for the Rescorla-Wagner model? due: Introduction The Rescorla-Wagner model was developed in the 1970s by Robert Rescorla and Allan Wagner. It is a model of classical conditioning whereby the animal is thought to learn from the differences between what is expected and the reality. The model involves trials in which stimuli are either present or not at different points in the trial. The outcome of the unconditioned stimulus for a particular trial is then represented by the total of all associative strengths for the conditioned stimuli present during the trial. This feature of the model puts it ahead of other previous models and also allows a clear explanation of other important experimental occurrences such as blocking, overshadowing and over expectation (Li, Schiller, Schoenbaum, Phelps & Daw, 2011). The model’s simple yet powerful explanations have made it one of the most influential learning models in the past four decades, although there has been much criticism on some of its weaknesses. Recent research on the brain structure and function has provided evidence to support the theory, especially with regard to the role of neurons and neurotransmitters in learning (Holroyd & Coles, 2002; Li et al, 2011). However, new findings have also differed with the claims of the Rescorla-Wagner model, identifying certain weaknesses in the model or finding contradictory evidence. The Role of Dopamine in Associative Learning Much research has conducted on the role of neurotransmitters in learning and predicting behaviour. Dopamine has been commonly associated with conditioned learning as it is integral in the process. According to some studies, dopamine acts by directing attention to salient events, and it only appears to be a prediction error since rewards and reward-predicting stimuli are both salient events (Berridge, 2007). Other studies have also shown the significant role of dopamine in certain mental illnesses such as schizophrenia, where the brain wrongly associated external stimuli to internal representations (Kapur, 2003). Dopamine, however, is not necessary for all forms of learning that involve reward cues becoming effective predictors. Rather, it is selectively associated with a form of stimulus-reward learning where incentive salience is linked to reward cues (Flagel et al., 2011). Research along this line of knowledge supports the Rescorla-Wagner model by providing evidence proving that association of stimuli with rewards is controlled in the brain, and rewards and punishments influence learning. Dopamine has however been associated with positive reinforcement (stimulus-reward) more than aversive predictions (Flagel et al., 2011). Functional Organization Animal studies have been employed to find out the functions of various aspects of the human brain about relational learning. The orbitofrontal cortex has been found to have a significant role in representing goal-directed value (Horvitz, 2000). There is also some indirect evidence on the role of dopamine and dopaminergic mechanisms in learning from reward prediction errors in human beings and also a more direct indication from experiments concerned with the pharmacological manipulation of dopamine. These studies, along with many others not only provide proof of learning through reward prediction errors; but also identify roles for key sections of the brain such as the orbitofrontal cortex, medial prefrontal cortical structures and the cingulate cortex. The cingulate cortex has been under research in animal studies focused on the cost-benefit analysis typical in the decision-making process, as well as the interactions of dopamine and the anterior cingulate cortex that postulate possible interaction with relational learning (Dayan & Niv, 2008). Computation of Prediction Errors In terms of pathways, recent findings indicate that the lateral happenings is responsible for suppressing the activity of dopamine neurons, in such a way that may be important for the representation of negative prediction errors that occur when reality turns out to be worse than was expected. It is then possible that the pauses in the burst firing of dopamine cells could code for the negative prediction errors. These findings have received support, even though the baseline rate of firing of the neurons is low. More studies done on the relationship between learning from positive and negative prediction errors and genetic polymorphisms involving the D2 dopaminergic receptor have established a functional link between dopamine and learning in cases where outcomes are worse than predicted (Klein et al, 2007). However, other experiments have resulted to new findings that hold different views on the distinction between the absence of an expected reward and a more directly aversive outcome. Much research ought to be done to resolve the discrepancy and provide more reliable data. The role of the amygdala and medial prefrontal cortex in coding for both the positive and negative prediction values and errors is still significant, especially in relation to the learning process (Klein et al, 2007). Other studies have come up with findings that highlight inconsistencies in relational learning as indicated by the Rescorla-Wagner model. These include questions of aversive and appetitive interactions, exploration and novelty, certain aspects of neuroeconomics, Pavlovian-instrumental interactions and certain structural findings. Since there are several control mechanisms involved in the learning process in the brain, it becomes difficult to interpret some of these results without ambiguity. This situation becomes worse as little is known concerning the activity of neurons and their interactions (Klein et al, 2007). Appetitive-Aversive Interactions Appetitive-aversive interactions continue to dog neural relational learning (including the Rescorla-Wagner model) since it is not yet clearly understood how the coding for aversive, rather than appetitive, prediction errors occurs. Although most studies find that aversive predictions have an inhibitory effect on dopamine, fMRI studies involving the ventral striatum in humans has led to production of mixed results, with aversive prediction errors leading to both above and below baseline results. In another study, withdrawing or administering dopamine-boosting medicine to patients with Parkinson’s disease, results to enhanced or repressed learning from negative outcomes, respectively (Frank, Seeberger & O’reilly, 2004). An investigation of the effects of negative prediction errors on certain areas of the brain have produced consistent results, although given that neurons adjacent to the amygdala code for both appetitive and aversive outcomes, fMRI may fail to address all such issues adequately. Owing to their multiphasic range of reactions, aversive outcomes are more complex than appetitive ones. Aversive predictions can also lead to active avoidance which then results to appetitive predictions linked to the achievement of safety. This mechanism is common in conditioned avoidance responses and has similarly been proven to cause spikes in BOLD in the same section of the orbitofrontal cortex; that is activated by positive prediction errors (Dayan & Niv, 2008). Novelty, Uncertainty and Exploration The main concept of the Rescorla-Wagner model focuses on exploration in that it uses past experience to predict the outcome of subsequent trials. Different instances occur when the agents seek to optimize exploration whereby they take into account not only the benefits of expected future rewards, but also the prospective benefits of discovering unknown punishments or rewards (Dayan & Niv, 2008). Balancing between these two requires careful accounting for uncertainty which has been linked to neuromodulators acetylcholine and norepinephrine. Uncertainty and novelty are closely related and seem to involve dopaminergic areas and some of their targets. Both of them also support various types of exploration; undirected exploration in which novel unknown areas of the surrounding are explored uniformly, and directed exploration where exploratory actions are taken in proportion to known uncertainty. The prefrontal cortex in humans, medial prefrontal cortex in monkeys and the amygdala have been associated with tracking uncertainty (Dayan & Niv, 2008). These new findings, therefore, imply that learning is also influenced by the uncertainty, as is the exploration. In a rapidly changing environment, learning and forgetting rates are supposed to be higher as compared to a stationary one where it is expected that learning (and forgetting rates) should be slower. Learning should, therefore, be determined by both the stimuli and environment. Risk, Regret and Neuroeconomics Recent findings in the field of behavioural economics, experimental economics and neuroeconomics have challenged the credibility of the Rescorla-Wagner model. They study neural and psychological factors underlying different situations where human behaviour departs from the normal ideals. An example is the influence of risk associated with the variability of the prediction outcomes. It has been shown that risk perceptions are linked to certain brain areas such as the orbitofrontal cortex, therefore, may also be a determinant of prediction. Regret can also influence the prediction choice as it induces a form of counterfactual thinking when foregone alternatives turnout to be better than the chosen ones. Accounting for the influence of regret is also absent in the Rescorla-Wagner model. These new findings, therefore, weaken the model although they provide more areas for research that can improve knowledge on relational learning. Challenges to the Model-Based Approach The model-based approach has a few shortcomings since it tries to identify a pattern from a series of experiences and generalizes the results. This is however inadequate in explaining the vast range of learning scenarios where optimal learning is dependent on detection of change and learning a ‘new’ concept, instead of updating previously acquired knowledge. Behavioural results show that animals and human beings do not simply ‘unlearn’ the preceding predictive information as is explained in the paradigm of extinction. On the contrary, they learn new predictive information that inhibits the old one (Bouton, 2002). In general, various factors can determine the optimal learning procedure, ranging from the uncertainty to change, yet cannot be adequately covered in one model. More approaches need to be invented to provide a clear picture of the process, as some scholars suggest (Redish, Jensen, Johnson & Kurth-Nelson, 2007). Conclusion In conclusion, we can say that recent neuroscientific research has both supported and disputed the Rescorla-Wagner model. Most of the research provided an in-depth analysis of how the learning process occurs, by investigating the role of neurotransmitters and certain sections of the brain in the whole process. These experiments confirmed the inferences that were made out of observational studies. Also, many of the contemporary learning theories are based on the Rescorla-Wagner model, only adding slight modifications. Although it has received much criticism, the theory is still sound as it proves certain examples of classical conditions, and it is also superior to the Pavlov classical conditioning theory. Moreover, the research has also differed with some of the principles of the Rescorla-Wagner theory, mainly based on the shortcomings and also new information that was unavailable at the time the theory was being developed. Loopholes such as the unclear role of dopamine in aversive prediction errors, influence of uncertainty and novelty and the reproducibility of animal studies in human settings among others should be addressed and this way, relational learning can be better understood and applied. References Berridge, K. C. (2007). The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology, 191(3), 391-431. Bouton, M. E. (2002). Context, ambiguity, and unlearning: sources of relapse after behavioral extinction. Biological psychiatry, 52(10), 976-986. Dayan, P., & Niv, Y. (2008). Reinforcement learning: the good, the bad and the ugly. Current opinion in neurobiology, 18(2), 185-196. Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., ... & Akil, H. (2011). A selective role for dopamine in stimulus-reward learning. Nature, 469(7328), 53-57. Frank, M. J., Seeberger, L. C., & Oreilly, R. C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science, 306(5703), 1940-1943. Holroyd, C. B., & Coles, M. G. (2002). The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychological review, 109(4), 679. Horvitz, J. C. (2000). Mesolimbocortical and nigrostriatal dopamine responses to salient non- reward events. Neuroscience, 96(4), 651-656. Kapur, S. (2003). Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. American Journal of Psychiatry, 160(1), 13-23. Klein, T. A., Neumann, J., Reuter, M., Hennig, J., Von Cramon, D. Y., & Ullsperger, M. (2007). Genetically determined differences in learning from errors. Science, 318(5856), 1642- 1645. Li, J., Schiller, D., Schoenbaum, G., Phelps, E. A., & Daw, N. D. (2011). Differential roles of human striatum and amygdala in associative learning. Nature neuroscience, 14(10), 1250-1252. Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007). Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychological review, 114(3), 784. Read More