\documentclass[10pt]{article}

\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}





\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother


\usepackage{natbib}




\renewenvironment{abstract}
  {{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
  {\bigskip}

\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}


\usepackage{authblk}


\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}%

\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[ngerman,english]{babel}



\usepackage{float}








\begin{document}

\title{Reinforcement-based cognitive biases as vulnerability factors in alcohol
addiction: From humans to animal models}



\author[1]{Karolina Noworyta}%
\author[1]{Agata Cieslik}%
\author[1]{Rafal Rygula}%
\affil[1]{Maj Institute of Pharmacology Polish Academy of Sciences}%


\vspace{-1em}



  
  \date{June 01, 2024}


\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup





\selectlanguage{english}
\begin{abstract}
Alcohol use disorder (AUD) is one of the most common but still poorly
treated psychiatric conditions. Developing new treatments requires a
better understanding of the aetiology of symptoms and evaluation of
novel therapeutic targets in preclinical studies. Recent developments in
our understanding of the reinforcement-based cognitive biases (RBCBs)
that contribute to the development of AUD and its treatment offer new
opportunities for both clinical and preclinical research. In this
review, we first briefly describe psychological and cognitive theories
that implicate various aspects of reinforcement sensitivity in the
development, maintenance, and recurrence of alcohol addiction.
Furthermore, in separate sections, we describe studies investigating
RBCBs and their neural, neurochemical, and pharmacological correlates,
and we discuss possible interactions between RBCBs and trajectories of
AUD. Finally, we describe how recent translational studies using
state-of-the-art animal models can facilitate our understanding of the
role of reinforcement sensitivity and RBCBs in various aspects of AUD.%
\end{abstract}%



\sloppy


\textbf{Introduction}

Alcohol addiction is one of the biggest problems of modern society
through its impact on health, social cohesion, crime, and its
comorbidity with other neuropsychiatric disorders. Major advances in
genetics and molecular neurobiology have led to the identification of
many of the primary targets of alcohol and revealed neuroadaptations
that develop with its chronic consumption. However, understanding
pre-existing cognitive deficits that serve as substrates compounding the
initiation of alcohol use and the development of alcohol use disorder
(AUD) remains a major challenge.

All goal-directed behaviours, including alcohol use and abuse, are
composed of two core components: the decision-making process, which
starts with representations of the available options that lead to the
selection of the option with the highest expected value, and
reinforcement learning, through which outcomes are used to refine value
expectations. The decisions based on the above-mentioned processes are
often biased by an abnormal sensitivity to positive and negative
reinforcement, which results in abnormal sensitivity to performance
feedback, skewed interpretations of ambiguity, inflated expectations,
and asymmetrical belief updating. We have defined this abnormal
cognitive processing as reinforcement-based cognitive biases (RBCBs).
While the investigation of cognitive biases has a long history in
economics and psychology, RBCBs in the context of AUD have been much
less systematically investigated and, perhaps, neglected. This is
surprising, as most of the decisions regarding alcohol-related
behaviours are related to their positive and negative reinforcing
effects. Although the causal role of the RBCBs in AUD has been
emphasized by various psychological theories and investigating the
interrelations between biased cognition and AUD could become a thriving
area of research that capitalizes on the newest advances in cognitive
neuroscience, psychology, and psychiatry, studying the nature and the
extent to which RBCBs play a role in the development and maintenance of
alcohol addiction still faces formidable methodological obstacles and
does not provide definite answers regarding underlying mechanisms.

In this review, we first briefly describe psychological and cognitive
theories that implicated various aspects of reinforcement-biased
cognition in the development, maintenance, and recurrence of alcohol
addiction. Furthermore, in separate sections, we describe recent studies
investigating the RBCBs, their neural, neurochemical, and
pharmacological correlates, and possible interactions between RBCBs and
trajectories of AUD. Finally, we describe how recent translational
studies using state-of-the-art animal models could facilitate our
understanding of the role of reinforcement sensitivity and RBCBs in
various aspects of AUD.

\textbf{Psychological and cognitive theories and experimental evidence
that implicated various aspects of reinforcement-biased cognition in the
development, maintenance, and recurrence of alcohol addiction}

Several psychological theories sought to describe reinforcement-biased
processes that are implicated in the development and progression of AUD
(Skinner \& Aubin, 2010). According to the allostatic model (Koob, 2003;
Koob \& Volkow, 2010), the early stages of alcohol addiction are
primarily driven by positive reinforcement, which generally refers to
the pleasant feelings and social enhancement effects that alcohol
provides, while the later and more severe stages are driven by negative
reinforcement, which encompasses relief from negative affective states
such as stress or anxiety. The second model, the incentive-sensitization
theory (Robinson \& Berridge, 1993), highlighted increasing subjective
`wanting' for alcohol that is driven by hypersensitization of the
dopaminergic system, while a related theory postulated by Field \& Cox
(2008) proposed that sensitization of the dopaminergic system drives
alcohol attentional biases related to alcohol-related cues, leading to
an increased craving. Finally, according to Cox \& Klinger (2011), some
individuals drink alcohol for its positive reinforcing properties to
increase their positive affective experience (e.g., mood enhancement),
while others drink alcohol for its negative reinforcing properties to
dampen their negative emotions and cope with distress and anxiety
(tension reduction). The former, whose drinking is maintained by the
rewarding effects of alcohol, are called `reward drinkers', while the
latter are called `relief drinkers' (Ooteman, Koeter, Verheul, Schippers
\& Van den Brink, 2006). Based on accumulating experimental evidence, it
seems that there may be distinct types of AUD in which alcohol abuse is
differentially driven by various mixtures of the above-mentioned
reinforcement-based processes.

Individual differences in vulnerability to substance use disorders,
including AUD, may be explained in light of reinforcement sensitivity
theory (Corr, 2004). According to this hypothesis, every behaviour is
governed by two conceptual brain systems: the behavioural approach
system (BAS) and the behavioural inhibition system (BIS). These two
systems differentially respond to the rewarding and punishing effects of
alcohol. The BAS underlies approach motivation, positive affect, and
reward learning processes and may contribute to the acquisition of AUD,
while the BIS inhibits behaviour in response to stimuli signalling the
loss of expected reward, uncertainty, and goal conflict and may support
the maintenance of AUD (Corr, 2008). The BIS also gives rise to
emotional distress and has been associated with trait negative affect,
especially anxiety (McNaughton \& Gray, 2000). Research has suggested
that the BAS can be further subdivided into three different types of
reward sensitivity: (a) reward drive, relating to persistent pursuit of
desired goals; (b) fun-seeking, relating to simple biological
reinforcers that do not require planning; and (c) reward responsiveness,
encompassing responses related to receiving or anticipating reward
(Carver \& White, 1994; Corr, 2008). As alcohol consumption represents a
high reward value (Everitt \& Robbins, 2005), it is not surprising that
several studies demonstrated a relationship between positive
reinforcement sensitivity, particularly in relation to fun-seeking, and
higher alcohol intake, particularly in the form of binge drinking (Feil
\& Hasking, 2008; Franken \& Muris, 2006; Loxton \& Dawe, 2001; O'Connor
\& Paley, 2009; Voigt et al., 2009). It was also found that BAS
sensitivity was related to both desire and negatively reinforcing
aspects of alcohol craving. Subjects with high BAS sensitivity scores
experienced significantly stronger desires and intention to drink
alcohol and negative reinforcement craving during exposure to
alcohol-related cues than subjects with low BAS sensitivity scores
(Franken, 2002). Furthermore, persons with high BAS sensitivity
experienced high negative reinforcement craving during this exposure
(Franken, 2002). The relationship between BIS sensitivity and alcohol
consumption is less clear. Although several investigators reported null
associations between BIS and AUD (Hundt, Kimbrel, Mitchell \&
Nelson-Gray, 2008; Kambouropoulos \& Staiger, 2001; O'Connor \& Colder,
2005), other studies suggested that the BIS, instead of inhibiting
behaviour, draws attention to the potential dangers of a situation and
functions as a conflict resolution system (Corr, 2008)). As a result, a
more sensitive BIS leads to high anxiety, which could be reduced by
indulging in drinking (Hasking, 2006). Indeed, it has been explicitly
proposed that a sensitive BIS acts anxiogenically in response to health
information, which may motivate protective health-related behaviours
(Norman, Boer \& Seydel, 2005). By the same token, a more sensitive BIS
may act as a protective factor due to the avoidance of potentially risky
situations or aversive consequences (e.g., hangovers). This theory has
been supported by several other studies (Kimbrel, Nelson-Gray \&
Mitchell, 2007; O'Connor, Stewart \& Watt, 2009; Pardo, Aguilar,
Molinuevo \& Torrubia, 2007) who found a negative correlation between
the sensitivity of the BIS and the frequency and quantity of alcohol
consumption. Similar results were reported by Knyazev and collaborators
(2004), who found that a more sensitive BIS protects against substance
abuse among youths. Taken together, it seems that the effects of BAS and
BIS on the trajectories of alcohol addiction might be interactive
(Figure 1). Indeed, several experiments demonstrated that high BAS
sensitivity coupled with low BIS sensitivity was associated with
significantly increased alcohol use (Kellough, Beevers, Ellis \& Wells,
2008; Wardell, O'Connor, Read \& Colder, 2011). Moreover, Wardell and
colleagues (2011) demonstrated high BIS sensitivity as a risk factor for
subsequent problematic drinking, but only when combined with high BAS
sensitivity. When the sensitivity of BAS was low, BIS sensitivity was
demonstrated to be protective against subsequent drinking. The
above-mentioned studies suggested that the relationship between BIS and
problematic drinking is moderated by BAS: in the absence of a sensitive
BAS to shift attention towards the rewarding, tension-reducing
properties of alcohol, the aversive consequences of drinking are more
salient, leading to alcohol avoidance (Wardell, O'Connor, Read \&
Colder, 2011).

The objective investigation of BAS and BIS sensitivity in people abusing
alcohol can be conducted using a framework of neurophysiological and
neuropsychological measures. The neurophysiological measures are mostly
based on the peripheral assessment of cardiac and electrodermal
reactivity during appetitive responding for reward and during extinction
of appetitive responding (i.e., frustrative non-reward), respectively
(Beauchaine, 2001; Fowles, 1988; Iaboni, Douglas \& Ditto, 1997; Tranel,
1983). In turn, the neuropsychological evaluation is usually conducted
using self-report questionnaires that assess individual differences in
BAS and BIS sensitivity by the degree to which respondents endorse
prototypical approach- and avoidance-related behaviours. This strategy
is exemplified in the BIS/BAS scales of Carver and White (1994), which
were developed with the explicit purpose of assessing individual
differences in state reactivity of these systems. Recently,
investigators have developed computerized neuropsychological tests to
experimentally measure sensitivity to reinforcement (Cools, Clark, Owen
\& Robbins, 2002; Evers et al., 2005; Paulus, Hozack, Frank \& Brown,
2002; Paulus, Hozack, Frank, Brown \& Schuckit, 2003; Slaney,
Hinchcliffe \& Robinson, 2018). These tests offer many advantages over
conventional pencil-and-paper testing because they standardize aspects
of administration and automate data collection and analysis. One of
these tests is the probabilistic reversal learning (PRL) test, offering
an effective way of measuring an individual's sensitivity to
reinforcement by assessing win-stay and lose-shift behaviours (WSLS)
following rewarding and punishing feedback, respectively. In this
paradigm, subjects are presented with two (sometimes more) stimuli on
each trial and using trial-and-error feedback after each response, learn
to select the stimulus that is usually correct (rewarded on a majority
or punished on a minority of trials) and to avoid the stimulus that is
usually incorrect (punished on a majority or rewarded on a minority of
trials). This rule intermittently reverses such that the stimulus that
was usually rewarded becomes usually punished/unrewarded and vice versa.
Consequently, responding must be adjusted to gain the reward and avoid
punishment. Rewarded outcomes followed by a decision to stay with the
response that delivered them (win--stays) constitute a measure of
sensitivity to positive reinforcement. Conversely, lose--shift ratios,
calculated by dividing punishing outcomes after which the subject
changed the choice by the total number of punished trials on a given
stimulus, represent a measure of sensitivity to negative reinforcement.
The use of probabilistic reinforcement increases the complexity of the
task in such a way that the information from any given choice is
insufficient to guide future behaviour, and subjects must engage
cognitive functions to track the reward and punishment history for both
stimuli to determine the stimulus that is more beneficial overall. The
PRL paradigm has been recently successfully applied in a number of
studies used to investigate the neuroanatomical and neurochemical
correlates of reinforcement sensitivity in humans and non-human animals
(Rygula, Noworyta-Sokolowska, Drozd \& Kozub, 2018). These will be
discussed in subsequent sections of this paper.

Individual differences in reinforcement sensitivity may influence the
acquisition and maintenance of positively and negatively biased
alcohol-drinking outcome expectancies, which can be defined as
positively or negatively inflated beliefs about the effects of alcohol
on behaviour, cognition, moods, and emotions (Leigh, 1989). These
expectations, in turn, have a pivotal role in determining decisions
about alcohol drinking and trajectories of its use (Jones, Corbin \&
Fromme, 2001). Beginning with the seminal work of Brown and her
colleagues (Brown, Goldman, Inn \& Anderson, 1980; Goldman, 1994), many
studies have documented positively biased outcome expectancies for
engaging in alcohol drinking, together with minimized negative
expectancies and poor self-efficacy or beliefs about one's ability to
cope without alcohol, can maintain addictive behaviours and predict
relapse (Brown, Christiansen \& Goldman, 1987; Christiansen \& Goldman,
1983; Floyd \& Widaman, 1995; Fromme \& D'Amico, 2000; Fromme, Stroot \&
Kaplan, 1993). In these experiments, alcohol-drinking outcome
expectancies were measured using structured interviews of focus groups
and the self-report questionnaires in which individuals endorse each
questionnaire item as to whether or not they hold that particular
expectancy. A sum of the expectancy endorsements representing the
individual's overall alcohol outcome expectancy is further positively
associated with alcohol consumption, e.g., subjects who drank in a
frequent, social manner, expected alcohol to enhance their social
behaviour (Christiansen \& Goldman, 1983; Lee, Maggs, Neighbors \&
Patrick, 2011). Other studies have investigated the potential
associations of different categories of alcohol outcome expectancies
with subsequent drinking behaviour. A study by Leeman and collaborators
(2009) found significant associations of euphoria and social enhancement
expectancies with binge alcohol use, while the study by Pabst and
colleagues (2010) found the same association concerning sexual and
social relationship enhancement expectancies. Other cross-sectional and
longitudinal studies have demonstrated that greater numbers of positive
alcohol outcome expectancies are associated with greater numbers of
negative consequences independent of the level of consumption (Blume \&
Blume, 2014; Blume, Lostutter, Schmaling \& Marlatt, 2003).

Abnormal perceptions of risks associated with alcohol-drinking outcomes
can also be considered within the concept of unrealistic optimism bias
(Sharot, Korn \& Dolan, 2011), which encompasses two different
phenomena: unrealistic comparative optimism and unrealistic absolute
optimism (Shepperd, Klein, Waters \& Weinstein, 2013). The former refers
to people's tendency to view the risks as lower for themselves than for
the others, while the latter refers to unrealistically positive risk
assessment when compared to an objective criterion, such as an actual
risk assessment. The mechanism that allows individuals to maintain or
arrive at unrealistically positive beliefs in the face of disconfirming
evidence was described for the first time by Sharot, Korn and Dolan
(2011). The results of this study demonstrated that when people update
their initial risk estimates, they tend to incorporate desirable
information (i.e., information that risks are lower than expected) to a
greater extent than undesirable information (i.e., information that
risks are higher than expected). In those with AUD, the asymmetric
reliance on information about alcohol-drinking outcomes, dependent on
their valence, may result in larger updates after desirable/positive
information than after undesirable/negative information and in this way
affect drinking trajectories. When considering expectancies and beliefs
about the effects of alcohol, it is also important to mention the
interpretation of ambiguity in drinking outcomes. Indeed,
positive/appetitive alcohol-related expectations are often associated
with more negative/inhibitory information arising from memories. This
conflict and the bias in its interpretation can be assessed using
various types of ambiguous-cue interpretation (ACI) paradigms. In this
experimental and fully translational approach, participants initially
learn to discriminate two stimuli (e.g., tones of different
frequencies), which acquire emotional and motivational value due to
subsequent feedback (monetary gain or avoidance of monetary loss). After
such an acquisition phase, the test phase introduces ambiguous stimuli
(e.g., tones of intermediate frequencies) that serve as a measure of
interpretation bias since the response to these stimuli indicates the
participants' expectation of rewarding or potentially punishing effects
of their decision (Papciak \& Rygula, 2017; Schick, Wessa, Vollmayr,
Kuehner \& Kanske, 2013). Other ways of assessing biased interpretations
of ambiguity include using single ambiguous words individually presented
to the participant, which are typically either homophones (words that
sound the same yet have different spellings and meanings, e.g.,
meat/meet or sea/see) or homographs (words with identical spelling yet
distinct meanings, e.g., bat---animal or wooden club; change---to alter
or money) (Drury, 1969; Gorfein \& Weingartner, 2008). Homophones and
homographs used in these studies usually have both threatening/emotional
and neutral interpretations, and an interpretation bias is established
as the number or proportion of each type of interpretation made by
participants from a list of such words (Hindash \& Amir, 2012; Warren,
Warren, Green \& Bresnick, 1978). Ambiguity interpretation biases can
also be investigated using ambiguous images, often with emotional and
neutral faces (Beevers, Wells, Ellis \& Fischer, 2009; Schaefer,
Baumann, Rich, Luckenbaugh \& Zarate Jr, 2010).

Another conceptual approach focuses not on expectancies regarding
alcohol-drinking outcomes \emph{per se} but on the mechanisms that
underlie them. These mechanisms have been defined as attributional
styles, which Alloy and colleagues (1984) described as the manner in
which a person explains the causes of prior outcomes. Indeed, following
positive or negative experiences with alcohol, people often wonder why
the event occurred (DeJoy, 1989; Jessup et al., 2014). To answer this
question, they make causal attributions based upon different dimensions:
internal-external (whether the event was caused by themselves (internal)
or by the situation (external)), stable-unstable (whether the cause of
an experience is constant and likely to happen again (stable) or pliable
and unlikely to reoccur (unstable)), and global-specific (whether the
cause may affect many areas of one's life (global) or only one area
(specific)) (Weiner, 1985). From this perspective, attributing negative
outcomes to causes that are internal to the person, stable and global
can reflect pessimism, learned helplessness, and addiction vulnerability
(Scheier \& Carver, 2018). Following the above-mentioned framework,
Weiner (1985) and Marlatt \& Donovan (2005) proposed that a certain
pattern of attribution promotes relapse in substance use. They argued
that a combination of internal, stable, and global attributions
contributes to maladaptive thought patterns concerning addiction and
promotes relapse. This pattern, often experienced by an individual after
a return to substance use following a period of self-imposed abstinence
from substances, has been named the abstinence violation effect (Curry,
Marlatt \& Gordon, 1987). Within this model, bleak attributions
regarding future reactions to alcohol exposure lead individuals to
believe that fighting addiction is pointless and, in turn, to relapse
rather than returning to abstinence (Walton, Castro \& Barrington,
1994). This conceptual orientation has given rise to assessment
techniques that focus on measuring the causes that people identify for
the outcomes that they experience. Typically, respondents are asked to
imagine being in outcome scenarios and to indicate how they would
explain the described outcomes (Peterson, Semmel, Von Baeyer, Abramson,
Metalsky \& Seligman, 1982) or to analyse naturally occurring verbatim
materials (e.g., newspaper articles or speeches) for explanatory style
(Schulman, Castellon \& Seligman, 1989).

Taken together, these studies consistently support the notion that
various types of RBCBs, such as abnormal sensitivity to performance
feedback, biased interpretation of ambiguity, inflated expectations, and
asymmetrical belief updating, can be differentially associated with
drinking at different stages of AUD (Figure 2). Understanding the
underlying mechanisms could provide useful information for tailoring
prevention and intervention efforts. For example, medications used in
the early stages of addiction should target positive reinforcement
sensitivity, while treatments that target alcohol-dependent individuals
in the late stages of addiction may need to simultaneously target
positive and negative reinforcement processes. In other words,
understanding individuals' positive and negative reinforcement profiles
and resulting cognitive biases can provide important information for
personalized medicine. Indeed, there is preliminary evidence for the
effectiveness of tailoring pharmacotherapies to AUD patients based on
their positive and negative reinforcement tendencies (Mann et al., 2018;
Roos, Mann \& Witkiewitz, 2017). For example, an opioid receptor
antagonist, naltrexone, has been found to be particularly effective for
reward drinkers, while acamprosate, which has been shown to
down-regulate the glutamatergic system, is particularly effective for
relief drinkers (Roos, Mann \& Witkiewitz, 2017). Specific neurochemical
correlates of the interactions between RBCBs and AUD, as well as
potential future treatment targets, are described in the next sections.

\textbf{Neurochemical correlates of reinforcement-based cognitive biases
as potential targets for pharmacological treatment of alcohol addiction}

Influential neurocomputational models emphasize dopamine (DA) as a
neurochemical correlate of reinforcement learning (Bromberg-Martin,
Matsumoto \& Hikosaka, 2010; Eisenegger et al., 2014; Frank, Seeberger
\& O'Reilly R, 2004; Samson, Frank \& Fellous, 2010). Indeed, the role
of DA in the mediation of positive reinforcement and resulting cognitive
biases is hard to overestimate. In a classic study, Frank and colleagues
(2004) demonstrated that lowered availability of DA (as in Parkinson's
patients off medication) was associated with worse learning from the
positive than from the negative outcomes. Importantly, this effect could
be reversed by pharmacological boosting of dopaminergic
neurotransmission. As suggested by Frank (2005), the opposite effects of
low and high DA availability reflected DA-induced shifts in the balance
between the BAS/Go/direct and BIS/NoGo/indirect pathways of the basal
ganglia with low DA shifting the balance towards the NoGo pathway
(impairing learning from positive feedback/reward relative to learning
from punishment) and high DA shifting the balance towards the Go pathway
(improving learning from positive feedback/reward relative to learning
from punishment). Several other studies reported impairments in learning
from punishment by dopaminergic therapy (Bodi et al., 2009; Cools,
Barker, Sahakian \& Robbins, 2001; Cools, Frank, Gibbs, Miyakawa, Jagust
\& D'Esposito, 2009; Moustafa, Cohen, Sherman \& Frank, 2008; Swainson,
Rogers, Sahakian, Summers, Polkey \& Robbins, 2000). In the study by
Pessiglione and colleagues (2006), enhancement of DA activity by
administration of L-3,4-dihydroxyphenylalanine (L-DOPA) increased the
frequency with which subjects chose high-probability gain but not the
frequency with which they chose low-probability loss. It has been
proposed that pharmacological boosting of DA neurotransmission increases
tonic levels of DA within the striatum, which in turn occludes DA dips,
which signal negative feedback, differentially affecting reward versus
punishment-based learning (Frank, 2005; Grace \& Rosenkranz, 2002). The
role of DA in regulating feedback sensitivity has been recently
confirmed in a preclinical study by Grospe and collaborators (2018). In
that study, 6-hydroxydopamine-induced dopaminergic lesions within the
rat dorsomedial striatum resulted in significantly increased negative
reinforcement sensitivity. Notably, the sensitivity to reinforcement and
associated cognitive biases also seem to be regulated at the receptor
level. The important role of dopamine D\textsubscript{2} receptor gene
polymorphism (DRD2-TAQ-IA) in reinforcement learning has been
demonstrated by Klein and colleagues (2007) using a neuroimaging
paradigm. In this study, in a probabilistic learning task, A1 allele
carriers with reduced dopamine D\textsubscript{2} receptor densities
learned to avoid actions with negative consequences less efficiently
than those without it. A1 subjects have also been shown to be impaired
in the ability to sustain a newly rewarded response after a reversal and
demonstrated a generally decreased tendency to stick with a rewarded
response (Jocham, Klein, Neumann, von Cramon, Reuter \& Ullsperger,
2009). In 2015, Cox and collaborators (2015), using positron emission
tomography (PET) with two selective DA receptor radioligands
({[}11C{]}SCH23390 and {[}11C{]}raclopride), demonstrated that
individual differences in dopaminergic D\textsubscript{1} and
D\textsubscript{2} receptor binding determine the effectiveness of
learning from positive and negative reinforcement, respectively. Studies
in humans were complemented by a series of elegant studies using animal
models. A study by Groman and collaborators (2016), using PET in rats
performing a three-choice spatial PRL task, demonstrated a role for
dopamine D\textsubscript{3}receptors in reinforcement learning. In that
study, greater midbrain dopamine D\textsubscript{3} receptor
availability (indicated by {[}11C{]}-(+)-PHNO binding) was associated
with a lower sensitivity to positive reinforcement, resulting in a lower
rate of learning. The role of dopamine D\textsubscript{3} receptors in
sensitivity to reinforcement was further confirmed following
administration of a dopamine
D\textsubscript{3}\textbackslash{}sout-receptor agonist, pramipexole,
which impaired the performance of rats in a very similar way (Groman et
al., 2016). These results suggest that in addition to dopamine
D\textsubscript{1}/D\textsubscript{2}, dopamine
D\textsubscript{3}receptor dysregulation may also underlie abnormal
reinforcement sensitivity, and they implicate that these receptors may
be a novel target for AUD treatment. In , Sharot and colleagues
demonstrated that administration of L-DOPA during the imaginative
construction of positive future life events subsequently enhanced
estimates of the hedonic pleasure to be derived from these same events
(Sharot, Shiner, Brown, Fan \& Dolan, 2009). This study was supported by
a report in 2012 (Sharot, Guitart-Masip, Korn, Chowdhury \& Dolan,
2012), which revealed that administration of L-DOPA increased optimism
bias by impairing the ability to update beliefs in response to
undesirable information about the future. The latter converged with the
above-mentioned observations from patients with Parkinson's disease,
where enhanced DA levels led to impaired learning from unwanted outcomes
(Frank, Seeberger \& O'Reilly R, 2004). These findings provided evidence
that DA modulates subjective hedonic expectations and impacts belief
formation by reducing negative expectations regarding the future.

Along with DA, serotonin (5-HT) is the second neurotransmitter crucially
implicated in reinforcement learning. Published reports (Bari et al.,
2010; Chamberlain, Muller, Blackwell, Clark, Robbins \& Sahakian, 2006;
Cools, Robinson \& Sahakian, 2008; Fischer \& Ullsperger, 2017; Rygula
et al., 2015; Sachs, Rodriguiz, Tran, Iyer, Wetsel \& Caron, 2015) have
suggested that increasing 5-HT neurotransmission leads to a reduced
sensitivity to aversive outcomes, whereas reducing 5-HT transmission, by
way of either acute tryptophan depletion (ATD), a procedure that has
been used extensively to study the effect of low 5-HT levels in the
human brain, pre-synaptic receptor stimulation (acting to temporarily
down-regulate 5-HT transmission), or up-regulation of the serotonin
transporter (SERT), leads to an increased sensitivity to aversive
outcomes. Indeed, a study by Chamberlain and colleagues (2006)
demonstrated that a low, acute dose of the selective serotonin reuptake
inhibitor (SSRI) citalopram, which has been postulated to affect
pre-synaptic serotonin 5-HT\textsubscript{1A} autoreceptors, increases
the sensitivity to negative feedback in the PRL task. Similar effects
were reported in a study by Cools and collaborators (2008), where ATD
enhanced the ability of subjects to predict punishment without affecting
reward predictions. The results of the above-mentioned studies were
complemented by the report by den Ouden and collaborators (2013), who
studied the role of 5-HT (and DA) in reinforcement sensitivity as a
function of two polymorphisms in the genes encoding the 5-HT and DA
transporters (SERT: 5HTTLPR plus rs25531; DAT1 3'UTR VNTR). The results
of this study revealed that allelic variation in SERT predicted
behavioural adaptation following punishment. Specifically, L'
homozygosity, which has been linked with increased SERT binding and
decreased levels of extracellular 5-HT (Willeit \& Praschak-Rieder,
2010), was associated with increased negative reinforcement learning
(den Ouden et al., 2013). The role of serotonin in RBCBs has been
supported by ample evidence from research using animal models. In 2010,
Bari and collaborators (2010) demonstrated that different manipulations
of 5-HT neurotransmission in rats resulted in altered sensitivity to
positive and negative reinforcement. In this study, acute administration
of a high dose of SSRI citalopram decreased negative feedback
sensitivity by lowering the ratio of lose-shift behaviours. In contrast,
acute administration of a low dose of this drug, which was postulated to
temporarily silence 5-HT system activity via inhibitory serotonin
5-HT\textsubscript{1A} autoreceptor activation in the raphe nuclei,
similar to the above-mentioned study by Chamberlain and collaborators
(2006), increased sensitivity to negative reinforcement. A similar
effect was reported following global 5-HT depletion (Bari et al., 2010).
In 2012, Ineichen and colleagues (2012) demonstrated in an automated
two-choice operant spatial discrimination paradigm that genetic
reduction in SERT function investigated in heterozygous mutant mice from
a SERT null mutant strain led to a decreased sensitivity to negative
feedback, which was an effect similar to that observed by Bari and
collaborators (2010) following acute administration of the higher dose
of citalopram. Both of the mentioned manipulations also caused increased
ratios of win-stay behaviours, indicating increased sensitivity to
positive reinforcement (Ineichen et al., 2012), an effect confirmed
recently by Wilkinson and colleagues (2020). The results of the study by
Rygula and collaborators (2014) revealed the important role of 5-HT in
modulating cognitive judgement bias in rats. In that study, the SSRI
citalopram at a low dose of 1 mg/kg significantly biased animals towards
positive interpretation of the ambiguous cues, while at higher doses (5
and 10 mg/kg), the animals interpreted the ambiguous cues more
negatively. Interestingly, a study from 2017 further demonstrated that
the effects of acute 5-HT manipulations on the interpretation of
ambiguity might depend on the basal valence of cognitive judgement bias
(Golebiowska \& Rygula, 2017). In that study, acute administration of
escitalopram caused a 'pessimistic' shift in the interpretation of
ambiguous cues in animals classified as `optimistic' and had no
significant effects on those previously classified as 'pessimistic'.

Taken together, published reports suggest that increasing 5-HT
transmission leads to a reduced sensitivity to aversive outcomes,
whereas reducing 5-HT transmission, by way of either ATD, pre-synaptic
receptor stimulation, or up-regulation of SERT, leads to an increased
sensitivity to aversive outcomes. These results strongly suggest 5-HT as
a potential target and serotonergic manipulations as effective treatment
strategies in modulating alcohol-drinking outcome expectations.

To summarize this section, psychopharmacologic manipulation of DA and
5-HT neurotransmitter systems implicated in RBCBs may have the potential
to provide insights into the development and maintenance of alcohol
addiction and should be considered as targets for pharmacological
treatment of AUD (Figure 3). Of particular interest is the potential to
experimentally vary reinforcement sensitivity and outcome expectations
and to do so in a way that builds a direct paradigmatic bridge with the
relevant alcohol literature. Further challenges include identification
of the specific interactions between neurochemical correlates of RBCBs
and brain processes involved in alcohol addiction.

\textbf{Preclinical and translational studies using state-of-the-art
animal models that are useful for elucidating the role of RBCBs in
various aspects of AUD and its treatment}

Creating preclinical models is essential for elucidating the
reinforcement-based cognitive mechanisms that contribute to
addiction-related behaviours. However, the main difficulty with
modelling RBCBs using non-human animals is capturing these inherently
complex processes using relatively simple behavioural protocols. One of
the most promising preclinical tools that can be used for the
investigation of biased sensitivity to reinforcement in animal models is
mentioned in the previous sections, the PRL paradigm (Figure 4). In the
rat version of the PRL task, the animals are trained to press levers
(Rychlik, Bollen \& Rygula, 2017; Rygula \& Popik, 2016) or nose-poke
holes (Bari et al., 2010) in an operant conditioning chamber to receive
a food reward. Choosing one of them (the `correct' lever or hole) is
associated with a high probability of receiving a reward (a drop of
sucrose solution or reward pellet) and a low probability of receiving a
punishment (mild electric foot shock or lack of reward and time out).
Conversely, choosing the other one (the `incorrect' lever or hole) is
associated with a high probability of receiving the punishment and only
a small chance of receiving the reward. The probabilities of receiving
the reward and punishment upon choosing the `correct' lever or hole are
usually set to 80\% and 20\%, respectively, and vice versa for the
`incorrect' lever or hole. The animals have to adjust their behaviour by
responding to the appropriate levers or holes to maximize reward and
minimize punishment delivery while disregarding occasional misleading
positive or negative feedback. After several consecutive choices of the
`correct' lever or hole, the reversal criterion is reached, and the
probabilities reverse; that is, the previously `correct' lever or hole
becomes `incorrect', and vice versa. Similar to humans, to assess the
sensitivity of the animals to positive and negative reinforcement,
behaviour is analysed according to the outcome of each preceding trial
and based on the WSLS analysis, where the proportion of win-stay
behaviours indexes sensitivity to positive reinforcement while the
proportion of lose-shifts indicates sensitivity to negative
reinforcement. In the aforementioned landmark study, using a PRL task
analogous to that used in humans, Bari and colleagues (2010)
demonstrated that different manipulations of 5-HT neurotransmission in
rats resulted in significant changes in reinforcement sensitivity. These
results were further complemented by a study in a non-human primate, the
marmoset, employing a preclinical and translational version of the PRL
task, which demonstrated that 5-HT depletions within the orbitofrontal
cortex and amygdala manifested impairments in overall reinforcement
sensitivity rates, including temporal learning from both rewarding and
negative outcomes (Rygula et al., 2015). A study by Rychlik and
collaborators (2017) demonstrated, using a preclinical version of the
PRL paradigm, that the non-monoaminergic compound ketamine selectively
down-regulates sensitivity to negative outcomes in animals. In this
study, acute treatment with ketamine significantly and persistently
decreased the ratio of lose-shift behaviours in a manner similar to that
observed following the administration of higher doses of the SSRI
citalopram (Bari et al., 2010). These results were recently supported by
a study by Wilkinson and collaborators (2020), who also observed an
interaction between ketamine treatment and feedback type with a trend
towards decreased negative feedback sensitivity in the preclinical PRL
paradigm. Ketamine was also reported to have robust positive effects on
the interpretation of ambiguous cues (Hales, Houghton \& Robinson,
2017). Given the similarity in the effects of citalopram and ketamine,
it was hypothesized that the effects of the latter were mediated
indirectly by 5-HT neurotransmission (Rychlik, Bollen \& Rygula, 2017).
Indeed, recent reports have demonstrated that in addition to affecting
glutamatergic neurotransmission via N-methyl-D-aspartate receptors
(NMDARs), ketamine also potentiates 5-HT release in the prefrontal
cortex via amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptors
(AMPARs) in the raphe nucleus (Nishitani et al., 2014). As demonstrated
by recent studies (Drozd, Rychlik, Fijalkowska \& Rygula, 2019; Noworyta
\& Rygula, 2021; Rygula \& Popik, 2016), the preclinical version of the
PRL task may be used not only for investigating neuroanatomical or
neurochemical bases of reinforcement sensitivity but also for evaluating
how this sensitivity interacts with other cognitive processes and
pharmacological treatment outcomes. In a study by Rygula and Popik
(2016), the PRL task was employed to investigate how 'optimistic' and
'pessimistic' animals incorporated feedback (both rewarding and
punishing) in their decisions in a changing and uncertain environment.
The results of this study demonstrated the interrelation and
co-existence of two cognitive biases (the rats classified as
`pessimistic' were significantly more sensitive to negative feedback
than their `optimistic' conspecifics) that may predict vulnerability to
various psychiatric disorders, including AUD. A study by
Noworyta-Sokolowska and colleagues (2019) demonstrated for the first
time in rodents that sensitivity to negative and positive reinforcement
could be considered a stable and enduring behavioural trait. It also
showed that these traits were independent of each other and that trait
sensitivity to positive reinforcement is associated with cognitive
flexibility. These results have been supported by computational
modelling (Noworyta-Sokolowska, Kozub, Jablonska, Rodriguez Parkitna,
Drozd \& Rygula, 2019). Several preclinical studies in rats have
investigated the neuroanatomical background of biased sensitivity to
reinforcement. In 2014, Dalton and colleagues (2014), using a
preclinical version of the PRL paradigm, found that the nucleus
accumbens shell and core facilitate reward-seeking in a distinct yet
complementary manner, with the shell guiding response selection to the
actions more likely to yield the reward and the core simply promoting
the approach towards reward-associated stimuli. In 2016, the same group
demonstrated that inactivation of the medial orbitofrontal cortex
rendered animals less sensitive to either positive or negative feedback,
while lateral orbitofrontal cortex activity was implicated in behaviours
following violations of reward expectancies signalled by negative
feedback (Dalton, Wang, Phillips \& Floresco, 2016). In the same study,
inactivation of the prelimbic cortex increased sensitivity to positive
feedback and reduced sensitivity to negative feedback. Since the animals
tended to select the recently rewarded choice more often, regardless of
whether the previous choice was correct or not, this surprising effect
has been proposed to result from a form of `reward myopia' (Dalton,
Wang, Phillips \& Floresco, 2016). In 2019, Drozd and colleagues (2019)
reported the reinforcement modulating effects of the antidepressant
drugs mirtazapine and agomelatine in the PRL task, and these results
were supported a year later in a study by Wilkinson and collaborators
(2020). Recent research using a preclinical version of the PRL paradigm
identified 4 phenotypes of sensitivity to negative and positive
reinforcement in rats and reported statistically significant differences
between the investigated phenotypes in the effects of acute treatment
with SSRI escitalopram on anxiety (Noworyta \& Rygula, 2021). These
results demonstrated that trait sensitivity to reinforcement could have
important implications for the effectiveness of pharmacological
interventions, including in AUD. Last but not least, as mentioned above,
the fear of negative outcomes has a powerful influence on decisions
regarding drinking. A study by Clarke and colleagues (2014) in marmoset
monkeys demonstrated, using an approach-avoidance conflict task, that
inactivation of anterior orbitofrontal or ventrolateral prefrontal
cortices increased general negative bias in decision making via two
distinct cognitive mechanisms---elevated uncertainty and attentional
disruption, respectively. The differentiation of the component neural
mechanisms underlying punishment processing revealed in that study
provided important insight into the efficacy of cognitive-behavioural
therapy in AUD, which may be more successful in a patient poor at
predicting than in one deficient in attentional control.

\textbf{Conclusions and future directions}

RBCBs occur in many neuropsychiatric disorders and may even be helpful
in defining them. Over the last decade, the consideration of RBCBs as
important factors in the development of alcohol addiction has enriched
and strengthened the field of inquiry regarding AUD and opened new
avenues of research. To further define the relationships between RBCBs
and AUD, in this review, we have advocated a psychological approach
based largely on cognitive and learning theories, which combined with
neural and neurochemical analyses may facilitate our understanding of
this disease. We have also highlighted the development of translational
tools, which herald a new phase of research in this field, that will
more readily make contact between the clinic and basic neuroscience. We
have also described the effects of pharmacological manipulations, as a
multi-level analysis of RBCBs may identify medications that can be used
for different expressions of what may, in fact, be common disorders
resulting from reinforcement biased cognition. The possible
`re-purposing' of agents such as SSRIs and ketamine may provide an
initial impetus for this enterprise. Finally, we also mentioned the
concept of endophenotypes, as a focus on individual differences in this
diverse disorder may inform genetic approaches to understanding its
aetiology. Based on available knowledge, it seems that using
rodent-based models, such as the preclinical PRL or ACI, can help to
reveal neurobiological processes that modulate RBCBs and their role in
AUD. This in turn may help to reveal novel drug targets for improved and
personalized treatments. Although we hope this review has provided
enough evidence to support the validity of the claim that RBCBs can
determine the trajectories of alcohol addiction, there are still a
number of outstanding issues that future research will need to address.
First, we still do not know the degree of the causal relationship
between increased/decreased sensitivity to negative/positive
reinforcement as a trait and vulnerability to AUD. The development of
translational preclinical tests of RBCBs should help to elucidate this
issue and may help to design personalized treatments based on these
cognitive variables. Second, although we know that there are distributed
changes in neural activity within multiple regions of the PFC and the
striatum associated with altered activity of serotonergic and
dopaminergic systems that occur over the course of AUD and can persist
into periods of abstinence, unfortunately, studies looking at
neurochemical correlates of altered reinforcement sensitivity in this
context are still limited, and further investigation in this area is
needed. We also still need more detailed pharmacological studies using
drugs with known profiles in humans to understand the value of targeting
RBCBs in AUD. It will be highly desirable to use voltammetry,
optogenetics, or other biosensors and electrophysiological measures to
characterize neuronal pathways and to elucidate the exact function and
dynamic balance between cortical and subcortical regions involved in
RBCBs and AUD. Finally, further conceptual and empirical development is
required to provide an integrated account of the role of reinforcement
sensitivity in the aetiology, development, and recurrence of AUD.

\textbf{Acknowledgement}

This work was supported by the Polish National Science Centre (Research
grant 2018/31/B/NZ7/03690 to RR) and by the statutory funds of the Maj
Institute of Pharmacology Polish Academy of Sciences.

\textbf{Figures}

\emph{\textbf{Figure 1.} Sensitivity of the behavioural activation
system (BAS) and the behavioural inhibition system (BIS) is associated
with various reward- and punishment-driven processes. Abnormal
sensitivity of these systems induces reinforcement-based cognitive
biases (RBCBs) affecting the development and trajectory of alcohol use
disorder (AUD)} \textbf{.}

\emph{\textbf{Figure 2.} Reinforcement-based cognitive biases associated
with the pathophysiology of alcohol use disorder (AUD) include abnormal
sensitivity to positive and negative feedback, biased interpretation of
ambiguity and conflict situations, lowered negative and inflated
positive outcome expectations, and unrealistic optimism. Understanding
the underlying mechanisms could provide useful information for tailoring
prevention and intervention efforts.}

\emph{\textbf{Figure 3.} Dopamine (DA) and serotonin (5-HT) modulate
reinforcement-based cognitive biases. Several pharmacological
manipulations (selective serotonin reuptake inhibitors (SSRIs) and
L-3,4-dihydroxyphenylalanine (L-DOPA)) have been demonstrated to
modulate feedback sensitivity and outcome expectations in humans and
animal models. Pharmacological targeting of these biases may help to
shape the trajectories of alcohol use disorder (AUD).}

\emph{\textbf{Figure 4.} The probabilistic reversal learning (PRL)
paradigm. One of the most promising, translational tools that can be
used for the investigation of biased sensitivity to reinforcement in
animal models and in humans.}

\textbf{References}

Alloy LB, Peterson C, Abramson LY, \& Seligman ME (1984). Attributional
style and the generality of learned helplessness. Journal of personality
and social psychology 46\textbf{:} 681.

Bari A, Theobald DE, Caprioli D, Mar AC, Aidoo-Micah A, Dalley JW\emph{,
et al.} (2010). Serotonin modulates sensitivity to reward and negative
feedback in a probabilistic reversal learning task in rats.
Neuropsychopharmacology 35\textbf{:} 1290-1301.

Beauchaine T (2001). Vagal tone, development, and Gray's motivational
theory: toward an integrated model of autonomic nervous system
functioning in psychopathology. Dev Psychopathol 13\textbf{:} 183-214.

Beevers CG, Wells TT, Ellis AJ, \& Fischer K (2009). Identification of
emotionally ambiguous interpersonal stimuli among dysphoric and
nondysphoric individuals. Cognitive therapy and research
33\textbf{:}283-290.

Blume AW, \& Blume AK (2014). Alcohol outcome expectancies and
consequences: Do people think themselves into and out of consequences.
Psychology Applications \& Developments\textbf{:} 17-23.

Blume AW, Lostutter TW, Schmaling KB, \& Marlatt GA (2003). Beliefs
about drinking behavior predict drinking consequences. J Psychoactive
Drugs 35\textbf{:} 395-399.

Bodi N, Keri S, Nagy H, Moustafa A, Myers CE, Daw N\emph{, et
al.}(2009). Reward-learning and the novelty-seeking personality: a
between- and within-subjects study of the effects of dopamine agonists
on young Parkinson's patients. Brain 132\textbf{:} 2385-2395.

Bromberg-Martin ES, Matsumoto M, \& Hikosaka O (2010). Dopamine in
motivational control: rewarding, aversive, and alerting. Neuron
68\textbf{:} 815-834.

Brown SA, Christiansen BA, \& Goldman MS (1987). The Alcohol Expectancy
Questionnaire: an instrument for the assessment of adolescent and adult
alcohol expectancies. Journal of studies on alcohol 48\textbf{:}483-491.

Brown SA, Goldman MS, Inn A, \& Anderson LR (1980). Expectations of
reinforcement from alcohol: their domain and relation to drinking
patterns. Journal of consulting and Clinical Psychology 48\textbf{:}419.

Carver CS, \& White TL (1994). Behavioral-Inhibition, Behavioral
Activation, and Affective Responses to Impending Reward and Punishment -
the Bis Bas Scales. Journal of Personality and Social Psychology
67\textbf{:} 319-333.

Chamberlain SR, Muller U, Blackwell AD, Clark L, Robbins TW, \& Sahakian
BJ (2006). Neurochemical modulation of response inhibition and
probabilistic learning in humans. Science 311\textbf{:} 861-863.

Christiansen BA, \& Goldman MS (1983). Alcohol-related expectancies
versus demographic/background variables in the prediction of adolescent
drinking. Journal of Consulting and Clinical Psychology
51\textbf{:}249-257.

Clarke HF, Cardinal RN, Rygula R, Hong YT, Fryer TD, Sawiak SJ\emph{, et
al.} (2014). Orbitofrontal dopamine depletion upregulates caudate
dopamine and alters behavior via changes in reinforcement sensitivity. J
Neurosci 34\textbf{:} 7663-7676.

Cools R, Barker RA, Sahakian BJ, \& Robbins TW (2001). Enhanced or
impaired cognitive function in Parkinson's disease as a function of
dopaminergic medication and task demands. Cereb Cortex
11\textbf{:}1136-1143.

Cools R, Clark L, Owen AM, \& Robbins TW (2002). Defining the neural
mechanisms of probabilistic reversal learning using event-related
functional magnetic resonance imaging. J Neurosci 22\textbf{:}4563-4567.

Cools R, Frank MJ, Gibbs SE, Miyakawa A, Jagust W, \& D'Esposito M
(2009). Striatal dopamine predicts outcome-specific reversal learning
and its sensitivity to dopaminergic drug administration. J Neurosci
29\textbf{:} 1538-1543.

Cools R, Roberts AC, \& Robbins TW (2008). Serotoninergic regulation of
emotional and behavioural control processes. Trends Cogn Sci
12\textbf{:} 31-40.

Cools R, Robinson OJ, \& Sahakian B (2008). Acute tryptophan depletion
in healthy volunteers enhances punishment prediction but does not affect
reward prediction. Neuropsychopharmacology 33\textbf{:} 2291-2299.

Corr PJ (2004). Reinforcement sensitivity theory and personality.
Neurosci Biobehav Rev 28\textbf{:} 317-332.

Corr PJ (2008). The reinforcement sensitivity theory of personality and
psychopathology. Int J Psychophysiol 69\textbf{:} 151-152.

Cox SM, Frank MJ, Larcher K, Fellows LK, Clark CA, Leyton M\emph{, et
al.} (2015). Striatal D1 and D2 signaling differentially predict
learning from positive and negative outcomes. Neuroimage
109\textbf{:}95-101.

Cox WM, \& Klinger E (2011). A motivational model of alcohol use:
Determinants of use and change. In Handbook of motivational counseling:
Goal-based approaches to assessment and intervention with addiction and
other problems, 2nd ed. Wiley Blackwell, pp 131-158.

Curry S, Marlatt GA, \& Gordon JR (1987). Abstinence violation effect:
validation of an attributional construct with smoking cessation. Journal
of Consulting and Clinical Psychology 55\textbf{:} 145.

Dalton GL, Phillips AG, \& Floresco SB (2014). Preferential involvement
by nucleus accumbens shell in mediating probabilistic learning and
reversal shifts. J Neurosci 34\textbf{:} 4618-4626.

Dalton GL, Wang NY, Phillips AG, \& Floresco SB (2016). Multifaceted
Contributions by Different Regions of the Orbitofrontal and Medial
Prefrontal Cortex to Probabilistic Reversal Learning. J Neurosci
36\textbf{:} 1996-2006.

DeJoy DM (1989). An Attribution Theory Perspective on Alcohol-Impaired
Driving. Health Education Quarterly 16\textbf{:} 359-372.

den Ouden HE, Daw ND, Fernandez G, Elshout JA, Rijpkema M, Hoogman
M\emph{, et al.} (2013). Dissociable effects of dopamine and serotonin
on reversal learning. Neuron 80\textbf{:} 1090-1100.

Drozd R, Rychlik M, Fijalkowska A, \& Rygula R (2019). Effects of
cognitive judgement bias and acute antidepressant treatment on
sensitivity to feedback and cognitive flexibility in the rat version of
the probabilistic reversal-learning test. Behav Brain Res
359\textbf{:}619-629.

Drury DA (1969). Homographs and pseudo-homographs. Word Ways
2\textbf{:}10.

Eisenegger C, Naef M, Linssen A, Clark L, Gandamaneni PK, M\selectlanguage{ngerman}üller
U\emph{, et al.} (2014). Role of dopamine D2 receptors in human
reinforcement learning. Neuropsychopharmacology 39\textbf{:} 2366-2375.

Everitt BJ, \& Robbins TW (2005). Neural systems of reinforcement for
drug addiction: from actions to habits to compulsion. Nat Neurosci
8\textbf{:} 1481-1489.

Evers EA, Cools R, Clark L, van der Veen FM, Jolles J, Sahakian
BJ\emph{, et al.} (2005). Serotonergic modulation of prefrontal cortex
during negative feedback in probabilistic reversal learning.
Neuropsychopharmacology 30\textbf{:} 1138-1147.

Feil J, \& Hasking P (2008). The relationship between personality,
coping strategies and alcohol use. Addiction Research \& Theory
16\textbf{:} 526-537.

Field M, \& Cox WM (2008). Attentional bias in addictive behaviors: a
review of its development, causes, and consequences. Drug and alcohol
dependence 97\textbf{:} 1-20.

Fischer AG, \& Ullsperger M (2017). An update on the role of serotonin
and its interplay with dopamine for reward. Frontiers in human
neuroscience 11\textbf{:} 484.

Floyd FJ, \& Widaman KF (1995). Factor analysis in the development and
refinement of clinical assessment instruments. Psychological assessment
7\textbf{:} 286.

Fowles DC (1988). Psychophysiology and psychopathology: A motivational
approach. Psychophysiology 25\textbf{:} 373-391.

Frank MJ (2005). Dynamic dopamine modulation in the basal ganglia: a
neurocomputational account of cognitive deficits in medicated and
nonmedicated Parkinsonism. J Cogn Neurosci 17\textbf{:} 51-72.

Frank MJ, Seeberger LC, \& O'Reilly R C (2004). By carrot or by stick:
cognitive reinforcement learning in parkinsonism. Science
306\textbf{:}1940-1943.

Franken IHA (2002). Behavioral approach system (BAS) sensitivity
predicts alcohol craving. Personality and Individual Differences
32\textbf{:} 349-355.

Franken IHA, \& Muris P (2006). BIS/BAS personality characteristics and
college students' substance use. Personality and Individual Differences
40\textbf{:} 1497-1503.

Fromme K, \& D'Amico EJ (2000). Measuring adolescent alcohol outcome
expectancies. Psychology of addictive behaviors 14\textbf{:} 206.

Fromme K, Stroot EA, \& Kaplan D (1993). Comprehensive effects of
alcohol: Development and psychometric assessment of a new expectancy
questionnaire. Psychological Assessment 5\textbf{:} 19-26.

Goldman MS (1994). The alcohol expectancy concept: Applications to
assessment, prevention, and treatment of alcohol abuse. Applied and
Preventive Psychology 3\textbf{:} 131-144.

Golebiowska J, \& Rygula R (2017). Lesions of the Orbitofrontal but Not
Medial Prefrontal Cortex Affect Cognitive Judgment Bias in Rats. Front
Behav Neurosci 11\textbf{:} 51.

Gorfein DS, \& Weingartner KM (2008). On the norming of homophones.
Behavior Research Methods 40\textbf{:} 522-530.

Grace AA, \& Rosenkranz JA (2002). Regulation of conditioned responses
of basolateral amygdala neurons. Physiology \& behavior
77\textbf{:}489-493.

Groman SM, Smith NJ, Petrullli JR, Massi B, Chen L, Ropchan J\emph{, et
al.} (2016). Dopamine D3 Receptor Availability Is Associated with
Inflexible Decision Making. J Neurosci 36\textbf{:} 6732-6741.

Grospe GM, Baker PM, \& Ragozzino ME (2018). Cognitive Flexibility
Deficits Following 6-OHDA Lesions of the Rat Dorsomedial Striatum.
Neuroscience 374\textbf{:} 80-90.

Hales CA, Houghton CJ, \& Robinson ES (2017). Behavioural and
computational methods reveal differential effects for how delayed and
rapid onset antidepressants effect decision making in rats. European
neuropsychopharmacology 27\textbf{:} 1268-1280.

Hasking PA (2006). Reinforcement sensitivity, coping, disordered eating
and drinking behaviour in adolescents. Personality and Individual
Differences 40\textbf{:} 677-688.

Hindash AHC, \& Amir N (2012). Negative interpretation bias in
individuals with depressive symptoms. Cognitive therapy and research
36\textbf{:} 502-511.

Hundt NE, Kimbrel NA, Mitchell JT, \& Nelson-Gray RO (2008). High BAS,
but not low BIS, predicts externalizing symptoms in adults. Personality
and Individual Differences 44\textbf{:} 565-575.

Iaboni F, Douglas VI, \& Ditto B (1997). Psychophysiological response of
ADHD children to reward and extinction. Psychophysiology
34\textbf{:}116-123.

Ineichen C, Sigrist H, Spinelli S, Lesch KP, Sautter E, Seifritz
E\emph{, et al.} (2012). Establishing a probabilistic reversal learning
test in mice: evidence for the processes mediating reward-stay and
punishment-shift behaviour and for their modulation by serotonin.
Neuropharmacology 63\textbf{:} 1012-1021.

Jessup MA, Ross TB, Jones AL, Satre DD, Weisner CM, Chi FW\emph{, et
al.} (2014). Significant life events and their impact on alcohol and
drug use: a qualitative study. Journal of psychoactive drugs
46\textbf{:} 450-459.

Jocham G, Klein TA, Neumann J, von Cramon DY, Reuter M, \& Ullsperger M
(2009). Dopamine DRD2 polymorphism alters reversal learning and
associated neural activity. J Neurosci 29\textbf{:} 3695-3704.

Jones BT, Corbin W, \& Fromme K (2001). Half full or half empty, the
glass still does not satisfactorily quench the thirst for knowledge on
alcohol expectancies as a mechanism of change.

Kambouropoulos N, \& Staiger PK (2001). The influence of sensitivity to
reward on reactivity to alcohol-related cues. Addiction
96\textbf{:}1175-1185.

Kellough JL, Beevers CG, Ellis AJ, \& Wells TT (2008). Time course of
selective attention in clinically depressed young adults: an eye
tracking study. Behav Res Ther 46\textbf{:} 1238-1243.

Kimbrel NA, Nelson-Gray RO, \& Mitchell JT (2007). Reinforcement
sensitivity and maternal style as predictors of psychopathology.
Personality and Individual Differences 42\textbf{:} 1139-1149.

Klein TA, Neumann J, Reuter M, Hennig J, von Cramon DY, \& Ullsperger M
(2007). Genetically determined differences in learning from errors.
Science 318\textbf{:} 1642-1645.

Knyazev GG, Slobodskaya HR, Kharchenko II, \& Wilson GD (2004).
Personality and substance use in Russian youths: The predictive and
moderating role of behavioural activation and gender. Personality and
Individual Differences 37\textbf{:} 827-843.

Koob GF (2003). Alcoholism: allostasis and beyond. Alcohol Clin Exp Res
27\textbf{:} 232-243.

Koob GF, \& Volkow ND (2010). Neurocircuitry of addiction.
Neuropsychopharmacology 35\textbf{:} 217-238.

Lee CM, Maggs JL, Neighbors C, \& Patrick ME (2011). Positive and
negative alcohol-related consequences: associations with past drinking.
J Adolesc 34\textbf{:} 87-94.

Leeman RF, Toll BA, Taylor LA, \& Volpicelli JR (2009). Alcohol-induced
disinhibition expectancies and impaired control as prospective
predictors of problem drinking in undergraduates. Psychology of
Addictive Behaviors 23\textbf{:} 553.

Leigh BC (1989). In search of the seven dwarves: Issues of measurement
and meaning in alcohol expectancy research. Psychological bulletin
105\textbf{:} 361.

Loxton NJ, \& Dawe S (2001). Alcohol abuse and dysfunctional eating in
adolescent girls: The influence of individual differences in sensitivity
to reward and punishment. International Journal of Eating Disorders
29\textbf{:} 455-462.

Mann K, Roos CR, Hoffmann S, Nakovics H, Leménager T, Heinz A\emph{, et
al.} (2018). Precision medicine in alcohol dependence: a controlled
trial testing pharmacotherapy response among reward and relief drinking
phenotypes. Neuropsychopharmacology 43\textbf{:} 891-899.

Marlatt GA, \& Donovan DM (2005) \emph{Relapse prevention: Maintenance
strategies in the treatment of addictive behaviors} . Guilford press.

McNaughton N, \& Gray JA (2000). Anxiolytic action on the behavioural
inhibition system implies multiple types of arousal contribute to
anxiety. Journal of affective disorders 61\textbf{:} 161-176.

Moustafa AA, Cohen MX, Sherman SJ, \& Frank MJ (2008). A role for
dopamine in temporal decision making and reward maximization in
parkinsonism. J Neurosci 28\textbf{:} 12294-12304.

Nishitani N, Nagayasu K, Asaoka N, Yamashiro M, Shirakawa H, Nakagawa
T\emph{, et al.} (2014). Raphe AMPA receptors and nicotinic
acetylcholine receptors mediate ketamine-induced serotonin release in
the rat prefrontal cortex. International Journal of
Neuropsychopharmacology 17\textbf{:} 1321-1326.

Norman P, Boer H, \& Seydel ER (2005). Protection motivation theory.
Predicting health behaviour 81\textbf{:} 126.

Noworyta K, \& Rygula R (2021). Phenotypes of reinforcement sensitivity
as predictors of the response to acute antidepressant treatment in rats.
European Neuropsychopharmacology 43\textbf{:} 102-115.

Noworyta-Sokolowska K, Kozub A, Jablonska J, Rodriguez Parkitna J, Drozd
R, \& Rygula R (2019). Sensitivity to negative and positive feedback as
a stable and enduring behavioural trait in rats. Psychopharmacology
(Berl) 236\textbf{:} 2389-2403.

O'Connor MJ, \& Paley B (2009). Psychiatric conditions associated with
prenatal alcohol exposure. Dev Disabil Res Rev 15\textbf{:} 225-234.

O'Connor RM, \& Colder CR (2005). Predicting Alcohol Patterns in
First-Year College Students Through Motivational Systems and Reasons for
Drinking. Psychology of Addictive Behaviors 19\textbf{:} 10-20.

O'Connor RM, Stewart SH, \& Watt MC (2009). Distinguishing BAS risk for
university students' drinking, smoking, and gambling behaviors.
Personality and Individual Differences 46\textbf{:} 514-519.

Ooteman W, Koeter M, Verheul R, Schippers G, \& Van den Brink W (2006).
Development and validation of the Amsterdam Motives for Drinking Scale
(AMDS): an attempt to distinguish relief and reward drinkers. Alcohol
Alcohol 41\textbf{:} 284-292.

Pabst A, Kraus L, Piontek D, \& Mueller S (2010). Age, period, and
cohort effects on time trends in alcohol consumption in the German adult
population. Sucht.

Papciak J, \& Rygula R (2017). Measuring Cognitive Judgement Bias in
Rats Using the Ambiguous-Cue Interpretation Test. Current protocols in
neuroscience / editorial board, Jacqueline N Crawley {[}et al{]}
78\textbf{:} 9 57 51-59 57 22.

Pardo Y, Aguilar R, Molinuevo B, \& Torrubia R (2007). Alcohol use as a
behavioural sign of disinhibition: Evidence from JA Gray's model of
personality. Addictive behaviors 32\textbf{:} 2398-2403.

Paulus MP, Hozack N, Frank L, \& Brown GG (2002). Error rate and outcome
predictability affect neural activation in prefrontal cortex and
anterior cingulate during decision-making. Neuroimage
15\textbf{:}836-846.

Paulus MP, Hozack N, Frank L, Brown GG, \& Schuckit MA (2003). Decision
making by methamphetamine-dependent subjects is associated with
error-rate-independent decrease in prefrontal and parietal activation.
Biol Psychiatry 53\textbf{:} 65-74.

Pessiglione M, Seymour B, Flandin G, Dolan RJ, \& Frith CD (2006).
Dopamine-dependent prediction errors underpin reward-seeking behaviour
in humans. Nature 442\textbf{:} 1042-1045.

Peterson C, Semmel A, Von Baeyer C, Abramson LY, Metalsky GI, \&
Seligman ME (1982). The attributional style questionnaire. Cognitive
therapy and research 6\textbf{:} 287-299.

Robinson TE, \& Berridge KC (1993). The Neural Basis of Drug Craving -
an Incentive-Sensitization Theory of Addiction. Brain Res Rev
18\textbf{:} 247-291.

Roos CR, Mann K, \& Witkiewitz K (2017). Reward and relief dimensions of
temptation to drink: construct validity and role in predicting
differential benefit from acamprosate and naltrexone. Addiction biology
22\textbf{:} 1528-1539.

Rychlik M, Bollen E, \& Rygula R (2017). Ketamine decreases sensitivity
of male rats to misleading negative feedback in a probabilistic
reversal-learning task. Psychopharmacology (Berl) 234\textbf{:} 613-620.

Rygula R, Clarke HF, Cardinal RN, Cockcroft GJ, Xia J, Dalley JW\emph{,
et al.} (2015). Role of Central Serotonin in Anticipation of Rewarding
and Punishing Outcomes: Effects of Selective Amygdala or Orbitofrontal
5-HT Depletion. Cereb Cortex 25\textbf{:} 3064-3076.

Rygula R, Noworyta-Sokolowska K, Drozd R, \& Kozub A (2018). Using
rodents to model abnormal sensitivity to feedback in depression.
Neurosci Biobehav Rev 95\textbf{:} 336-346.

Rygula R, Papciak J, \& Popik P (2014). The effects of acute
pharmacological stimulation of the 5-HT, NA and DA systems on the
cognitive judgement bias of rats in the ambiguous-cue interpretation
paradigm. European neuropsychopharmacology : the journal of the European
College of Neuropsychopharmacology 24\textbf{:} 1103-1111.

Rygula R, \& Popik P (2016). Trait ''pessimism'' is associated with
increased sensitivity to negative feedback in rats. Cogn Affect Behav
Neurosci 16\textbf{:} 516-526.

Sachs BD, Rodriguiz RM, Tran HL, Iyer A, Wetsel WC, \& Caron MG (2015).
Serotonin deficiency alters susceptibility to the long-term consequences
of adverse early life experience. Psychoneuroendocrinology
53\textbf{:}69-81.

Samson RD, Frank MJ, \& Fellous J-M (2010). Computational models of
reinforcement learning: the role of dopamine as a reward signal.
Cognitive Neurodynamics 4\textbf{:} 91-105.

Schaefer KL, Baumann J, Rich BA, Luckenbaugh DA, \& Zarate Jr CA (2010).
Perception of facial emotion in adults with bipolar or unipolar
depression and controls. Journal of Psychiatric Research
44\textbf{:}1229-1235.

Scheier MF, \& Carver CS (2018). Dispositional optimism and physical
health: A long look back, a quick look forward. American Psychologist
73\textbf{:} 1082.

Schick A, Wessa M, Vollmayr B, Kuehner C, \& Kanske P (2013). Indirect
assessment of an interpretation bias in humans: neurophysiological and
behavioral correlates. Frontiers in human neuroscience 7\textbf{:} 272.

Schulman P, Castellon C, \& Seligman ME (1989). Assessing explanatory
style: The content analysis of verbatim explanations and the
Attributional Style Questionnaire. Behaviour research and therapy
27\textbf{:} 505-509.

Sharot T, Guitart-Masip M, Korn CW, Chowdhury R, \& Dolan RJ (2012). How
dopamine enhances an optimism bias in humans. Curr Biol
22\textbf{:}1477-1481.

Sharot T, Korn CW, \& Dolan RJ (2011). How unrealistic optimism is
maintained in the face of reality. Nature neuroscience
14\textbf{:}1475-1479.

Sharot T, Shiner T, Brown AC, Fan J, \& Dolan RJ (2009). Dopamine
enhances expectation of pleasure in humans. Curr Biol
19\textbf{:}2077-2080.

Shepperd JA, Klein WM, Waters EA, \& Weinstein ND (2013). Taking Stock
of Unrealistic Optimism. Perspect Psychol Sci 8\textbf{:} 395-411.

Skinner MD, \& Aubin HJ (2010). Craving's place in addiction theory:
contributions of the major models. Neurosci Biobehav Rev
34\textbf{:}606-623.

Slaney C, Hinchcliffe JK, \& Robinson ESJ (2018). Translational Shifts
in Preclinical Models of Depression: Implications for Biomarkers for
Improved Treatments. Curr Top Behav Neurosci 40\textbf{:} 169-193.

Swainson R, Rogers RD, Sahakian BJ, Summers BA, Polkey CE, \& Robbins TW
(2000). Probabilistic learning and reversal deficits in patients with
Parkinson's disease or frontal or temporal lobe lesions: possible
adverse effects of dopaminergic medication. Neuropsychologia
38\textbf{:} 596-612.

Tranel DT (1983). The Effects of Monetary Incentive and Frustrative
Nonreward on Heart Rate and Electrodermal Activity. Psychophysiology
20\textbf{:} 652-657.

Voigt K, Twork S, Mittag D, Göbel A, Voigt R, Klewer J\emph{, et
al.}(2009). Consumption of alcohol, cigarettes and illegal substances
among physicians and medical students in Brandenburg and Saxony
(Germany). BMC Health Serv Res 9\textbf{:} 219.

Walton MA, Castro FG, \& Barrington EH (1994). The role of attributions
in abstinence, lapse, and relapse following substance abuse treatment.
Addict Behav 19\textbf{:} 319-331.

Wardell JD, O'Connor RM, Read JP, \& Colder CR (2011). Behavioral
approach system moderates the prospective association between the
behavioral inhibition system and alcohol outcomes in college students. J
Stud Alcohol Drugs 72\textbf{:} 1028-1036.

Warren RE, Warren NT, Green JP, \& Bresnick JH (1978). Multiple semantic
encoding of homophones and homographs in contexts biasing dominant or
subordinate meanings. Memory \& Cognition 6\textbf{:} 364-371.

Weiner B (1985). An attributional theory of achievement motivation and
emotion. Psychological Review 92\textbf{:} 548-573.

Wilkinson MP, Grogan JP, Mellor JR, \& Robinson ES (2020). Comparison of
conventional and rapid-acting antidepressants in a rodent probabilistic
reversal learning task. Brain and neuroscience advances
4\textbf{:}2398212820907177.

Willeit M, \& Praschak-Rieder N (2010). Imaging the effects of genetic
polymorphisms on radioligand binding in the living human brain: A review
on genetic neuroreceptor imaging of monoaminergic systems in psychiatry.
Neuroimage 53\textbf{:} 878-892.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-1/Figure-1}

\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-2/Figure-2}

\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-3/Figure-3}

\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Figure-4/Figure-4}

\end{center}
\end{figure}

\selectlanguage{english}
\FloatBarrier
\end{document}

