Prior neuroimaging studies of music-evoked emotions have shown that music listening involves the activation of cortical and subcortical regions. However, these regions could be differentially activated by music stimuli with different affective valence. To better understand the neural correlates involved in the processing of pleasant and unpleasant emotions induced by music, we conducted a quantitative activation likelihood estimate (ALE) meta-analysis. We performed separate ALE analyses for the overall brain activation during music listening (63 studies), for the brain activation during listening to unpleasant music (23 studies) and for the brain activation while listening to pleasant music (21 studies). Our results showed an activation of a range of cortical and subcortical regions, including the amygdala, insula, striatum, thalamus, parahippocampal gyrus, anterior cingulate gyrus and superior temporal gyrus. Moreover, our findings showed that pleasant and unpleasant music specifically activated different brain regions. Particularly, unpleasant music activated the amygdala, hippocampus and the anterior cingulate cortex, whereas pleasant music activated the striatum, thalamus and the hippocampus. The identification of brain networks preferentially activated during listening to pleasant and unpleasant music provide useful clinical information for the development of therapies in psychological disorders with emotion reactivity problems.