Primary auditory cortical neurons show sensitivity to deviations in streams of sound tokens forming the basis of mismatch negativity. Models based on synaptic depression can explain the above phenomenon. However, such models can only explain selectivity towards oddball tones, based on the tonotopic arrangement of the auditory pathway. In the present study, we have used an oddball presentation of stimuli with different spectral contents to study the selectivity towards deviations in specific spectral shape of auditory stimuli. We used the metric of auditory contrast as a measure of the spectral shape. With single-unit in-vivo electrophysiology in the primary auditory cortex of adult mice, we found selectivity shown by the neurons towards specific spectral shapes of relatively high or low contrast value. The above selectivity was further dependent on the spectral context or spectral shape of the preceding stimulus tokens. Functional connectivity estimated using pairwise noise-correlations revealed differential connectivity within the local circuitry shown by neurons preferring spectral shapes of different contrast values in different contexts. Furthermore, using Jaccard index we found a high degree of heterogeneity among populations which prefer spectral shapes with low-contrast value. Using 2-photon Ca2+ imaging we investigated the role of two inhibitory interneuron types the Parvalbumin-expressing (PV) and the Somatostatin-expressing (SOM) in the selectivity for spectral shapes with different auditory contrast at different contexts. We found selective inhibitory subnetworks with specific excitatory neurons playing an important role in encoding for the spectral shape and context information in the auditory stimuli.