Livestock grazing can strongly determine how grasslands function and their role in carbon cycle. However, how ecosystem carbon exchange responds to grazing and the underlying mechanisms remain unclear. We measured ecosystem carbon fluxes to explore the changes in carbon exchange and their driving mechanisms in a 16-year long term experiment with different grazing intensities in a desert steppe grassland. We found that grazing intensity influenced above- and belowground biomass during the peak growing season, primarily by decreasing shrubs and semi-shrubs and perennial forbs. Furthermore, alter patterns of net ecosystem exchange primarily via their negative influence on the biomass of shrub and semi-shrub. In addition, grazing-induced reduction belowground biomass, as well as in total plant nitrogen and soil ammonium nitrogen, can strongly influence ecosystem carbon exchange and soil respiration. When nitrogen is lost from the soil due to grazing, plants reallocate resources belowground to maintain growth and development, thus promoting photosynthesis and respiration. Our study indicates that soil available nitrogen and shrubs and semi-shrubs are important factors in regulating ecosystem carbon exchange under grazing disturbance in the desert steppe, which provide a basis for grazing management.