In recent decades, alpine grassland has been serimously degraded across the Qinghai Tibetan Plateau (QTP), although grazing exclusion has been widely adopted to restore degraded QTP grassland. It remains unknown whether this management approach is effective for all degraded alpine grasslands. In this study, plots with three grazing management treatments (free grazing, FG; reduced grazing, RG; grazing exclusion, GE) and four degradation stages (non-degradation, ND; light degradation, LD; moderate degradation, MD; heavy degradation, HD) were compared. Our results showed that the total aboveground biomass (AGB) and species richness (SR) were reduced while total belowground biomass (BGB) increased with increasing degradation, and the responses of SR, AGB and BGB to grazing management varied with the degree of degradation. The total AGB in the LD, MD and HD stages reduced significantly after 6 years under RG and GE, but there was no significant change of AGB in the ND stage. Meanwhile, SR reduced significantly after 6 years under RG and GE across all degradation stages except for HD. Furthermore, the responses of plant functional groups to grazing management varied. After 6 years under RG and GE, the Gramineae AGB increased significantly across all degradation levels; that of the sedges decreased (except in the MD stage); and that of the forbs increased significantly in LD and HD but decreased significantly in ND. Our result suggested that the light degradation grassland can be restored by reducing grazing, and moderate degradation and heavy degradation grassland can restored by grazing exclusion.