Plant-soil microbe linkage is theoretically positive, but rarely observed in natural ecosystems. We hypothesized that animal grazing decouples this linkage, for plant removal usually decreases plant influences on soil microbiota. We tested the hypothesis by exploring plant and soil microbial diversity in grazed versus non-grazed grasslands along a gradient of plant diversity. Grazing slightly decreased the diversity of plants and soil microbes in high-diversity sites, but increased them in low-diversity sites. Plant and microbial diversity positively correlated in non-grazed grasslands (P < 0.001), but not in grazed grasslands, indicating that grazing decoupled the biodiversity linkage. The decoupling was attributed to the change of the principal driver of soil microbiota from plant variables in non-grazed grasslands to soil variables in grazed grasslands. The diversity decoupling was also characterized by reduced plant-microbe interactions and lower heterogeneity in grazed grasslands. Our findings provide significant insights into the cross-biota linkages in grassland ecosystems.