We sought to generate a preliminary demographic framework for Psammochloa villosa to support of future studies of this ecologically important desert grass species, its conservation, and sustainable utilization. Psammochloa villosa occurs in the Inner Mongolian Plateau where it is frequently the dominant species and is involved in sand stabilization and wind breaking. Here, we characterized the genetic diversity and structure of 210 individuals from 43 natural populations of P. villosa using amplified fragment length polymorphism (AFLP) markers. We obtained 1728 well-defined amplified bands from eight pairs of primers, of which 1654 bands (95.72%) were polymorphic.All these values indicate that there is abundant genetic diversity, but limited gene flow in P. villosa. However, an analysis of molecular variance (AMOVA) showed that genetic variation mainly exists within 43 populations of the species (64.16%), and we found that the most genetically similar populations were often not geographically adjacent. Thus, this suggests that the mechanisms of gene flow are surprisingly complex in the species and may occur over long distances. In addition, we predicted the distribution dynamics of P. villosa based on the spatial distribution modeling and found that its range has contracted continuously since the last inter-glacial period. We speculate that dry, cold climates have been critical in determining the geographic distribution of P. villosa during the Quaternary period. Our study provides new insights into the population genetics and evolutionary history of P. villosa in the Inner Mongolian Plateau, which can be used to design in-situ conservation actions and to prioritize sustainable utilization of germplasm resources.