Numerous specimens stored in natural history collections have been involuntarily preserved together with their associated microbiomes. We propose exploiting centuries-old soils occasionally found on the roots of herbarium plants to assess the diversity of ancient soil microbial communities originally associated with these plants. We validated this approach extracting and sequencing DNA from rhizospheric soils and roots of four plant species preserved in herbaria for more than 120 years. Extracted DNA displayed typical features of ancient DNA, with cytosine deamination at the ends of fragments predominantly shorter than 50 bp. When compared to extant microbiomes, herbarium microbial communities clustered with soil communities and were distinct from communities from other environments. Herbarium communities also displayed biodiversity features and assembly rules typical of soil and plant-associated ones. Soil communities were richer than root-associated ones with which they shared most taxa. Regarding community turnover, we detected collection site, soil versus root, and also plant species effects. Eukaryotic taxa that displayed a higher abundance in roots were mostly plant pathogens or obligate symbionts that were not identified among soil-enriched ones. Conservation of these biodiversity features and assembly rules in herbarium-associated microbial communities indicates that herbarium-extracted DNA could reflect the composition of the original plant-associated microbial communities and that preservation in herbaria seemingly did not alter these characteristics. Through the use of this approach, it should be possible to investigate historical soils and herbarium plant roots to explore the diversity and temporal dynamics of soil microbial communities.