In principle, populations with higher genetic diversity and larger effective sizes have greater evolutionary capacity (i.e., higher adaptive potential) to respond to ecological stressors. Small isolated populations tend to lose genetic diversity rapidly due to the detrimental effects of drift and inbreeding via genetic erosion. We are interested in how adaptive potential persists in small populations and how it fluctuates relative to overall genomic diversity. We analyzed individual whole genome sequences from different populations of Montezuma Quail (Cyrtonyx montezumae), a small ground-dwelling bird that is sustainably harvested in some portions of its range but is of conservation concern elsewhere. Our results indicate that overall, Montezuma Quail populations in the U.S. exhibit low levels of genomic diversity due in large part to long-term declines in effective population sizes over nearly a million years. The smaller and more isolated Texas population is significantly more inbred than the large Arizona and the intermediate-sized New Mexico populations. The Texas gene pool has a significantly lower proportion of deleterious alleles than the Arizona gene pool, but also significantly more high-frequency deleterious alleles that, coupled with elevated inbreeding, elevate the realized genetic load in Texas. Our results highlight that smaller, isolated populations are at higher risk of inbreeding depression as detrimental mutations rise in frequency due to drift and weakened purifying selection. Our study illustrates how population genomics can be used to proactively assess both neutral and adaptive aspects of contemporary genetic diversity in a conservation framework while simultaneously considering deeper demographic histories.