With continued global change, recovery of species listed under the Endangered Species Act is increasingly challenging. One rare success was the recovery and delisting of the Channel Island fox (Urocyon littoralis) after 90-99% population declines in the 1990s. While their demographic recovery was dramatic, less is known about their genetic recovery. To address genetic changes we conducted the first multi-individual and population-level direct genetic comparison of samples collected before and after the recent bottlenecks. Using whole exome sequencing, we found that already genetically depauperate populations were further degraded by the 1990s declines and remain low, particularly on San Miguel Island which underwent one of the most severe bottlenecks. The three other islands that experienced recent bottlenecks (Santa Rosa, Santa Cruz, and Santa Catalina islands) showed mixed results based on multiple metrics of genetic diversity. Previous island fox genomics studies showed low genetic diversity before the declines and no change after the demographic recovery, thus this is the first study to show a decrease in genetic diversity over time in U. littoralis. Additionally, we found that divergence between populations consistently increased over time, complicating prospects for using inter-island translocation as a conservation tool. The Santa Catalina subspecies is now federally listed as threatened, yet other de-listed subspecies are still recovering genetic variation which may limit their ability to adapt to changing environmental conditions. This study further demonstrates that species conservation is more complex than population size and that some island fox populations are not yet “out of the woods”.