Population recovery following environmental stress is known to depend on demographic structure, life-history and evolutionary dynamics. However, it is unclear how evolution of traits shaped by sexual selection affect population dynamics and recovery. We examined this by manipulating presence/absence of males expressing either a non-aggressive “scrambler” phenotype or an aggressive and lethally armed “fighter” phenotype in soil mite populations of different size. We experimentally altered the male phenotype in populations, subjected them to heat stress, and analyzed their population dynamics and recovery. We show that populations with fighter males exhibited i) reduced population size and stability, ii) greater decline in response to heat stress in larger populations, iii) higher rate of growth, and iv) incomplete population recovery. Such reduced population stability and recovery linked with armed and aggressive phenotypes underlines the importance of sexual selection in mediating population dynamics and resilience to environmental change with implications for managing natural populations.